Executive Summary

EXECUTIVE SUMMARY

In Focus 2018: Campaign Evaluations in West Virginia, Illinois, and Nevada

DATE PRESENTED TO: PRESENTED BY: May 14, 2019 Susan Thompson Buffett Foundation The Public Affairs and Media Research Department and The Social Data Collaboratory

NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Table of Contents

Overview of Study Objectives and Approach ...... 4 Campaign Monitoring and Survey Research ...... 5 Twitter Analysis ...... 6

Key Findings ...... 7

Survey and Campaign Monitoring: Summary of Findings ...... 9 Research Question 1 ...... 10 Similarities across states ...... 10 Differences across states ...... 11 West Virginia ...... 11 Illinois ...... 11 Nevada ...... 12 Research Question 2 ...... 12 Similarities across states ...... 12 Differences across states ...... 15 West Virginia ...... 15 Illinois ...... 16 Nevada ...... 16 Research Question 3 ...... 16 Similarities across states ...... 16 Differences across states ...... 17 West Virginia ...... 17 Illinois ...... 17 Nevada ...... 17 Research Question 4 ...... 18

Twitter: Summary of Findings ...... 19 Twitter: West Virginia Findings ...... 19 Twitter: Illinois Findings ...... 20 Twitter: Nevada Findings ...... 21 Twitter: Overall Abortion Narrative ...... 21 Overview of Twitter Data – Abortion Topic – Illinois, Nevada, West Virginia...... 22 Characterizing Message Source ...... 25 Amount of Tweets by Source ...... 25 Average Twitter Activity by Source ...... 26 Engagement Overall and by State ...... 27

EXECUTIVE SUMMARY | I NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Twitter Messaging Source: Takeaways ...... 27 Twitter: Characterizing Message Sentiment ...... 27 Twitter Message Sentiment: Takeaways ...... 28 Twitter: Sentiment over time ...... 28 Twitter Message Sentiment over Time: Takeaways ...... 29 Twitter: State-level Trends in Pro-choice Versus Pro-life Sentiment ...... 30 Twitter Message Sentiment by State over Time: Takeaways ...... 31 Twitter: Top National Hashtags ...... 32

Methodology...... 34 Qualitative Campaign Monitoring ...... 34 Collection and Analysis Processes ...... 34 Campaign and relevant organizations’ websites and Facebook pages ...... 35 Campaign and relevant organizations’ emails ...... 35 Debate transcripts ...... 36 Political advertisements ...... 36 Campaign, candidate and relevant organization tweets ...... 36 Media coverage ...... 37 Campaign contributions ...... 37 Twitter Analysis ...... 38 Data Collection ...... 38 Comprehensiveness of the keywords...... 40 Twitter: Sentiment Classification ...... 41 Source Coding ...... 42 Source Sample ...... 43 Bot Filter ...... 43 Top terms and hashtags by source ...... 44 Identifying West Virginia Amendment 1 Predictive Terms ...... 44 State-Level Surveys ...... 44 AmeriSpeak Sample ...... 44 TargetSmart Sample ...... 44 Field Period, Respondent Screening and Weighting ...... 47 Analysis ...... 49 Voter Validation ...... 49

Appendix ...... 50

EXECUTIVE SUMMARY | II NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

List of Exhibits

Percentage of candidates’ campaign content that was related to abortion ...... 9

Considerable inconsistency among voters in party identification and abortion attitudes ...... 10

Republican pro-choice voters were more likely to cross party lines than Democratic pro-life voters ...... 11

Abortion ranked lower on the issue priority list and was more important for Democrats in Nevada and Illinois ...... 13

Almost no difference between pro-choice and pro-life voters in candidate personality traits they value ...... 14

Abortion was a higher priority for pro-life than pro-choice voters in West Virginia and Illinois .... 15

No difference between pro-choice and pro-life voters in campaign engagement ...... 18

Weekly volume of abortion-related tweets in the United States ...... 22

Weekly volume of abortion-related tweets in Illinois, Nevada, and West Virginia ...... 22

Weekly volume of abortion-related tweets in Illinois, Nevada, and West Virginia (per 100,000 population) ...... 24

National and state-level trends of tweets on abortion (with Pearson correlations between the states and national level) ...... 24

Amount of abortion-related tweets by source type (among the tweets posted from Illinois, West Virginia, and Nevada)...... 26

Average number of abortion-related tweets per account by source type ...... 26

Engagement: amount of tweets among other individual/person that are retweeting, quoting, or replying to a verified or influential user ...... 27

Number of tweets categorized as having pro-choice sentiment, pro-life sentiment, or neither for each state ...... 28

Valence of state-specific abortion-relevant tweets at each week in the study period ...... 29

Valence of abortion-relevant tweets by state at each week in the study period ...... 31

Top 50 national hashtags (larger font indicates greater number of posts using that word as a hashtag) ...... 32

Example Recruitment Brochure: Illinois ...... 46

EXECUTIVE SUMMARY | III NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Overview of Study Objectives and Approach

In this study, The Susan Thompson Buffett Foundation funded NORC at the University of Chicago to comprehensively assess campaigns in three states in 2018. The research examines the conditions under which the issue of abortion can impact a variety of campaign and electoral outcomes. The goals of this study were to understand the campaign process, how the issue of abortion is framed within that context, and how that might affect election dynamics and outcomes. Empirical research to date shows mixed results on whether the content of campaigns can influence voters, especially in this era of deep political polarization.

As summarized in a recent meta-analysis of nearly 50 campaign field experiments, there is little to no evidence of candidate campaigns’ ability to persuade voters (neither undecided voters nor supporters of the other candidate) to vote for their candidate in general election campaigns. However, it would be short- sighted to say that campaigns do not matter. The research assessed in the meta-analysis was focused on a particular impact of campaigns—changing minds in general elections. While this is arguably an important outcome, it would be a mistake to generalize the research too broadly. Other empirical studies show that campaigns do matter in a variety of contexts.

First, campaigns are dynamic and candidates are strategic. Candidates and their campaigns are successful at changing the focus of the debate in ways that differentiate them from the other candidate and align them with majority segments of the electorate. In other words, campaigns can elevate or depress the salience of issues that the public is considering as they make decisions about whether to vote and for whom they will vote. If abortion is seen by a candidate as a salient issue likely to raise engagement from their base, it can become a strategic issue for the campaign. The success of a candidate in competitive races often depends on the ability of campaigns to convert supporters to voters by identifying, energizing, and getting their supporters to the polls. Assessments of campaign experiments focused on voter turnout do show that campaigns can be successful.

Second, candidates can persuade their own supporters to care about, and sometimes change, their attitudes on issues. And, as even the meta-analysis shows, campaigns can be successful in persuading undecided and opposition voters in both primary and ballot initiative campaigns. Other research has also found that campaigns can be effective in changing voters’ minds in some unique circumstances.

Given this research, we focused this study on three races to better understand the role of abortion in campaign dynamics in different political contexts. This enabled us to evaluate the following research questions:

1. Under what conditions does a candidate’s pro-choice or pro-life position, and communication around that position, help, hurt, or have no effect on their electoral chances? 2. How do ballot initiatives on the issue of abortion interact with candidates’ campaigns? 3. What motivates or demotivates pro-choice and pro-life voters? How does the issue of abortion specifically interact with other issues or partisanship for these voters?

EXECUTIVE SUMMARY | 4 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

4. Does the issue of abortion play a role in persuading voters, mobilizing them, both, or neither? If it impacts persuasion or mobilization, which voters are most affected? To answer these questions, we pursued a multi-method research design in three states. A comprehensive description of the study methods appears in the final section of this executive summary. The states included are: 1) West Virginia, where there was a highly competitive U.S. Senate election with two pro- life candidates and a ballot initiative to restrict access to and funding for abortion, 2) Illinois, where there was a governor’s race with two pro-choice candidates, and 3) Nevada, where there was a highly competitive U.S. Senate election with a pro-life Republican and a pro-choice Democrat.

Race Candidate Vote share Party Abortion Stance Illinois: Governor Winner J.B Pritzker 54.5 Democrat Pro-choice Bruce Rauner 38.8 Republican Pro-choice Nevada: Senate Winner Jacky Rosen 50.4 Democrat Pro-choice Dean Heller 45.4 Republican Pro-life West Virginia: Senate Winner Joe Manchin 49.6 Democrat Pro-life Patrick Morrisey 46.3 Republican Pro-life West Virginia Amendment 1

Approved Yes 51.7 - Pro-life stance No 48.3 - Pro-choice stance

Campaign Monitoring and Survey Research

In each state, we conducted three waves of surveys during the 2018 midterm election campaigns. The first was conducted at the beginning of the midterm election campaign cycle in late August to early September. The second was conducted in the final week before the November 6 election. And the third was conducted shortly after the general election. These three surveys in each state allowed us to evaluate how opinions about and support for the candidates and the ballot initiative changed throughout the election cycle.

We also gathered extensive information about the campaign context to enable us to contextualize the public opinion data and to understand any changes or patterns in the survey data. The data for this part of the project included the content of websites, emails, and social media posts from the campaigns and relevant organizations, general Twitter conversation about the campaigns, political advertisements, debate transcripts, and newspaper coverage of the election within each state.

With this comprehensive evaluation of the election in three states, we are able to evaluate the research questions outlined above as well as recommend avenues for further research.

EXECUTIVE SUMMARY | 5 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Twitter Analysis

In addition to the survey and campaign monitoring research, NORC’s Social Data Collaboratory conducted a complementary analysis of abortion-related content on Twitter, spanning a five-month period from July 1–December 10, 2018. The broad goal of the work was to explore and understand the nature of public and political discourse around abortion prior to the 2018 midterm elections. We examined the general conversation about abortion, unrestricted by geography, and also aimed to assess whether and to what extent political campaigns and candidates’ positions on abortion in the states of West Virginia, Illinois, and Nevada had an energizing, neutral, or de-energizing impact on voting behavior.

We aimed to map our approach to the key research questions, data collection and analyses conducted by NORC’s survey research team, providing a detailed description of the campaign context with a focus (narratively and visually) on the timeline of key events with the potential to impact attitudes and other outcomes being measured. To accomplish this, we analyzed the number, sources, content, and diffusion of abortion-related messaging overall and in the three sentinel states. We conducted a comprehensive social media inventory of all messages posted by accounts from major candidates for governor or senate and containing Twitter handles of those candidates, ballot organizations, advocacy groups, and PACs supporting either side to insure that all major voices were included in our database and analysis. We used a combination of supervised and unsupervised machine learning to extract themes and user types from the retrieved data.

Social media is a key component of modern political discourse analysis. It has emerged as an important place for stakeholders, advocates, and regular people to express their opinions and react to events both within their community and in the context of a larger conversation. Political candidates use Twitter to make public announcements, interact with the public and the media, and promote awareness of their policy positions. One of the key functions of Twitter is that it also supports two-way communication: advocacy organizations and regular people can engage with each other about political topics, as well as with candidates and campaigns. This engagement between candidates, stakeholders, and the general public represents a dynamic that was rare before the age of social media.

From its inception, Twitter has been characterized as a public sphere,1 where debate and discourse can flow freely. At its best, Twitter can empower voters by enhancing deliberative democracy as users refine their own views, gain exposure to different opinions, and identify common means and ends. However, research has also shown that online discussions can amplify division among social groups holding different views, rather than forming consensus among them.2 Further, Twitter’s algorithms and features, such as its hashtags and retweet functions, influence the discourse itself. Indeed, these algorithms determine the flow of information, enabling prominent topics and content posted by prominent users to achieve higher exposure and engagement.3 Hashtags serve as a strategic communication device, enabling

1 Fuchs C. 2014. Twitter and democracy: a new public sphere? In Social Media: A Critical Introduction, pp. 176-209. London: Sage. 2 Yardi, S., and Boyd, D. 2010. Dynamic debates: an analysis of group polarization over time on Twitter. Bulletin of Science, Technology & Society, 30(5), 316–327. 3 http://www.slate.com/articles/technology/cover_story/2017/03/twitter_s_timeline_algorithm_and_its_effect_on_us_explained.ht ml

EXECUTIVE SUMMARY | 6 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA individuals to find content that interests them, join a larger conversation, and amplify their own voice. Indeed, research has shown that posts that feature hashtags achieve greater influence compared to untagged posts.4

Social media research does not represent public opinion. In fact, in recent years, numerous studies have documented differences between Twitter users and the general public. Rather, the advantage of using Twitter as a source of information is that it functions as a platform for agenda-setting by groups often classified as “elites,” including journalists and politicians. At the same time, it also supports engaged activism: expression by the most opinionated members of the public,5 who are also more likely to vote, compared with the general public.6 Twitter does not fully represent the average citizen’s opinions. However, what it does represent extends beyond messages from the traditional elite to include how citizen influencers and the engaged public react to those messages and how their opinions and disagreements are framed in the public sphere. In this way it affords users (and researchers) exposure to voices and stakeholders who are not part of the traditional elite.7

Whereas survey research can provide detailed insight into the attitudes and beliefs held by individuals in a representative sample of the study population, social media analysis can provide important contextual information about the amount and content of messages posted by agenda-setters and other stakeholders: how and to what extent are these actors influencing and reflecting the opinions of the general public?

In addition, analysis of Twitter conversations can place a regional discourse in the context of a broader national or international conversation—identifying key players outside of a particular geographic community. Finally, analysis of Twitter discourse can uncover unexpected actors, ideas, and language used by both sides of a political issue.8 Thus, Twitter analysis can help inform the interpretation of survey-based public opinion research, offer insights about campaign strategy, reveal key stakeholders, and suggest opportunities and potential risks for developing public engagement and issue-oriented strategies.

Key Findings

Although the social media conversation is not representative of public opinion, the abortion discourse on Twitter provided some confirmation of and context for the survey findings. The surveys found that overall, abortion was not a strongly motivating issue in the election cycle and not among the most important issues in most people’s voting behavior. The Kavanaugh Supreme Court justice nomination and

4 Davis, Bud. 2013. “Hashtag politics: the polyphonic revolution of #Twitter.” Pepperdine Journal of Communication Research: Vol. 1, Article 4. Available at: http://digitalcommons.pepperdine.edu/pjcr/vol1/iss1/4. 5 https://www.nytimes.com/interactive/2019/04/08/upshot/democratic-electorate-twitter-real-life.html 6 https://www.pewinternet.org/2019/04/24/sizing-up-twitter-users/ 7 Freelon D, Karpf D. 2015. Of big birds and bayonets: hybrid Twitter interactivity in the 2012 presidential debates. Information, Communication & Society, 18: 390–406. Papacharissi Z. 2013. A networked self: identity performance and sociability on social network sites. In Frontiers in New Media Research, eds. F Lee et al., pp. 207–21. New York, NY: Routledge. 8 Sedereviciute K, Valentini C. 2011. Towards a more holistic stakeholder analysis approach. Mapping known and undiscovered stakeholders from social media. International Journal of Strategic Communication, 5: 221–39.

EXECUTIVE SUMMARY | 7 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA confirmation hearings drove much of the Twitter discourse across all three states, and election-related Tweets largely avoided the issue of abortion. Analyzing the ratio of pro-choice to pro-life tweets across the three states showed that the Twitter discourse largely mirrored abortion sentiment in those states.

There was little evidence that social media influenced individual voting behavior, but strong evidence that it reflected elite viewpoints about abortion and revealed ways in which the elite groups engage with the public. Generally, the abortion discourse on Twitter involved expected voices: advocacy groups, celebrities, political parties and candidates, and regular people who hold a strong opinion about abortion politics.

The unique contribution of the Twitter analysis was to provide context, identify key stakeholders, and uncover rhetoric used in the political discourse about abortion overall and in each of the three states. Our analyses confirmed that the ballot initiative added a unique dimension to the state-level conversations on Twitter. In West Virginia and Nevada, where the political races were more highly contested, celebrity accounts weighed in on the issue of abortion and received significant exposure and engagement. The biggest missing piece of information in our Twitter analysis was whether and how abortion co-occurred with other issues that motivate voters, such as gun control and immigration. The surveys, however, provide several insights into the relative position of abortion among other key election issues. We also did not look for misinformation overall or by user type.

Abortion did play a significant role in the Twitter conversation in West Virginia; the per capita rate of tweeting about abortion was significantly higher in West Virginia than in the other states or overall. The ballot initiative accounted for a significant portion of the abortion discourse on Twitter in West Virginia, and many of the messages reflected confusion over what the Amendment was about. Similarly, survey respondents indicated confusion about the ballot initiative. Tweets posted by candidates, political and advocacy organizations in West Virginia leaned strongly pro-life and Tweets about Amendment 1 were largely supportive. While the surveys found that pro-life voters were motivated when messages were framed as moral, religious, or personal choice, the rhetoric of tax-payer responsibility was commonly used in pro-life advocacy for Amendment 1 on Twitter.

The abortion-related Twitter discourse in Illinois was relatively contained and largely reflected the national discourse, following the trends in volume around Kavanaugh’s nomination and confirmation. In a state where most voters indicated a pro-choice stance and where both major candidates were pro-choice, abortion was simply not a priority topic in the context of the political campaigns. Those posts which were campaign-related largely focused on a vocal pro-life minority trying to elevate the issue, with very limited effect. These findings are consistent with survey results showing that abortion was not one of the top issues motivating Illinoisans to vote.

The abortion discourse on Twitter in Nevada reflected that the Democratic Senate candidate was a vocal pro-choice candidate on Twitter. While her campaign messaging on broadcast ads and in debates largely avoided the topic of abortion, her Twitter messages highlighted her pro-choice stance and achieved high exposure and engagement. This finding supports the survey results that showed the Democratic Senate candidate was able to garner more support from opponents even though she was pro-choice, and voters more accurately identified her position.

EXECUTIVE SUMMARY | 8 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Survey and Campaign Monitoring: Summary of Findings

Coupled with the Twitter analysis, the survey and campaign monitoring research provide a wealth of data to answer the key research questions of interest. While several facets of the data provide insights across several questions, the findings below are what we consider the high-level results. Each case study section walks through the findings in further detail.

One task of this research was to determine the role abortion plays in campaigns with differing contexts. Both the Twitter analysis and campaign monitoring analysis show that abortion was most salient in West Virginia, and least salient in Illinois. In the Nevada Senate race, the Democratic Senate candidate brought up the issue more than the Republican incumbent and embraced the issue directly.

Percentage of candidates’ campaign content that was related to abortion

Illinois Nevada West Virginia Democrat % Republican% Democrat% Republican Democrat % Republican % % Ads 4 - - - - 21 Twitter <1 - 2 - <1 5 Web content 10 - 6 5 - 23 Debates - - - - 5 6 Emails 2 - 7 1 - <1 Funding <1 - 1 <1 <1 - Articles 3 2 4 8 13 10 Broadcast 7 4 1 1 2 2

The West Virginia ballot measure drove a lot of the Twitter traffic coming from pro-choice messengers early in the campaign, and late in the campaign, pro-life conversations dominated. At the same time, the U.S. Senate race featured two pro-life candidates, however, the Republican candidate was much more likely to discuss the topic of abortion than the Democrat. For his part, the Democratic incumbent reiterated his pro-life position, but at the same time opposed the ballot measure late in the campaign, lining up with pro-choice rather than pro-life interests. This campaign dynamic shaped voters’ level of knowledge about the issue as it relates to the campaign. Voters were actually not very familiar with candidates’ issue positions generally, and abortion was one of the least familiar issues.

A majority of voters in West Virginia considered themselves pro-life, while majorities in Illinois and Nevada considered themselves pro-choice. Across parties, identification as pro-choice or pro-life varied by state. This context is important for understanding the detailed survey findings that follow.

EXECUTIVE SUMMARY | 9 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Considerable inconsistency among voters in party identification and abortion attitudes

West Virginia Illinois Nevada % Pro- % Pro- % Pro- % Pro- % Pro- % Pro- choice life choice life choice life Overall 44 55 64 35 66 34 Democrats 67 32 83 17 84 16 Independents 47 51 61 38 70 30 Republicans 25 75 36 63 44 56

The Illinois gubernatorial election featured two pro-choice candidates, yet neither focused on the issue of abortion much at all. In both West Virginia and Illinois, voters were confused about candidates’ positions on abortion when they didn’t line up with traditional party opinions. Most voters thought that Democrat Senate incumbent was pro-choice despite his pro-life stance, and most viewed the incumbent Republican Governor of Illinois as pro-life rather than pro-choice.

In Nevada, a competitive race for Senate resulted in a Democratic win, all while the Democrat embraced the issue of abortion directly. The Twitter analysis shows that the Democrat was one of the top communicators about the issue during the campaign, retweeting interest groups’ content and calling herself pro-choice.

It is under this context that we observe several key findings related to the research questions below. It is important to note that across the research questions highlighted, the findings hold up when examining respondents determined to have actually voted in the election, based on a voter match.9

Research Question 1 Under what conditions does a pro-choice or pro-life position, and communication around that position, from a candidate help, hurt, or have no effect on their electoral chances?

Similarities across states

■ Context matters for knowledge on issue positions when candidates hold the same views and when one holds a position contrary to the majority of their party. In Illinois and West Virginia, majorities could not accurately describe the candidate with views discordant with their party’s conventional view on abortion.

■ Crossover voting was more likely for Republican pro-choice voters than Democratic pro-life voters. Democratic candidates received more support from pro-life Democrats and independents than Republican candidates received from pro-choice Republicans. And Democratic candidates also did better among independent voters. Democratic candidates were more likely to get the support of pro- choice independents than Republicans were to get the support of pro-life independents.

9 See methodology section for voter match procedure.

EXECUTIVE SUMMARY | 10 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Republican pro-choice voters were more likely to cross party lines than Democratic pro-life voters

Illinois Nevada West Virginia % of __ voting for__ Pritzker Rauner Rosen Heller Manchin Morrisey Democrats, Pro-choice 88 8 94 3 91 5 Democrats, Pro-life 77 17 88 7 82 16 Indeps, Pro-choice 48 35 60 28 68 20 Indeps, Pro-life 40 41 32 46 39 49 Republicans, Pro-choice 20 74 16 78 40 54 Republicans, Pro-life 7 81 5 92 19 76

Question: With respect to the abortion issue, would you consider yourself to be pro-choice or pro-life?

Source: In Focus Wave 3 poll conducted by NORC at the University of Chicago, November 7-November 21, 2018, with 2,525 registered voters in Illinois, 2,049 registered voters in Nevada, and 2,831 registered voters in West Virginia

Differences across states

■ Despite relatively low familiarity with candidates’ positions on the issue of abortion observed across the campaigns, Rosen was able to garner more support from opponents. Rosen did so as a pro-choice candidate, with most voters accurately identifing her position.

■ Voters were more confused in West Virginia and Illinois, where candidates aligned on the issue of abortion despite being from opposing parties.

■ In West Virginia and Nevada, the Democratic candidates were considered better able to handle the issue of abortion, while in Illinois the candidates were rated similarly.

West Virginia

■ Manchin discussed his position on abortion far less than Morrisey. Manchin also held a position inconsistent with the majority of his party. Voters were confused about Manchin’s position, with most viewing him as pro-choice rather than pro-life even before he came out against the ballot measure. Within this context, Manchin was still considered better able to handle the issue of abortion than Morrisey.

■ Likely driven by his incumbency advantage, Manchin received a larger share of pro-life voters than Morrisey received of the pro-choice vote, as well as a larger share of Republicans than Morrisey received of the Democratic vote, which pushed Manchin to victory in a tight election. He also earned the support or more independents.

Illinois

■ Very little information was communicated by either campaign in the race on the issue of abortion. Pritzker got support, however, from more pro-life voters than Rauner did from pro-choice voters. This occurred even though there was substantial confusion around Rauner’s position—most thought he was pro-life.

EXECUTIVE SUMMARY | 11 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

■ At the same time, abortion ranked lower as a salient issue to vote choice when compared to other issues, so there is a lack of evidence that this issue would have been pivotal in this race—especially given the wide spread in the election result.

Nevada

■ While neither candidate in the Nevada race focused on the issue significantly, Rosen did so more than Heller. At the same time, she was able to consolidate more pro-choice independents and Republicans than Heller was able to obtain from pro-life Democrats or independents. This finding holds up among voters determined to have actually voted in the election.10

■ Voter contact mattered a lot for Rosen. Those who had any form of political or other organizational contact during the campaign were more likely to vote for her, as well as understand her position on abortion.11

■ Despite being pro-choice, over a quarter of Republicans said Rosen would do a better job on abortion than Heller. And more Democrats thought Rosen could better handle the issue than did Republicans for Heller.

■ Despite being contextually different, the overall implications of the role of abortion in Illinois and Nevada were quite similar.

Research Question 2 What motivates or demotivates pro-choice and pro-life voters? How does the issue of abortion specifically interact with other issues or partisanship for these voters?

Similarities across states

■ Across states, abortion was predictive of vote choice even when controlling for partisanship and other demographics. After controlling for these other factors, individuals’ who were pro-choice were more likely to vote for Democrats. The Democratic candidate in each of these elections won the race. 12 However, other issues including immigration and gun control/gun rights were also associated with vote choice and, in some cases, had a larger effect in the model than abortion. In all cases, partisanship is more predictive of vote choice than issue attitudes.

■ In each state, abortion ranked relatively low on issue importance, and there are few single issue voters.

10 As part of the research study, NORC conducted a voter match to validate which survey respondents actually voted in the midterm election based on official records. NORC used a vendor to conduct this analysis. See the Methodology section for a full description of the procedure. 11 Among actual voters, the latter finding is not statistically significant. 12 See multivariate models from Appendix.

EXECUTIVE SUMMARY | 12 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Abortion ranked lower on the issue priority list and was more important for Democrats in Nevada and Illinois

61 (#8 of 9) Nevada 48 (#8) 41 (#8)

48 (#9) Illinois 43 (#9) 41 (#7)

59 (#7) West Virginia 60 (#6) 65 (#7)

0 10 20 30 40 50 60 70 80 90 100 Percent extremely/very (rank among nine issues asked about)

Democrats Independents Republicans

Question: How important were each of the following issues to you in deciding how to vote in this year’s election for [governor/senator] between…? …Abortion

■ Motivations for being pro-choice or pro-life were nearly identical across states, and opinions about the roles of gender and race in society also reflect clear patterns.

► Pro-life voters were motivated when the issue was framed as moral, religious, and a personal health issue.

► Pro-choice voters were motivated when an issue was framed as one of freedom, an individual’s personal health, and privacy.

► Voters across states also ranked candidate traits important to their vote choice similarly, with alignment on issues between the voter and a candidate among the most important, as well as the candidates’ values.

EXECUTIVE SUMMARY | 13 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Almost no difference between pro-choice and pro-life voters in candidate personality traits they value

Nevada 88 Honest 89 89 Integrity 88 77 Cares about people 84 86 Moral 84

Illinois 89 Honest 87 90 Integrity 85 76 Cares about people 84 83 Moral 80

West Virginia 85 Honest 83 82 Integrity 82 78 Cares about people 84 81 Moral 76 0 10 20 30 40 50 60 70 80 90 100 Percent extremely/very important Pro-life Pro-choice

Question: Thinking about your vote for [governor/Senate], how important was each of the following when deciding who to vote for? That the candidate cares about people that need help, That the candidate was honest, That the candidate is moral, That the candidate has integrity

EXECUTIVE SUMMARY | 14 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Differences across states

■ Importance of abortion as an issue varies by state and context among pro-choice and pro-life voters. It was most important for pro-life voters in West Virginia.

Abortion was a higher priority for pro-life than pro-choice voters in West Virginia and Illinois

70 West Virginia 52

52 Illinois 40

50 Nevada 51

0 10 20 30 40 50 60 70 80 90 100

Percent extremely/very important

Pro-life Pro-Choice

Question: How important were each of the following issues to you in deciding how to vote in this year’s election for [governor/senator] between…? …Abortion

West Virginia

■ In an election where both candidates claimed the same position, abortion position was a significant predictor of vote choice, but so were other factors, such as partisanship, position on gun rights or gun control, immigration, and the economy. These other factors are more predictive of vote choice than is position on abortion.13

■ Pro-life voters are motivated by the issue when was framed as a moral, religious, and personal health issue. Pro-life voters also preferred candidates who are honest, have integrity, and seen as moral.

■ Pro-choice voters were motivated by the issue when it was framed as an issue of freedom, personal health, and privacy. Pro-choice voters were motivated by similar candidate traits as pro-life voters; they were slightly more likely to prefer a candidate who cares about people who need help.

13 See multivariate models from Appendix.

EXECUTIVE SUMMARY | 15 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Illinois

■ Being pro-choice or pro-life was a significant indicator of one’s vote choice, but similar to the other states, it’s not the only predictor. Partisanship and issue position on guns, immigration, and the economy mattered more.14

■ Voters were also more polarized in their responses about whether guns and immigration were critical to their vote choice—while sizable proportions did consider abortion, there are not significant disparities between pro-choice and pro-life voters in considering it.

■ Pro-choice and pro-life voters alike preferred candidates who share their positions on the issues generally and on those most important to them, as well as their values.

■ Pro-life voters were motivated when the issue is framed as moral, religious, and a personal health issue.

■ Pro-choice voters were motivated when an issue is framed as one of freedom, an individual’s personal health, and privacy.

Nevada

■ Healthcare was a major mobilizing factor for pro-choice voters and Democrats in Nevada. For Republicans and pro-choice voters, the importance was placed on the economy.

■ Being pro-choice or pro-life is a significant indicator of one’s vote choice, but similar to the other states, it’s not the only predictor. Partisanship and issue position on guns, immigration, and the economy mattered more.15

■ Similar to the other states, voters were also more polarized in their responses about whether guns and immigration were critical to their vote choice. Pro-choice and pro-life voters placed similar levels of importance on the issue of abortion when voting.

■ Pro-choice and pro-life voters alike preferred candidates who share their positions on the issues generally and on those most important to them, as well as their values.

■ Pro-life voters are motivated when the issue is framed as moral, religious, and a personal health issue.

■ Pro-choice voters are motivated when an issue is framed as one of freedom, an individual’s personal health, and privacy.

Research Question 3 Does the issue of abortion play a role in persuading voters, mobilizing them, both, or neither? If it impacts persuasion or mobilization, which voters are most affected?

Similarities across states

■ Voter turnout and campaign participation/engagement is similar between pro-life and pro-choice voters despite varying election contexts and issue positions of candidates.

14 See multivariate models from Appendix. 15 See multivariate models from Appendix.

EXECUTIVE SUMMARY | 16 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

■ In every candidate race, pro-choice Republicans were more likely to support the Democratic candidate than pro-life Democrats were to support the Republican candidate.

Differences across states

■ Respondents across states heard relatively less about the issue of abortion than other issues. Yet voters who heard more information about the issue were more likely to vote for the Democratic candidates in Illinois and Nevada, and the Republican candidate in West Virginia.

West Virginia

■ Abortion did not impact propensity to vote in any state, including West Virginia. Regardless of issue position, voters were equally likely to participate in the election even when pro-choice voters didn’t have a candidate aligned with their position on the issue.

■ Voters across issue positions were also equally likely to be interested in the election and follow the news about it.

■ Sizable proportions of voters aligned themselves with abortion opinions that are discordant with their party, and these voters are more moveable on the issue. A quarter of Republicans identified as pro- choice and a third of Democrats identified as pro-life.

■ There is no direct evidence that the issue of abortion mobilized voters to engage more or less in the campaign.

Illinois

■ Illinois voters across issue positions were equally motivated to participate in the election.

■ The issue of abortion itself did not impact reported voter turnout.

■ Voters were also equally likely to say they were interested in the election and paid attention to news about it, across issue positions.

■ Pro-choice Republicans tended to be upper-income, older, white, or married. Pro-life Democrats tended to be black, Hispanic, or lower-income.

Nevada

■ Nevada voters across issue position were equally motivated to participate in the election, and did so at a similar rate, with over 90 percent each engaging in the campaign either actively or passively.

■ As in the other states, abortion is not a “single issue to vote on” for many people. Voters do label it as very important, but not as important as some of the other issues tested in the survey.

■ The issue of abortion itself did not impact reported voter turnout.

■ Voters were also equally likely to say they were interested in the election and paid attention to news about it, across abortion issue positions.

■ Pro-choice Republicans tended to be upper-income, older, white, have no religion or identify as Protestant, or be married. Pro-life Democrats tended to be black, Hispanic, or lower-income.

EXECUTIVE SUMMARY | 17 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Research Question 4 How do ballot initiatives on the issue of abortion interact with candidates’ campaigns?

■ There was confusion over wording and what the ballot measure would do. Confusion was highest among those who voted for the measure.

■ Candidate endorsements did not play a significant role in support for the ballot measure, though they mattered for some key demographic groups, like lower-income voters and women.

■ Despite the ballot measure being solely about abortion, the candidates did not spend relatively much time discussing the issue during the campaign, and the issue was not a top priority for voters.

■ Voter contact mattered significantly on the ballot measure. Those who were contacted by a political, organization or PAC, or a civil society group about the ballot measure were more likely to vote for the measure, understsand it, and accurately say what it would do. However, when looking just at pro- choice respondents, contact did not impact propensity to vote for Amendment 1.

■ Campaign engagement—particularly passive engagement—was similar in the ballot campaign when compared to the candidate campaigns in Illinois and Nevada. However, in the latter two states, people were more likely to talk about the campaign with friends and family.

No difference between pro-choice and pro-life voters in campaign engagement16

Nevada Talked to family or 76 friends 73 Wore buttons or 52 watched debates 51

Illinois Talked to family or 78 friends 79 Wore buttons or 56 watched debates 56

West Virginia Talked to family or 68 friends 66 Wore buttons or 41 watched debates 45

0 10 20 30 40 50 60 70 80 90 100 Percent once/more than once Pro-life Pro-Choice

Question: We find that many people participate in politics in other ways besides voting. For the November 6 election in [state], how many times did you do each of the following to support a candidate in the campaign…? If you [have not done/did not do] any of the items you may tell us that too. How many times did you talk with family or friends about the campaign…? If you did not do this you may tell us that too.

16 Note: West Virginia data refer to engagement on the Amendment 1 campaign.

EXECUTIVE SUMMARY | 18 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Twitter: Summary of Findings

Overall, the social media analyses showed that state-level discourse around abortion generally reflected the national discourse. The Kavanaugh nomination and confirmation accounted for major peaks in the overall conversation, nationally and in each state. Controlling for population size, the amount of Twitter discourse in each state reflected the contentiousness of the races. With both a senate race and the ballot initiative to restrict access to abortion, West Virginia generated the most abortion-related tweets per capita. While individuals (regular users) accounted for 95 percent of all tweets, elite accounts generally tweeted most actively and achieved the highest reach and engagement in each state. Indeed, retweets of elite’s messages account for much of the volume of tweets coming from regular people. This points to the important role of elites/influencers in shaping the political discourse. Nationally and across each state, sentiment on Twitter was mostly pro-choice as measured by tweet volume; if measured by reach, sentiment was more evenly divided. The volume of pro-life tweets increased steadily between September and November, and then dropped off, suggesting organized mobilization within the pro-life movement.

■ During the study timeframe from July 1, 2018, through December 10, 2018, overall 16,064,399 tweets were collected with an average of 669,350 per week.

■ During the study timeframe from July 1st, 2018 through December 10th, 2018, overall 16,064,399 tweets were collected with an average of 669,350 per week.

■ Illinois, Nevada, and West Virginia followed national trends with the exception of the West Virginia election. Other individuals/regular users accounted for 95 percent of tweets about abortion during the study period.

■ Celebrity accounts were the most active sub-group of users among the elite accounts that tweeted about abortion during study period.

■ Overall, pro-choice sentiment was more common than pro-life sentiment.

■ The top 10 most active accounts overall were mainly pro-life or right leaning.

■ While pro-choice tweets outnumbered pro-life tweets during most of the study period, by late October, pro-life tweets became more common than pro-choice. Following the election, pro-choice was again the dominant sentiment.

■ The top five national hashtags included “prolife” (239,743 tweets), “abortion” (95,548 tweets), “Kavanaugh” (91,753 tweets), “MAGA” (66,238 tweets), and “StopKavanaugh” (66,139 tweets).

Twitter: West Virginia Findings

Abortion was a heated topic in West Virginia leading up to the midterm election. Political campaigns actively tweeted about the issue, including messages about the national discourse, political candidates, and the ballot initiative. Pro-life messages dominated the discourse in West Virginia and achieved the greatest reach, due to the role of elite accounts. Bots played a larger role in the West Virginia discourse, compared to other states, and were used more to communicate a pro-life message. The pro-life movement

EXECUTIVE SUMMARY | 19 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA used the rhetoric of protecting taxpayers in its messaging supporting Amendment 1; the pro-choice movement used language about protecting women and providing health care.

■ Of the three states, West Virginia had the most campaign-related Twitter activity.

■ Engagement was high for the West Virginia campaign-related tweets, likely because they included tweets from celebrities like Donald Trump Jr., John Cusak, John Leguizamo, and Holly Figueroa O'Reilly.

■ Pro-choice groups tweeted much more than pro-life groups, but pro-life groups had a significantly higher proportion of tweets relevant to abortion.

■ Pro-life organizations and individuals were potentially more influential because those accounts had more followers.

■ Celebrities participated in the conversation in West Virginia, increasing the visibility of the abortion- related messages on Twitter.

■ Pro-life sentiment was dominant across most of the user-type categories in West Virginia, in a way that it was not in either of the other states.

■ Pro-choice sentiment was much higher among non-affiliated groups and organic users on Twitter, but the pro-life sentiment was higher among other elite user categories, leading to a high exposure of pro- life messages over pro-choice messages.

■ Bots were more likely to be used by and for pro-life groups, indicated by the sentiment and hashtags.

■ West Virginia Amendment 1 was a major topic of the abortion-related discussion in the state.

■ Posts about the Amendment often reflected incorrect information about the circumstances under which the new law would cover abortion services.

■ Support for the amendment was expressed in terms of protecting taxpayers, while opposition for the amendment used language about providing access to healthcare and protecting women.

Twitter: Illinois Findings

In contrast to West Virginia, abortion was not discussed much on Twitter in Illinois. Neither campaign mentioned abortion in its campaign messaging, nor were the major candidates mentioned in the discourse about abortion. Advocacy groups posted messages mainly related to the national discourse. Illinois Democratic organizations were more active than Republican groups, but pro-life and pro-choice groups posted at similar rates. Pro-life groups used more inflammatory language in their posts and achieved higher exposure because the accounts had more followers. Because both candidates were pro-choice and the race was not close, the pro-life activity on Twitter neither reflected nor influenced the overall discourse or voting behavior.

■ There was little conversation about abortion from either Democratic or Republican campaigns or the Illinois Democratic and Republican Parties. Also, Twitter mentions of each candidate contained little mention of abortion keywords.

■ Both Planned Parenthood Illinois Action and the Illinois Family Institute tweeted about abortion equally and their accounts had the most tweets out of any state-based advocacy organization.

EXECUTIVE SUMMARY | 20 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

■ Some smaller political players from Illinois tried to emphasize abortion, but the issue was not adopted by the major candidates.

■ The official Democratic Party account tweeted more overall than the Republican Party account, but neither tweeted significantly about abortion.

■ Pro-choice and pro-life groups tweeted at similar rates overall, and had similar proportions of tweets relevant to abortion.

■ Overall, pro-life and pro-choice groups sent out a similar number of relevant tweets, but pro-life groups used explicit language such as “murder” and “abort.”

■ Top influencer accounts were pro-life.

■ Bots generated a relatively small portion of tweets about abortion from Illinois.

■ Tweets by news organizations were relatively neutral.

■ Illinois advocates, other groups, and bots had a pro-life sentiment.

■ All other Illinois sources, including political accounts and regular people, were pro-choice.

Twitter: Nevada Findings

As in Illinois, the abortion-related conversation in Nevada tracked with the national discourse, with a focus on the Kavanaugh confirmation. In contrast to Illinois, the Democratic candidate for Senate took on abortion as a campaign issue and was vocal on Twitter about her pro-choice stance. Her tweets and tweets about her pro-choice position achieved high exposure and engagement. Republican voices were largely absent in the abortion-related discourse in Nevada. Regular people posted the majority of tweets about abortion in Nevada, and strongly leaned pro-choice. In Nevada, abortion rights was a rallying call for Democratic politicians, advocacy organizations, and pro-choice individuals.

■ While the Nevada Democratic Party, the Democratic candidate and her campaign account posted messages on abortion, the Republican Party and the Republican candidate avoided the issue entirely.

■ Tweets from the Democratic candidate campaign account were retweeted and mentioned the most during the data collection time period.

■ Celebrity account users were the most vocal group.

Twitter: Overall Abortion Narrative

During the study timeframe from July 1, 2018, through December 10, 2018, the total amount of abortion- related tweets was 16,064,399, with an average of 669,350 tweets per week.

EXECUTIVE SUMMARY | 21 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Weekly volume of abortion-related tweets in the United States

National Weekly Volume of Tweets

1800000

1600000

1400000

1200000

1000000

800000

600000

400000

200000

0 7/2/2018 8/2/2018 9/2/2018 10/2/2018 11/2/2018 12/2/2018

Overview of Twitter Data – Abortion Topic – Illinois, Nevada, West Virginia

Specifically for each state, the total amount of abortion-related tweets was 211,336 from Illinois, 59,508 from Nevada, and 24,433 from West Virginia, with weekly averages of 9,076, 2,556, and 1,049 respectively.

Weekly volume of abortion-related tweets in Illinois, Nevada, and West Virginia

Weekly Tweets

30000

25000

20000

15000

10000

5000

0 7/2/2018 8/2/2018 9/2/2018 10/2/2018 11/2/2018 12/2/2018

IL NV WV

EXECUTIVE SUMMARY | 22 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

When adjusted for population, West Virginia had a disproportionate amount of tweets about abortion. The number of abortion-related tweets per 100,000 people was 1,657 from Illinois, 2,226 from Nevada, and 2,525 from West Virginia.

EXECUTIVE SUMMARY | 23 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Weekly volume of abortion-related tweets in Illinois, Nevada, and West Virginia (per 100,000 population)

350

300

250

200

150

100

0 7/2/2018 8/2/2018 9/2/2018 10/2/2018 11/2/2018 12/2/2018

IL NV WV

National and state-level trends of tweets on abortion (with Pearson correlations between the states and national level)

USA

Illinois Pearson Correlations

(0.98)

Nevada (0.95)

West (0.66) Virginia

July 10: President Trump Sept 6: Oct 5: Nov 7: West Virginia nominates Kavanaugh for Kavanaugh Kavanaugh Amendment 1 passes SCOTUS hearing sworn in The trends in Illinois and Nevada are strongly correlated with the national trend, and the trend in West Virginia is moderately correlated with the national trend.

EXECUTIVE SUMMARY | 24 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Characterizing Message Source The American states have often been described as “laboratories of democracy” because of their ability to experiment with and adopt new policies. Building on this perception, prior research investigated how policy innovations are formulated, promoted, implemented, and spread across the states and jurisdictions. Recently, however, journalists have documented that interest groups exert considerable influence over policy adoption and emulation.17 This indicates a prominent role for interest groups in policy adoption, particularly at the state- and local- level.

Political and social institutions, as well as leaders of opinion on the abortion rights issue are widely present on social media platforms. Social media allow us to identify and prioritize these known and undiscovered stakeholders based on their online level of activity, engagement, and demonstrated concern with the issue (and to potentially engage the “movers and shakers”/influencers by directly communicating with them, bypassing the traditional mass media channels).18

Account holders posting about abortion-related issues were categorized as members of one of the following groups:19

■ news media

■ advocates

■ political organizations

■ politicians

■ celebrity

■ other individuals/regular people

Together, the news media, advocacy groups, political organizations, politicians, and celebrities are referred to as elite social media accounts throughout this report. We juxtapose this elite group from other individuals/regular people because of their cultural influence.

Amount of Tweets by Source The table below shows that within the larger group of elite accounts, celebrity users accounted for the largest subgroup of elites. Celebrity accounts included any users who indicated that they were actors, musicians, performers, writers, or entertainers, and who had more than 8,000 followers. Other individuals/regular users accounted for 95 percent of the tweets about abortion during the study period.

The second table also shows that advocates and celebrities were the most vocal elite groups on the issue of abortion rights nationally and across the three states, with celebrity users posting the largest number of tweets nationally (3,450 tweets), followed by advocacy groups (3,388 tweets). It is worth noting that, besides the other individuals (~95 percent), advocates (1.2 percent) posted the highest number of tweets

17 Greenblatt, Alan. 2011. “Right-minded.” Governing 25 (3): 32–36. 18 Kristina Sedereviciute and Chiara Valentini. 2011. Towards a more holistic stakeholder analysis approach. Mapping known and undiscovered stakeholders from social media. International Journal of Strategic Communication. 5:4, 221-239. 19 See methodology section for definitions.

EXECUTIVE SUMMARY | 25 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA in Illinois, celebrities (1.9 percent) tweeted the most about abortion in Nevada, and bots (2.1 percent) posted the highest number of tweets in West Virginia.

Amount of abortion-related tweets by source type (among the tweets posted from Illinois, West Virginia, and Nevada)

Source Type Total Illinois Nevada West Virginia News media 2,205 (0.7%) 1,643 (0.8%) 350 (0.5%) 212 (0.5%) Advocate/advocacy group 3,388 (1.0%) 2,470 (1.2%) 323 (0.5%) 615 (1.3%) Political organization 585 (0.2%) 116 (0.1%) 164 (0.2%) 305 (0.7%) Other group/organization 898 (0.3%) 741 (0.3%) 66 (0.1%) 92 (0.2%) Celebrity 3,450 (1.1%) 1,758 (0.8%) 1,288 (1.9%) 424 (0.9%) Politician 1,073 (0.3%) 766 (0.4%) 235 (0.4%) 77 (0.2%) Other individual/person 308,999 (95.4%) 202,663 (95.6%) 63,755 (95.6%) 43,148 (94.2%) Bot 3,194 (1.0%) 1,791 (0.8%) 495 (0.7%) 948 (2.1%)

Average Twitter Activity by Source The following table shows that while celebrities accounted for the most tweets among the elite subgroups, politicians and advocacy groups on average posted more tweets per account, with politicians posting the greatest average number of tweets nationally (11.7 tweets per account) and advocacy groups being particularly vocal on abortion-related issues in the state of Illinois (32.9 tweets per account).

Average number of abortion-related tweets per account by source type

Source Type Total Illinois Nevada West Virginia News media 5.7 6.3 6.1 2.8 Advocate/advocacy group 11.6 32.9 4.1 3.5 Political organization 10.6 8.9 7.8 13.3 Other group/organization 6.4 7.8 3.3 3.2 Celebrity 7.8 8.0 10.1 3.9 Politician 11.7 19.2 7.8 2.9 Other individual/person 4.6 5.6 4.7 2.4 Bot 3.1 3.5 2.3 2.6

It is noteworthy that exploratory analyses of account users helped uncover another, less prominent, influencer group—religious institutions and leaders—which predominantly fell under the “other group/organization” category. Religious organizations are well established as opinion leaders and stakeholders on the issue of abortion rights and supporters of the pro-life position.

EXECUTIVE SUMMARY | 26 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Engagement Overall and by State

Engagement with abortion-related tweets is defined as any retweet, quote, or reply to an original post, and reflects a greater interest in and wider exposure with the content than is achieved by tweets that do not generate engagement. The table below shows that the vast majority of posts overall (77 percent) were not retweeted, quoted, or replied to. Abortion content posted in Illinois received the least engagement, followed by Nevada. In contrast, over half of the abortion-related tweets in West Virginia received some engagement.

Engagement: amount of tweets among other individual/person that are retweeting, quoting, or replying to a verified or influential user

Total Illinois Nevada West Virginia Engagement 70,131 (22.7%) 31,747 (15.7%) 17,871 (28.0%) 20,970 (48.6%) Retweet 50,646 24,028 12,702 14,223 Quote 15,689 6,234 4,309 5,273 Reply 3,796 1,485 860 1,474 Not engagement 238,868 (77.3%) 170,916 (84.3%) 45,884 (72.0%) 22,178 (51.4%)

Twitter Messaging Source: Takeaways

■ Politician and advocacy groups were the most vocal; they posted an average of 11-12 tweets per account.

■ While there were more tweets from individuals, on average advocates, political organizations, and politicians were more active on the topic.

■ The level of engagement was the highest in West Virginia (48.6 percent), followed by Nevada (28 percent) among the “other individual/person” category.

■ Hypothesis: the difference in level of engagement with gatekeepers in West Virginia versus Nevada and Illinois was due to the presence of the West Virginia ballot measure as well as the Democratic candidate’s vote on Kavanaugh. These issues were missing in Nevada and Illinois. However, there was still a large amount of organic conversation about the topic but less engagement.

Twitter: Characterizing Message Sentiment This section summarizes key trends in the pro-choice/pro-life sentiment of abortion-related tweets in West Virginia, Illinois, and Nevada from July to December 2018.

We developed models to predict whether individual tweets advocated for pro-choice positions, for pro-life positions, or neither. Prediction models were trained using information about the account authoring the tweet and the combination of words and phrases in the tweet text (see Methodology section for more detail). Because we did not perform source coding of the non-geolocated tweets, our sentiment analyses were conducted on the subgroup of tweets that were geolocated for the three states. Thus, the trends reported below reflect overall sentiment and sentiment by state for only the subset of approximately 300,000 geolocated tweets, rather than the full archive of over 16 million.

EXECUTIVE SUMMARY | 27 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Number of tweets categorized as having pro-choice sentiment, pro-life sentiment, or neither for each state

Three-State Sentiment Category Illinois Nevada West Virginia Total Pro-choice 75,858 (36%) 19,536 (33%) 7,866 (32%) 103,260 (35%) Pro-life 38,108 (18%) 13,764 (23%) 6,932 (28%) 58,804 (20%) Neither (neutral or irrelevant) 97,370 (46%) 26,208 (44%) 9,635 (39%) 133,213 (45%) Pro-choice to pro-life ratio 2.0 1.4 1.1 1.8

Twitter Message Sentiment: Takeaways

■ Pro-choice sentiments were more common than pro-life sentiments across all three states; however, the magnitude of the difference differed substantially across states.

■ The ratio of supporting to opposing sentiment was the highest in Illinois (2:1), followed by Nevada (1.4:1), and lowest in West Virginia (1:1).

■ West Virginia had the lowest proportion of neutral tweets, suggesting that the issue was most divisive in that state.

Twitter: Sentiment over time

There was an apparent rise in pro-life content in the weeks leading up to the election in West Virginia, Illinois, and Nevada. Immediately following the election, the share of pro-life content decreased. This trend is most pronounced in West Virginia, which is also the only state with an abortion-specific ballot measure—and the state in which abortion featured most prominently as a campaign issue (see the Survey campaign monitoring section for additional background about the West Virginia races). These facts provide preliminary evidence for the hypothesis that there was a coordinated effort—most concentrated in West Virginia—to use Twitter to influence citizens to vote in favor of pro-life policies during the 2018 midterm elections. Some of the most influential pro-life accounts certainly played a role, but pro-life advocacy in West Virginia also took place at the level of individual voters voicing their opinions on Twitter. Further targeted analyses could characterize the nature of pro-life advocacy on social media more precisely.

EXECUTIVE SUMMARY | 28 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Valence of state-specific abortion-relevant tweets at each week in the study period

Twitter Message Sentiment over Time: Takeaways

■ The overall pro-choice/pro-life trend:

► For much of August and early September, the proportion of pro-choice tweets was much higher than that of pro-life.

► Starting in early/mid-September, there was a steady rise in the proportion of pro-life tweets. By late October, pro-life tweets became slightly more common than pro-choice tweets.

► Starting in the days immediately following the midterm elections (November 6), the proportion of pro-life tweets decreased substantially, and the proportion of pro-choice tweets returned to its previous peak from the first week of September.

■ What explains the trend?

► Hypothesis 1 (Kavanaugh): In September, the Brett Kavanaugh affair generated a large amount of abortion-related content on Twitter, sparked largely by suspicion among Democrats that Kavanaugh was sympathetic to the idea of repealing Roe v. Wade (while also being clueless about the medical details of abortion). Much of this content consisted of outcries from the political left about the importance of Roe v. Wade. ● This theory could partially explain the rise in pro-choice content during late August and early September. ● However, the Kavanaugh hearing did not take place until September 24, so the timeline is not exactly as one would expect under this theory.

EXECUTIVE SUMMARY | 29 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

► Hypothesis 2 (pro-life activism): For most of September and October, the proportion of pro-life content grew steadily, as the proportion of pro-choice content shrunk at a similar (albeit less consistent) rate. This trend could be the result of a targeted strategy from religious and pro-life advocacy groups and individuals to spread pro-life messaging across Twitter, in an effort to influence the results of abortion-relevant races and ballot measures across the country in the direction of pro-life policies. ● This theory would explain why the proportion of pro-life content grew in the weeks leading up to the election (the period during which voting decisions are finalized). ● It would also explain why the proportion of pro-life content dropped immediately following the election (after which point votes had been cast and there was no strategic political reason to push partisan content).

Next we’ll inspect the state-level trend of pro-choice/pro-life sentiment, which will give us finer-grained information to adjudicate between the two explanations of the trends offered here. While the Kavanaugh confirmation hearing and associated news cycle was largely responsible for the growth in pro-choice sentiment during September, examination of the state-level trends suggests that the rise of pro-life sentiment in October was due at least in part to advocacy for pro-life political causes proliferated on Twitter.

Twitter: State-level Trends in Pro-choice Versus Pro-life Sentiment The following plot shows the weekly percentage of tweets with pro-choice and pro-life sentiment, for each state of interest. This is equivalent to the overall plot above, but broken down by state.

EXECUTIVE SUMMARY | 30 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Valence of abortion-relevant tweets by state at each week in the study period

Twitter Message Sentiment by State over Time: Takeaways

■ The rise in pro-life sentiment leading up to the election was strongest in West Virginia.

► At its September-November peak, pro-life sentiment in West Virginia was above 40 percent—a percentage never reached in the entire study period in either of the other states.

► For six consecutive pre-election weeks (weeks of September 24 through October 29), pro-life sentiment was more prevalent than pro-choice sentiment in West Virginia. No other state experienced more than three such consecutive weeks.

► The drop-off in pro-life sentiment after the election was also the most dramatic in West Virginia, where the pro-life sentiment rate is cut nearly in half in the week following the election.

■ West Virginia is the only one of the three states that had an explicitly abortion-related ballot measure during the midterm election (West Virginia Amendment 1). Therefore, it is likely the state in which abortion was the most salient as a political issue (see population-adjusted by-state volume graph above for further evidence of this).

■ The theory of coordinated pro-life campaigning on Twitter neatly explains why the rise in pro-life sentiment was most extreme in West Virginia: groups would have concentrated efforts in the place where there was a specific ballot measure they could encourage voters to take a position on.

EXECUTIVE SUMMARY | 31 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

■ This theory also explains why the immediately post-election drop in pro-life sentiment was most extreme in West Virginia: after the election, ballots had been cast and the strategic reason to push pro-life messaging disappears.

Twitter: Top National Hashtags Hashtags represent a signpost for conversational threads on Twitter. They enable people to find content of interest by searching a hashtag and indicate to others that they are participating in a larger discourse by flagging their post with a hashtag.

The top five national hashtags included “prolife” (239,743 tweets), “abortion” (95,548 tweets), “Kavanaugh” (91,753 tweets), “MAGA” (66,238 tweets), and “StopKavanaugh” (66,139 tweets).

Top 50 national hashtags (larger font indicates greater number of posts using that word as a hashtag)

EXECUTIVE SUMMARY | 32 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Study Summary Across this three-state case study, there is mixed evidence that a candidate’s pro-life or pro-choice position and communication of this position affected electoral chances. In Illinois and West Virginia, majorities could not accurately describe the candidate with views discordant with their party’s conventional view on abortion. That being said, Republican pro-choice voters were more likely to vote across party lines than Democratic pro-life voters in each state. In Illinois and Nevada, pro-choice attitudes were associated with pro-choice, Democratic candidates winning office, despite controlling for other factors. And while the Democrat in West Virginia was pro-life, he did eventually communicate his position against the ballot inititive and held his seat against a staunchly pro-life Republican challenger. The evidence suggests that communication of abortion position did not hurt pro-choice candidates. There is also mixed evidence that abortion position motivated vote choice. Across states, abortion was predictive of vote choice even when controlling for partisanship and other demographics, but so are other issues, such as immigration and gun control/gun rights. Partisanship was always more predictive of vote choice. In addition, gun control/gun rights and immigration were more highly associated with vote choice in a number of cases. In each state however, abortion ranked relatively low on issue importance, and there were few single issue voters. A conglomerate of issue positions, including abortion position, affected vote choice. The issue of abortion did not seem to mobilize pro-choice and pro-life voters differentially. Voter turnout and campaign participation were similar between pro-life and pro-choice voters. Still, in each candidate race, pro-choice Republicans were more likely to support the Democratic candidate than pro-life Democrats were to support the Republican candidate. The ballot initiative on the issue of abortion had little interaction with the candidates’ positions. Despite the ballot measure in West Virginia being solely about abortion, the candidates did not spend much time discussing it during the campaign relative to other issues, and the abortion issue was not a top priority for voters. Voter contact mattered significantly in voting for Amendment 1, but not among a subset of pro- choice voters.

This study has many strengths in the comprehensive nature of the data and the research design. The multi- method research design allowed for a complex contextualization of public opinion surrounding abortion and the ways candidates and campaigns directly acted and communicated positions. It also allowed us to assess attitudes before the campaign was fully underway, during the height of the campaign, and after voters cast their ballots. The unusual case of having two pro-choice candidates head-to-head, two pro-life candidates head-to-head, and a third race with one pro-life and one pro-choice candidate, also allowed us to compare results with varied electoral contexts.

The study was somewhat limited by the fact that abortion was not a very salient issue in the election. In two states, Illinois and Nevada, candidate communication about abortion was relatively low, and in the third, West Virginia, a majority of voters did not accurately understand candidate positions. Individual state- and candidate-level dynamics across the races studied also limits our findings—in a perfect experimental setting, we would run each of these races in the same state, with the same political context, and with the same demographic composition of the electorate.

EXECUTIVE SUMMARY | 33 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

There are interesting opportunities for future research. We believe these data can be leveraged to understand policy discordant voters. In addition to questions on knowledge of candidate position, the survey also asked questions on voters’ issue positions. We believe there is a large opportunity to profile voters who voted against their policy preferences and understand if misunderstanding of candidates, contact with campaigns, or demographics drove their decision to vote. The large variety of data sources also offers a great chance to conduct more complex analyses using data linkage to understand if there were temporal relationships between spending and advertising and beliefs and positions in survey data and on social media.

Overall, the results show differences in motivations between pro-choice and pro-life voters that are consistent across the three states and campaigns. Several multi-item questions revealed underlying values and pre-dispositions around race, ethnicity, and feminism. Even controlling for demographics and partisanship, pro-choice voters tend to hold more liberal attitudes on race/ethnicity and women. These data suggest that additional issue framing and message testing research could identify abortion messages that resonate with different types of voters.

Methodology

This study was funded by the Susan Thompson Buffett Foundation and conducted by the Public Affairs and Media Research Department and Social Data Collaboratory at NORC at the University of Chicago.

Qualitative Campaign Monitoring

The goal of the campaign monitoring process was to have a deep, qualitative understanding of the campaign context as it related to the topic of abortion. The data collected provides a descriptive summary of how central abortion was to the campaign and the extent to which it was discussed over time.

We collected a range of materials meant to capture the main ways that an election campaign is framed for and communicated to voters. Specifically, we collected the content of campaigns’ and relevant organizations’ websites, Facebook pages, and emails; debate transcripts; political advertisements; Tweets from the candidates’ accounts; and media coverage of the campaign. We also evaluated the source of campaign contributions for the candidates and recorded a timeline of major occurrences in each of the campaigns to provide interpretative context. For a significant portion of the material collected, we worked with a data analytics firm to collect the data.

Using content analysis, we evaluated the volume of campaign communications, ads, and media coverage that covered the topic of abortion and whether the focus on the topic was sustained throughout the campaign.

Collection and Analysis Processes The sections below describe the collection and analysis processes for each type of data collected. In many of the datasets, we searched the content for mentions of abortion with word searches. To be classified as having mentioned the topic, a material had to include one of the following words: abort* as a word stem,

EXECUTIVE SUMMARY | 34 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA pro-life, pro-choice, Planned Parenthood, Emily’s list, Hyde Amendment, reproductive, 20 week ban, right to life, SJR12 and Amendment 1 (in West Virginia), and HB40 (in Illinois).

Campaign and relevant organizations’ websites and Facebook pages The Virtual Tracker data were collected by a data analytics firm. The timeframe for collection of the data was September 11, 2018 until November 6, 2018, Election Day. We followed the websites and Facebook pages of candidates and other important players in the campaigns. The Virtual Tracker monitored these websites by taking screenshots each time the websites were updated or changed in any way. The data show each change of additional text or deletion of text in the sites over time. This allowed us to monitor if candidates mentioned abortion on their websites and see when this content was added or changed. Each website and its pages were recorded in separate datasets.

For each unique page of the websites we tracked, we analyzed whether it mentioned abortion at any point during the time we were tracking changes. We then calculated the percentage of unique pages of the candidate’s website and Facebook profile that mentioned abortion.

Campaign and relevant organizations’ emails We collected emails from each campaign and from relevant organizations working on the issue of abortion in the state. The timeframe for collection of the data was August 24, 2018 until November 6, 2018, Election Day. We received the emails by signing up for the respective campaign or organization distribution list. For each email sent, we analyzed whether it mentioned abortion. We then calculated the percentage of emails sent that mentioned abortion.

Below is a list organizations and the number of emails collected.

■ West Virginia

► Manchin: 38

► Morrisey: 33

► West Virginia for life: 15

► West Virginia Free: 12

► Vote no on 1: 7

► West Virginia Dems: 2

► Planned Parenthood Action local West Virginia: 2

■ Illinois

► Rauner: 62

► Pritzker: 41

► Illinois Family: 51

► Personal PAC: 41

► Planned Parenthood Action Illinois: 19

► ICE PAC: 10

► Democratic party of Illinois: 1

EXECUTIVE SUMMARY | 35 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

► Illinois GOP: 1

■ Nevada

► Rosen: 490

► Heller: 140

► Nevada Democrats: 81

► Nevada GOP: 1

Debate transcripts Debates in all three states were transcribed by a data analytics firm. The West Virginia U.S. Senate debate between Manchin and Morrisey is 10,969 words transcribed and occurred on November 1, 2018. The Nevada U.S. Senate debate is 10,892 words and occurred on October 19, 2018. The three Illinois gubernatorial debates between Pritzker and Rauner totaled 32,357 words and occurred on September 20, October 3 and October 11, 2018.

For analysis, the debate transcripts were read into R with the textreadr package and analyzed for word count and frequency of abortion mentions with the tidytext library. Transcripts were read through to ensure accuracy of the search.

The debate transcripts were also analyzed for frequency of mentions of three other topics – economic issues, guns, and immigration. The words used to identify mentions of these topics are listed below.

■ Economic issues: the word stems econo*, unemploy*, tax*, and wage*, as well as budget, jobs, class, inequality, deficit, debt

■ Guns: the word stems gun* and background check*, as well as second amendment and NRA

■ Immigration: the word stems immigr* and caravan*, as well as citizenship, family separations, separating families, families separated, dreamers, and borders

Political advertisements Political advertisements run on broadcast TV were collected and analyzed by Kantar Media. The timeframe for collection of the data was August 1, 2018 until November 6, 2018, Election Day. Kantar tracks every single ad run, whether it was a unique ad or a rerun of an earlier ad, who sponsored the ad, and what topics were covered in the ad, among other information.

The data provided were analyzed in R and visualized in dygraphs javascript charting library. Based on Kantar’s content coding of the ads, we calculated the percentage of ads run and of ad spending that addressed the issue of abortion, broken down by the sponsor of the ad. We also summarized the number of ads run that focused on abortion over time.

Campaign, candidate and relevant organization tweets A data analytics firm provided tweets for all of the candidate and organization Twitter accounts we requested for Illinois, Nevada and West Virginia. The timeframe for collection of the data was July 1,

EXECUTIVE SUMMARY | 36 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

2018 until November 6, 2018, Election Day. The tweets provided were analyzed in R for mentions of abortion-related words.

We also used the Twitter REST API and rtweet package to scrape tweets for all major party Senate candidates for the 2018 election campaign across all states. We searched these 150,000 tweets for mentions of abortion-related words to provide context for the candidates’ frequency of tweeting about the topic.

Media coverage We used Cisionpoint and Critical Mention to collect information on the news media coverage around the campaigns. The timeframe for collection of the data was August 1, 2018 until November 6, 2018, Election Day.

Cisionpoint was used to collect print and online articles and media related to the campaigns. Short sections of articles were used for content analysis. In Illinois there were 49,064 mentions of Rauner or Pritzker in print and online content. Nevada Senate candidates had 142,796 mentions. West Virginia Senate candidates had 61,648 mentions. All searches through Cision were conducted on the last name of each candidate and then further filtered to include only mentions of the candidate’s full name. Cision data were analyzed via R with the tidytext library.

Critical Mention was used to collect mentions of the candidates on television and radio broadcasts. Short transcripts of broadcasts were used for content analysis. For the Illinois Gubernatorial candidates, there were 24,845 mentions, 43,913 mentions for Nevada U.S. Senate candidates, and 24,849 mentions for West Virginia U.S. Senate candidates. Critical Mention data was collected for all mentions of both the candidate’s first and last name. The data included common misspellings of candidates’ last names, such as “Morrissey” and “Morissey.” Critical mention data were analyzed via R with the tidytext library.

We collected mentions of the West Virginia Amendment 1 in the Critical Mention database, which allows for more complex word searches. This search was conducted with the following Boolean search terms:

“west virginia” AND (abort* OR “hyde amendment” OR “pro life” OR “pro choice” OR reprod* OR “planned parenthood” OR “pro-life” OR “pro-choice”) OR “West virginia free” OR “West Virginians for Life”

This search was designed to represent any mentions of West Virginia and abortion-related terminology or direct mentions of Amendment-1-related organizations.

Media clips were then analyzed for mentions of abortion-related terms. This generated a percentage of news media mentions of the candidates that focused on the topic of abortion.

Campaign contributions A data analytics firm collected all contributions to the federal campaign candidates from the Federal Election Commission. The timeframe for collection of the data was Q1 2018 through the end of the Q4 reporting period. Data on contributions for the Illinois governor’s race came from the Illinois State Board of Elections. In the campaign contributions data, we coded whether the source of the contribution was an

EXECUTIVE SUMMARY | 37 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA organization that advocates or lobbies on the issue of abortion, and then calculated the percentage of all contributions to each candidate that came from abortion-related organizations.

Twitter Analysis

Data Collection Given the vast diversity of users and messages on social media platforms, finding a relevant signal in the noise can be a challenge. Also, with rate limits and other platform restrictions on data collection, researchers need to be careful to ensure that the data collected and analyzed are useful to answer the types of questions being asked. The Twitter platform, being principally regarded as a public forum, provides data access options that facilitate targeted, yet thorough, collection for research. The Twitter Historical Powertrack was used to collect the data for this project, which applies a collection of targeted queries, called search rules, retrospectively on the entire archive of public tweets during a specified time period.

The data collection strategy for this project was conceptualized in two primary themes: abortion-related content, and specific political contest content. These two themes were operationalized across four archives and 59 individual search rules, which were understood to have some overlap. The time period was 7/1/2018 – 12/10/18 in order to capture the run up to voting day as well as 1 month post-election. We created four Archives of relevant data for this project. The table below illustrates the structure of the data sets.

The first dataset type, covering general abortion-related content, included 4 datasets. The primary body of each search rule included multiple terms which were specifically related to abortion, such as “abortion,” “pro-life,” and “pro-choice.” Each term that was included in the search rules was first entered into Twitter search to review the tweets returned, confirm the term has high relevance, and scan for related terms in the tweet content. Several terms were tested, but not included, due to mixed relevance. These terms, such as “women’s health” and “planned parenthood,” often related to abortion, also introduced themes which were not abortion-related, such as maternal mortality. Prior experience with the challenges of cleaning data after collection incentivized the adoption of search rules that were specifically targeted and highly relevant to the theme at the step of data collection.

EXECUTIVE SUMMARY | 38 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Archive 1: We constructed a corpus of tweets containing covered general abortion-related content. The primary body of each search rule included multiple terms which were specifically related to abortion, such as “abortion,” “pro-life,” and “pro-choice.” Each term that was included in the search rules was first entered Search Rule Strategy by into Twitter search to review the tweets returned, confirm Archive the term has high relevance, and scan for related terms in the tweet content. Several terms were tested, but not Archive 1: Terms testing for specific abortion-related content; subsets of included, due to mixed relevance. These terms, such as Archive 1 used operators to filter “women’s health” and “planned parenthood,” often related users from each study state to abortion, also introduced themes which were not Archive 3: Official campaign abortion-related, such as maternal mortality. Prior accounts experience with the challenges of cleaning data after Archive 3: State politics, including collection incentivized the adoption of search rules that tweets that contained campaign hashtags and URLs were specifically targeted and highly relevant to the theme at the step of data collection. We also created subsets of Archive 4: Advocacy group inventory to identify official, verified Archive 1, using geography filters for each study state. state-based abortion advocacy This allowed us to explore what portion of abortion-related groups and political party accounts content in each state intersected with each campaign, as and pages well as a comparison of general attitudes towards abortion across states.

Archive 2 focused on specific campaign accounts of the campaigns of interest for this research. Specific twitter accounts, including official, campaign, and personal accounts which were verified by Twitter, for each candidate in each race of interest were identified and the search rules specifically pulled tweets posted by those accounts. Additionally, to the extent that official twitter accounts existed for ballot measures in West Virginia, tweets posted from those accounts were also collected.

Archive 3 included campaign-related posts beyond those coming specifically from the candidates themselves. Again, given the possibility of so much noise coming from general discussion of politics, the rules focused on capturing posts mentioning the specific campaign accounts in the contests of interest, those posts which included a link to an official campaign website, and posts that used campaign-related hashtags used by candidate accounts.

Archive 4 focused on state-specific political organizations and abortion-related organizations. These search rules included mentions of official accounts representing state-based political parties or abortion advocacy groups in the states-of-interest, and also posts which include links to the official websites of these state-based advocacy groups.

EXECUTIVE SUMMARY | 39 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Dataset Type Dataset Tweet Count d01_abortion_allgeo 16,064,399 General d01_abortion_IL 211,555 Abortion d01_abortion_NV 59,576 Keywords d01_abortion_WV 24,456 d02_accounts_IL_gov 1,527 Specific d02_accounts_NV_gov 2,159 Campaign d02_accounts_NV_sen 4,151 Accounts d02_accounts_WV_amendment1 3,580 d02_accounts_WV_sen 983 d03_hashtags_keywords_IL_gov 214,329 d03_hashtags_keywords_NV_gov 150,123 d03_hashtags_keywords_NV_sen 1,055,647 d03_hashtags_keywords_WV_amendment1 27,371 State d03_hashtags_keywords_WV_sen 1,295,858 Campaign d03_websites_IL_gov 4,261 Politics d03_websites_NV_gov 5,008 d03_websites_NV_sen 24,291 d03_websites_WV_amendment1 1,010 d03_websites_WV_sen 4,296 d04_groups_IL 3,893 State d04_groups_NV 2,432 Abortion d04_groups_WV 4,126 Advocacy d04_groupsites_IL 2,999 Groups d04_groupsites_NV 26,060 d04_groupsites_WV 548

Comprehensiveness of the keywords As an additional quality control check regarding the comprehensiveness of the search terms which were adopted for this research, some terms which were not adopted, yet could be relevant, were tested more thoroughly. This was done by scanning the public 1 percent twitter stream (a 1 percent sample of all tweets across Twitter), and cross checking tweets found there both for relevance and also against the archive created for this research. The proportion found from the 1 percent which was also present in the project archive allowed an estimate of the volume that would have been added, had these terms been adopted. The additional terms tested for this including variations with and without a space of “planned parenthood,” “my body my choice,” “women health,” and “reproductive right.”

The table below suggests that, of these terms, the only concept which would have provided a meaningful amount of relevant content was “planned parenthood” (or “plannedparenthood”). However, due to the fact that only a fraction of these data were clearly relevant to abortion, this additional volume would have also introduced non-relevant noise. Without an adequate relevance-cleaning process, which can be quite cumbersome, this extra noise may have distorted some results.

EXECUTIVE SUMMARY | 40 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Key term quality control questions:

Do the tweets that match these terms were retrieved from Historical PowerTrack? Are they abortion-related?

Match in Match in PowerTrack Relevance in Projected Amount of Keywords Streaming Data (our DB) sample Relevant Tweets Planned parenthood 16,488 3,169 50% 665,950 (4.1%) My body my choice 338 79 68% 17,742 (0.1%) Women health 483 4 0% 0 Reproductive right 50 28 100% 2,200 (0.01%) The tweets that match “planned parenthood” would likely have influenced the amount of tweets we collected. We observed that a tweet is often relevant to abortion when the term is mentioned in the perspective of pro-life or anti- abortion although it is tricky otherwise.

Data Collection Limitations Data collection has several limitations. Twitter users are not representative of the general population, or of registered voters. Due to the company’s protective policies it can be difficult to know precise demographics of any subset of the Twittersphere and therefore adjust outcomes to represent general populations. Additionally, the company’s diligence in purging bots and bad actors (those who violate Twitter terms & conditions) may result in older portions of the historic timeline containing fewer bots and bad actors than more recent portions of a timeline. However, Twitter, remains a leading social media network for thought leaders and influencers who have the potential to sway the opinion of their followers. Further, the network allows full access to researchers unobstructed by privacy settings. Data collection was defined by the keywords and search filters used to retrieve data from Twitter. As such, search filters are the lens through which we can observe what and how people communicate. This lens should be appropriately focused, so we can identify the content of interest without the noise of unrelated conversation. If a search is too narrow, it may miss important data, and conclusions may be biased. Conversely, if it is too broad, there is a risk collecting irrelevant and potentially misleading material. Given the limited budget and time, this project used a pre-defined and restricted set of search terms and hashtags, which may have missed some abortion-related Twitter content. For instance, under the campaign datasets, the decision to search based on hashtags which appeared on official campaign posts as opposed to any hashtag which could be campaign relevant insured a high precision of the campaign hashtag data, but likely excluded posts featuring more obscene and vitriolic hashtags than those which the campaigns themselves would publish. In addition, other abortion related terms like “reproductive rights,” “women’s health”, and “planned parenthood” were excluded because these terms are more expansive and include health related topics other than abortion. To compensate for the commitment to precise keywords, a large number of search terms were applied, capturing the data from multiple dimensions (e.g. account handles, hashtags, and URL links). The abortion terms, specifically, are discussed under the “comprehensiveness of the keywords” heading.

Twitter: Sentiment Classification We developed supervised classification models to categorize tweets as expressing a pro-choice sentiment, a pro-life sentiment, or neither. The model development process consisted of first hand-coding several

EXECUTIVE SUMMARY | 41 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA thousand tweets, and then training binary classifiers to predict the human-assigned categories as accurately as possible (one to predict pro-choice sentiment, one for pro-life sentiment, and one for "neither"). After optimal model parameters were determined (given the available training data), we generated predictions over all abortion-relevant tweets collected from Illinois, Nevada, and West Virginia. Predicting the sentiment of a tweet was a two-step procedure: tweets were first screened for “polarity”, meaning that they express a (subjective) opinion or attitude about the issue of abortion. Tweets not deemed to express an opinion/attitude were categorized as “neither.” The remaining tweets were then categorized as either “pro-choice” or “pro-life,” depending on which category's predicted probability was higher.

The three broad sentiment categories are defined as follows:

■ ‘Pro-choice’ category: A tweet is counted as “pro-choice” if it promotes the concept of (access to) abortion as a healthcare or women’s rights issue, if it expresses support for pro-choice organizations or political candidates, or if it disparages pro-life positions, organizations, politicians, etc. Tweets are also counted as pro-choice if they express support for a political candidate on the grounds of their pro-choice positions on abortion.

■ ‘Pro-life’ category: A tweet is counted as “pro-life” if it frames abortion as a moral or religious issue as opposed to a healthcare issue, if it frames abortions as “murder” or fetuses as “tiny lives,” if it disparages abortion providers or politicians with pro-choice stances, etc. Tweets are also counted as pro-choice if they express support for a political candidate on the grounds of their pro-choice positions on abortion.

■ ‘Neither’ category: A tweet is counted as “neither” if it does not clearly express a sentiment, attitude, or opinion in favor of a pro-choice or pro-life position. About 40% of human-labeled tweets were categorized as “neither.” A label of “neither” doesn’t mean a tweet is irrelevant to abortion, but rather that the tweet text itself does not express an attitude or opinion about abortion. For example news headlines would count as neither, as would jokes or irrelevant passing references to abortion. Our sentiment model aimed to identify tweets the text of which takes an explicit position on the political/ethical/medical issue of abortion, and so many abortion-relevant tweets simply do not fall into the category of “pro-choice” or “pro-life” – such tweets fall into the elsewhere-category of “neither.”

Source Coding Source coding was performed by humans, hand-coding several thousand tweets to determine the type of user that created the tweet. For all individual verified and unverified users, the content of tweets were assessed in order categorize the user as one of the following types:

■ News organization and news media professionals: Any account that represents a news organization, a freelance journalist, or an employee, reporter, or correspondent of a news organization.

■ Abortion advocacy group and advocate: Any account that represents an advocacy group, organization, advocate, or activist that focuses on abortion-related issues.

■ Political organization: Any account that represents a political organization.

EXECUTIVE SUMMARY | 42 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

■ Other Group/Organization: Any group or organization account that does not fit the categories above, i.e., an organization that is not a news organization, abortion advocacy group, or political organization.

■ Celebrity: Any account that belongs to a famous person in the entertainment or culture industry, including but not limited to: writers, actors, directors, producers, musicians. The account was required to either be a verified user, or have 8,000+ followers. Note that celebrities working in news media were not in this category, and instead would be labeled into the category for news organization and news media professionals.

■ Politician: Any account that belongs to a person who is a politician or a candidate for public office.

■ Other Individual/Person: Any individual that is not celebrity or politician.

Source Sample A total of 4172 accounts were collected among in the state abortion-related tweets for source coding. These include 518 verified account, which represent all verified accounts found in the data. Also included are 3654 influential accounts based on number of followers (at least 3223, the top 10 percentile within the relevant data) and ratio of follower to following (greater than 1). For each of those account, the account metadata and all of their tweets were collected into a spreadsheet for coders to perform source coding.

Bot Filter A total of 69,017 unique users found in our state abortion-related data was sent to Botometer API20 through their official Python client library21 for bot detection.

Botometer is a collaboration between Indiana University Network Science Institute (IUNI) and the Center for Complex Networks and Systems Research (CNetS).22 It detects bot using a machine learning algorithm trained on a dataset of 5.6M tweets coming from 15k verified bots and 16k verified human accounts. The algorithm generates a Complete Automation Probability (CAP), which is the probability that an account is a bot, based 6 type of characteristics extract from the account:

■ Network: based on retweets, mentions, and hashtag co-occurrence.

■ User: based on Twitter meta-data related to an account, including language, geographic locations, and account creation time.

■ Friends: based on an account’s social contacts.

■ Temporal: based on timing patterns of content generation and consumption.

■ Content: based on linguistic characteristics of the tweets from the account.

■ Sentiment: based on general sentiment detection of the tweets from the account and usage of emoticon

After obtaining CAP for all of the accounts, we used a cut-off point of 0.5 to flag an account as bot. This yield 1030 bots. The number seems low because bot detection algorithm tends to be conservative. As the

20 https://botometer.iuni.iu.edu/#!/api 21 https://github.com/IUNetSci/botometer-python 22 https://arxiv.org/abs/1602.00975

EXECUTIVE SUMMARY | 43 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA website of Botometer states: “CAP may look like a conservative estimate, but one must consider that there are a lot more humans than bots, so you'd better be sure before accusing someone of being a bot!”

Top terms and hashtags by source All of the tweets coming from each source type are used to count top terms and hashtags by source. For hashtags we use the meta data field provided by GNIP, which contains a list of all hashtags found in each tweet, if any. For top terms, we first remove hashtag, mention, URLs, punctuation, and stop words before conducting the count.

Identifying West Virginia Amendment 1 Predictive Terms A Support Vector Machine model was fitted for Support vs. Not-Support and Oppose vs. Not-Oppose using the Unigram, Bigram, and Trigram from the tweets body as features. Both model achieved an overall F-1 score of 90% on 5-fold cross-validation. The top 10 most predictive features from the model for Support and Oppose are reported.

State-Level Surveys

The study included three Waves of surveys among adults 18 and older who are registered voters in Nevada, Illinois, and West Virginia. Respondents came from two samples – the AmeriSpeak panel and TargetSmart’s VoterBase.

AmeriSpeak Sample Funded and operated by NORC at the University of Chicago, AmeriSpeak® is a probability-based panel designed to be representative of the US household population. Randomly selected US households are sampled using area probability and address-based sampling, with a known, non-zero probability of selection from the NORC National Sample Frame. These sampled households are then contacted by US mail, telephone, and field interviewers (face to face). The panel provides sample coverage of approximately 97 percent of the U.S. household population. Those excluded from the sample include people with P.O. Box only addresses, some addresses not listed in the USPS Delivery Sequence File, and some newly constructed dwellings. While most AmeriSpeak households participate in surveys by web, non-internet households can participate in AmeriSpeak surveys by telephone. Households without conventional internet access that have web access via smartphones are allowed to participate in AmeriSpeak surveys by web. For this study, survey respondents were offered a small monetary incentive per Wave of the survey they completed.

TargetSmart Sample To supplement the surveys conducted via AmeriSpeak in each state, sample lists of registered voters were obtained from TargetSmart, a commercial database vendor. For the first Wave, a mail campaign comprised of an early-notification postcard and a follow-up invitation packet with a $1 bill was used to recruit registered voters. Respondents were invited to complete the survey either on the web at a study- specific website or via phone using a study-specific toll-free number. Respondents were given a personal

EXECUTIVE SUMMARY | 44 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA identification number and offered a monetary incentive as a thank you for completing the survey. An example of the recruitment brochure is in the table below.

For the second and third Wave, qualified respondents who completed the first survey and gave permission to be recontacted were mailed an invitation packet with a $2 bill and sent an email invitation with a link to the survey. The identity of the respondent was confirmed before rescreening for registration status. For each Wave, respondents were sent multiple reminder emails.

EXECUTIVE SUMMARY | 45 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Example Recruitment Brochure: Illinois

EXECUTIVE SUMMARY | 46 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Field Period, Respondent Screening and Weighting Respondents from the AmeriSpeak panel and those recruited from the TargetSmart sample were surveyed in three Waves. The first Wave was conducted from August 13 to August 24, 2018. The second Wave was completed just before Election Day, fielding from October 24 to November 5, 2018. And the final Wave of the survey was a post-election survey conducted from November 7 to November 21, 2018.

Respondents in the first Wave were screened for whether they reported being registered to vote or planning to register to vote before Election Day. For the second Wave, respondents were reasked whether they were registered to vote only in Nevada and West Virginia. In Illinois, they were also reasked whether they intended to register, since it is possible to register up until Election Day in Illinois. For the third Wave, respondents were screened for self-reported voter registration only.

Surveys were provided in both English and Spanish, depending on the preference of the respondent.

Once the study data had been collected and made final, the two samples were weighted together to create a sample that is representative of registered voters in each state. The samples were weighted for probability of selection and poststratified to adjust for any survey nonresponse as well as any noncoverage or under- and over-sampling resulting from the study-specific sample design. Poststratification variables included age, gender, race/ethnicity, and education. Weighting targets were obtained from the 2016 Current Population Survey voter supplement. The weighted data, which reflect Illinois, West Virginia, and Nevada registered voters, age 18 and older, were used for all analyses.

The tables below report the sample sizes, margins of error and response rates by state for each Wave of the survey.

Illinois Wave 1 Wave 2 Wave 3 Field dates in 2018 8/8 - 9/4 10/23 - 11/5 11/7 - 11/21 Overall sample size 3,378 2,383 2,525 Online completes 3,202 2,316 2,463 Phone completes 176 67 62 AmeriSpeak completes 309 250 237 TargetSmart completes 3,069 2,133 2,288 Design effect 1.41 1.66 1.60 Margin of Sampling Error (pct. pt) +/- 2.0 +/- 2.6 +/- 2.5 Cumulative response rate (%) 8.8 6.2 6.6 AmeriSpeak: Final stage completion rate (%) 92.0 96.2 97.1 AmeriSpeak: Weighted HH panel response rate (%) 28.6 28.6 28.6 AmeriSpeak: Weighted HH panel retention rate (%) 86.0 86.0 86.0 AmeriSpeak: Cumulative response rate (%) 7.6 6.2 5.9 TargetSmart: Final stage completion rate (%) 91.9 97.0 98.7 TargetSmart: Cumulative response rate (%) 8.9 6.2 6.7

EXECUTIVE SUMMARY | 47 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Nevada Wave 1 Wave 2 Wave 3

Field dates in 2018 8/8 - 9/4 10/23 - 11/5 11/7 - 11/21 Overall sample size 3,021 2,049 2,196 Online completes 2,974 2,035 2,172 Phone completes 47 14 24 AmeriSpeak completes 56 39 39 TargetSmart completes 2,965 2,010 2,157 Design effect 1.20 1.36 1.30 Margin of Sampling Error (pct. pt) +/- 2.0 +/- 2.5 +/- 2.4 Cumulative response rate (%) 6.7 4.6 4.9 AmeriSpeak: Final stage completion rate (%) 96.6 97.5 100.0 AmeriSpeak: Weighted HH panel response rate (%) 28.6 28.6 28.6 AmeriSpeak: Weighted HH panel retention rate (%) 86.0 86.0 86.0 AmeriSpeak: Cumulative response rate (%) 7.3 5.6 5.2 TargetSmart: Final stage completion rate (%) 92.7 97.3 98.7 TargetSmart: Cumulative response rate (%) 6.7 4.6 4.9

West Virginia Wave 1 Wave 2 Wave 3 Field dates in 2018 8/8 - 9/4 10/23 - 11/5 11/7 - 11/21 Overall sample size 4,013 2,657 2,831 Online completes 3,858 2,590 2,778 Phone completes 155 67 53 AmeriSpeak completes 48 39 36 TargetSmart completes 3,965 2,618 2,795 Design effect 1.45 1.56 1.58 Margin of Sampling Error (pct. pt) +/- 1.9 +/- 2.4 +/- 2.3 Cumulative response rate (%) 11.2 7.6 7.9 AmeriSpeak: Final stage completion rate (%) 92.3 92.9 92.3 AmeriSpeak: Weighted HH panel response rate (%) 28.6 28.6 28.6 AmeriSpeak: Weighted HH panel retention rate (%) 86.0 86.0 86.0 AmeriSpeak: Cumulative response rate (%) 9.4 8.2 7.4 TargetSmart: Final stage completion rate (%) 91.6 95.8 98.6 TargetSmart: Cumulative response rate (%) 11.2 7.6 7.9

EXECUTIVE SUMMARY | 48 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Analysis All analyses were conducted using Stata (version 14), which allows for adjustment of standard errors for complex sample designs. All differences reported between subgroups of the population are at the 95 percent level of statistical significance, meaning that there is only a 5 percent (or less) probability that the observed differences could be attributed to chance variation in sampling. Additionally, bivariate differences between subgroups are only reported when they also remain robust in a multivariate model controlling for other demographic, political, and socioeconomic covariates.

Open ended survey responses were coded in two ways. Short open ends were coded completely by hand by a single coder. Questions Q10 and Q11 from Wave 1 and Wave 2, which asked respondents to name the top problems in the country and in their state, were coded using supervised machine learning. Data from these open ends were trained on 20,000 responses from past AmeriSpeak Omnibus surveys asking respondents what the top five problems were facing the nation. For Q11, an additional thousand state- based responses for each state were included in the training data to provide more nuanced classification for state based issues. The data were used to train a support vector machine to classify responses into nearly 30 codes. These codes were collapsed into more general categories based on topic.

Voter Validation For the voter validation, a vendor, L2 attempted to match each respondent’s name and address to records on the state voter files. If a respondent matched a record on the voter file, the state voter file documents whether or not that person voted in the 2018 election. This information was appended to the survey data. Among those who reported voting in Wave 3 of the survey, 83 percent in Nevada, 78 percent in Illinois, and 78 percent in West Virginia were verified as having voted in the 2018 election via this method. For every analysis of attitudinal and behavioral differences among self-reported voters, we conducted a robustness check on the findings by analyzing whether the differences held up among validated voters. All key findings in the report hold up among validated voters.

EXECUTIVE SUMMARY | 49 NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

Appendix

*P<.05 **P<0.01 ***P<.001

______1 Table represents sample among only respondents validated as voters 2 Religious refers to a binary recode of any indicated religion or an indication of no religion 3 Immigration opinion scaled from 1=strongly favor path to legality to 5=strongly oppose path to legality 4 Healthcare opinion scaled from 1=strongly agree it is the responsibility of the federal government to provide healthcare to 5=strongly disagree 5 Abortion opinion rescaled to 1=legal in all cases to 5=illegal in all cases 6 Economic importance scaled from 1=not at all important to 5=extremely important 7 Gun opinion scaled from 1=much more strict to 5=much less strict

EXECUTIVE SUMMARY | 50