The Consequences of Online Partisan Media Contents

The consequences of online partisan media Supplemental Materials

Contents

1 Study methodology 3 1.1 Selection of treatment conditions ...... 3 1.2 Study design ...... 4 1.2.1 Panel structure ...... 4 1.2.2 Panel attrition ...... 5 1.2.3 Encouragement design ...... 6 1.3 Model speciﬁcations ...... 9 1.4 Variable speciﬁcations ...... 10 1.4.1 Issue opinions ...... 10 1.4.2 Agenda-setting effects ...... 11 1.4.3 News browsing behavior ...... 11 1.4.4 Media trust ...... 12

2 Descriptive statistics 13 2.1 Covariate balance ...... 13 2.2 Compliance ...... 15

3 Full results 22 3.1 Issue opinions ...... 25 3.2 Subsequent news browsing behaviors ...... 27 3.3 Subsequent social media behaviors ...... 29 3.4 Affective polarization ...... 30 3.5 Perceived polarization ...... 32 3.6 Agenda-setting ...... 33 3.7 Approval of President/Congress ...... 34 3.8 Voting behavior ...... 36 3.9 Media trust ...... 37 3.10 Knowledge ...... 40 3.11 Election prediction ...... 42 3.12 Mail-bombing incident ...... 44

4 Additional results: Wave 5 46 4.1 Issue opinions ...... 46 4.2 Agenda-setting ...... 47 4.3 Approval of President ...... 48

1 4.4 Media trust ...... 49 4.5 Knowledge ...... 51

5 Additional results: Waves 7 and 8 52 5.1 Issue opinions ...... 52 5.2 Agenda-setting ...... 53 5.3 Media trust ...... 53

6 Additional results: Comparing two treatment groups 55 6.1 Issue opinions ...... 57 6.2 Subsequent news browsing behaviors ...... 58 6.3 Subsequent social media behaviors ...... 59 6.4 Affective polarization ...... 60 6.5 Perceived polarization ...... 61 6.6 Agenda-setting ...... 62 6.7 Approval of President/Congress ...... 63 6.8 Voting behavior ...... 64 6.9 Media trust ...... 65 6.10 Knowledge ...... 66 6.11 Election prediction ...... 67 6.12 Mail-bombing incident ...... 68

7 Additional results: Multiple testing adjustments 69

8 Deviations with respect to pre-analysis plan 73

9 Survey questionnaire 75 9.1 Issue opinions ...... 76 9.2 Affective polarization ...... 79 9.3 Perceived polarization ...... 80 9.4 Agenda-setting effects ...... 81 9.5 Approval of President/Congress ...... 82 9.6 Voting behavior ...... 82 9.7 Media trust ...... 83 9.8 Knowledge ...... 84 9.9 Election prediction ...... 85 9.10 Mail-bombing incident ...... 87 9.11 Experiment factual questions ...... 87 9.12 Other variables used as controls ...... 88

2 1 Study methodology

This section offers a detailed description of the methodology of our study, including how the treatment conditions were selected, how the randomized encouragement was administered, and how the variables and models are speciﬁed. It follows the pre-analysis plan posted at https: //osf.io/zj65h.

1.1 Selection of treatment conditions The selection of the encouragement to consume news information from either Fox News or HuffPost was based not only on the signiﬁcance of the two websites in partisan news consumption in the current political environment but also on empirical web-tracking data during the pre-treatment period, and on the approximate equivalence of their ideological slant estimated in a previous study Bakshy et al. (2015). Using data from April and May 2018, Table 1 shows the average number of visits to a page on foxnews.com and huffingtonpost.com by each respondent. Over the two-month period, respondents saw a Fox News article roughly 13 times and a HuffPost article between 8 and 9 times on average. There is of course a lot of variation: Table 2 shows that around 70-80% of respondents did not visit pages on either site. These results show both that the two sites are roughly equally visited (although Fox News is somewhat more common in the data) and that a large share of the sample would not be categorized as “Always-Takers” in the experiment. Finally, Table 3 illustrates that the estimated ideological slant of the two news sources, based on the domain audience “alignment” scores computed by Bakshy et al. (2015) based on Facebook data, is well to the right and left of center respectively, and in roughly symmetrical positions.

Table 1: Average number of visits to Fox News and HuffPost per person

Fox News HuffPost 13.1 8.5

Table 2: Average share (in %) of respondents who visited Fox News and HuffPost at least once

Fox News HuffPost Fox News and HuffPost 30.4 19.0 10.2

3 Table 3: Slant estimates for Fox News and HuffPost based on Bakshy et al. (2015)

Domain Alignment Score foxnews.com 0.775 hufﬁngtonpost.com -0.618

1.2 Study design 1.2.1 Panel structure The experiment was embedded in a panel survey fielded on initially N = 1, 551 respondents recruited from the YouGov U.S. Pulse panel, which enables tracking of people’s web usage on desktop and mobile devices. The Pulse panel is a subset of YouGov’s traditional survey panels, where respondents opt in to install tracking software on their devices. Figure 1 provides a conceptual overview of the study data collection process. We conducted eight survey waves: a baseline survey (Wave 1), a survey with pre-treatment covariates (Wave 2), another survey with pre-treatment covariates in which the treatment was administered (Wave 3), and two post-treatment surveys that contain our outcome measures (Waves 4 and 5). In addition, we conducted three additional waves that were not pre-registered, which allowed us to explore the longevity of the treatment effects. The main survey (Wave 1) launched on July 3, 2018. The encouragement was issued in Wave 3, which was fielded on October 5 — a month before the midterm elections. Outcomes were collected in Wave 4 before Election Day (November 6) and Wave 5 after Election Day. The rest of the waves were fielded in late January 2019 (Wave 6), early April 2019 (Wave 7), and late October 2019 (Wave 8). We also collected web tracking data (desktop and mobile) and Twitter data (for the N = 471 respondents who shared it) throughout this entire period.

Figure 1: Panel setup

Election date

W1 W2 W3 W4 W5 W6 W7 W8

2018 Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct 2020 Randomized encouragement

More in detail, the pre-registered structure of our panel design is as follows:

• Wave 1 (baseline) included a battery of questions about news media consumption, attitudes about domestic and foreign policy issues, turnout and vote choice, and presidential

4 approval. It was deployed by YouGov starting on July 3, 2018. Due to a planned tran- sition in the web tracking software used by YouGov (from Wakoopa to Reality Mine), Pulse recruitment was not completed until late July. All respondents successfully transi- tioned to the new software by July 31, and thus we consider the beginning of our period of analysis for the web visit data to be August 1, 2018.

• Wave 2 (ﬁelded starting August 28) contained numerous questions related to our larger project. Questions relevant for the experiment included affective polarization and social distance measures, news reception, policy attitudes, and projected U.S. midterm election outcomes.

• During Wave 3 (ﬁelded starting October 5), we administered the randomized encouragement, which we describe in greater detail in the following section. Wave 3 also included a few questions related to other pre-treatment covariates: attitudes about domestic and foreign issues, most important problem, presidential feeling thermometer, Congress control preference after the midterms, and turnout intention.

• Wave 4 (ﬁelded starting October 30, up until November 5, the day before the election). This survey included a battery of outcome variables, such as attitudes about domestic and foreign issues, most important problem, political knowledge, affective polarization, presidential approval and vote intention/choice, racial resentment, threat perceptions, beliefs about the effects of trade, social media use, and media trust (including trust speciﬁcally in HuffPost and Fox News).

• Wave 5 (ﬁelded starting December 20): Similar to Wave 4, this survey included a battery of outcome variables related to political attitudes and behavior.

• Waves 6 to 8 (ﬁelded starting January 24, 2019; April 1, 2019; and October 9, 2019) contained similar items as Wave 5, as well as other survey items related to e.g. the ongoing impeachment inquiry. These additional waves were not part of the registered pre-analysis plan.

1.2.2 Panel attrition Figure 2 computes the attrition rates across waves. Each cell indicates the number of respondents who completed the wave on the x-axis among those who had completed the wave on the y-axis. For example, out of the 1,098 respondents who completed Wave 3, a total of 1,037 also completed Wave 4 (an attrition rate of 5.6% between these two waves). As this ﬁgure indicates, the wave-to-wave attrition rates vary between 4.7% and 18%. Not surprisingly, attrition rates increase as the time between waves increases. The number of respondents who completed Waves 2, 3, and 4 (our key waves for this study) is N = 1, 037, which represents an attrition rate of 22.5% with respect to Wave 2).

5 Figure 2: Attrition in multi-wave panel design

8 N=1089

988 7 N=1129 (12.5%)

1086 1038 6 N=1215 (10.6%) (14.6%) % attrition 1139 1058 1013 5 N=1195 (4.7%) (11.5%) (15.2%) 20% 1118 1130 1064 1016 4 N=1197 10% (6.6%) (5.6%) (11.1%) (15.1%) 0% Respondent's first wave 1037 1014 1032 959 926 3 N=1098 (5.6%) (7.7%) (6.0%) (12.7%) (15.7%)

1098 1197 1195 1215 1129 1089 2 N=1339 (18.0%) (10.6%) (10.8%) (9.3%) (15.7%) (18.7%)

1339 1098 1197 1195 1215 1129 1089 1 N=1551 (13.7%) (29.2%) (22.8%) (23.0%) (21.7%) (27.2%) (29.8%)

1 2 3 4 5 6 7 8 Respondent wave of comparison

1.2.3 Encouragement design Our randomized encouragement was administered during Wave 3. One third of the sample was asked to change their browser homepage to a left-leaning news outlet (HuffPost); another third was asked to change it to a right-leaning news outlet (Fox News); and another third received no treatment (control group). The encouragement also asked respondents to follow the outlets’ corresponding Facebook pages and sign up for related email newsletters. Randomization was blocked by respondents’ browser type (Chrome / Firefox / Safari / IE / Edge / Other). Respon- dents received clear instructions about how to do this and were compensated for doing so with a points reward equivalent to $8. To incentivize compliance over time, they were also told that they will receive an additional bonus if they pay attention to the website and answer questions about “about what you may have seen on the site”. In Wave 5, we included a battery of questions regarding knowledge of current events, and we gave all respondents a bonus. The speciﬁc messages that subjects saw in Wave 3 as part of our encouragement are shown in Figure 3. Respondents could answer yes or no to the encouragement. Those who answered “yes” were presented with instructions. These instructions varied by treatment (Fox News or HuffPost) and browser used (Chrome / Firefox / Safari / IE / Edge / Other) and included screen shots where possible (see Figure 4 and below for examples).

6 Figure 3: Treatment encouragement

Example: Homepage setup instructions, Fox News assignment and Chrome usage Thank you very much! You are providing a very important contribution to research. It appears that you are using the Google Chrome browser. Please follow these simple steps to temporarily change the contents of your New Tab page.

1. At the top right of the current Chrome window (to the right of the address bar), click ... > Settings.

2. Scroll down until you see the “On startup” header, then select Open a speciﬁc page or set of pages.

3. Click Add a new page. Enter foxnews.com and click Add.

Then, for respondents who said in Wave 1 that they have a Facebook account, we asked the following:

7 Figure 4: Example of homepage setup instructions

Example: Facebook page like instructions, Fox News assignment In a previous survey you told us that you use Facebook. We would also like to invite you to follow the ofﬁcial Facebook page of this news and information site by clicking the button “Like Page” below.

We used iframe to embed the Facebook page for Fox News or HuffPost depending on condition. Finally, for the Fox News condition we asked the following:

Email newsletter subscription instructions, Fox News assignment You’re almost done. Thank you! Now there’s just one more step. Please click here. In the new tab, select “Subscribe” under “Fox News First” and enter your personal email address. Please also select “Subscribe” under “The Scoop” and enter your personal email address. (This will not be shared with the researchers.) Close the tab or window once you are done to return to this page.

And for HuffPost:

Email newsletter subscription instructions, HuffPost assignment You’re almost done. Thank you! Now there’s just one more step. Please click here, enter your personal email address, check “The Morn- ing Email” and “Politics,” and click on “Subscribe.” (This will not be shared with the researchers.) Close the tab or window once you are done to return to this page.

8 1.3 Model specifications Following our pre-analysis plan, “don’t know” responses were considered missing data for our outcome measures. Missing values in the covariates were treated as missing, unless inclusion of covariates per our pre-specified models resulted in dropping 20% or more of observations. In such cases, we would have used multiple imputation. However, none of the regressions testing our hypotheses had this level of missingness. For all experimental analyses, we report both unadjusted and covariate-adjusted differences in means. Since randomization was blocked by the detected web browser used by respondents as they entered the Wave 3 survey, we need to account for differential probability of assignment across blocks in both sets of analyses. For unadjusted estimates, we use block-wise differences in means.1 For adjusted estimates, we employ Lin’s (2013) saturated regression approach, which simultaneously allows us to control for differential probability of assignment and to maximize precision via covariate adjustment. We use HC2 robust standard errors in all analyses and report p-values from two-tailed t-tests. Additionally, we report both Intent-to-Treat (ITT) and Complier Average Causal Effect (CACE) estimates. For the CACE, we provide separate estimates of the effect of exposure to Fox News vs. control and of the effect of exposure to HuffPost vs. control among compliers. For the CACE, we use an IV framework and operationalize treatment receipt according to our pre-specified definition. Depending on the comparison, we define as “treated” any respondent who (1) visited the FoxNews.com homepage at least once during each full 7-day period beginning with completion of Wave 3 through completion of Wave 4 (inclusive), or (2) visited the huffingtonpost.com homepage at least once during each full 7-day period beginning with completion of Wave 3 through completion of Wave 4 (inclusive). Then, we subset to the relevant treatment/control pair (dropping the other treatment group) and, using an IV setup, instrument receipt of the relevant treatment using this definition with treatment assignment. For all covariate-adjusted models, we select covariates for inclusion using lasso with default options in glmnet. We run this separately for each analysis specified in this document. Following our pre-analysis plan, our list of pre-treatment covariates for possible inclusion was: gender, education, age, age squared, party ID, race/ethnicity, ideology, income, employment status, state of residence, political interest, news consumption from Pulse (log count of URLs from domains in Bakshy et al. 2015 list), average media diet slant using Bakshy et al. list Guess (2021), average frequency of political information consumed from media sources (TV, newspaper, radio, internet, political discussions), pre-treatment version of the DV (if available). We generally find that ideology and partisanship are included in most models, as well as the pre-treatment version of the DV when available.

1See https://declaredesign.org/blog/biased-fixed-effects.html.

9 1.4 Variable specifications In this section, we describe the specifications of the variables we constructed for our models. To see the full wording of the survey items, see Section 9. The specifications follow our pre- analysis plan unless noted in Section 8.

1.4.1 Issue opinions We measure the extent to which respondents’ issue opinions are conservative or liberal by con- ducting a principal components analysis (PCA) of their responses to a set of policy issue questions. We run PCA in the set of responses in Wave 2 to derive a policy opinions index by taking the ﬁrst principal component, and then predict the policy opinions index in Wave 4 by using the same set of weights computed in Wave 2. The set of issues we consider (see questions on “Domestic Issues”, “Foreign Issues”, and “Policy Trade-offs”) include both domestic and foreign issues, as well as questions speciﬁcally related to current events. Several of these items (5, 7, 8) were adapted from Bail et al. (2018) and the “Ideological Consistency Scale” in Dimock et al. (2014). As shown in Table 4, all of these items are highly correlated, with a single principal component (which we label “conservatism”) explaining 68% of the variance.

Table 4: PCA loadings: Issue opinions scale (conservatism)

Survey item Loading Gun control laws in the United States should be stricter -0.80 Free trade agreements like NAFTA have helped the U.S. economy -0.70 Pres. Trump should have the ability to pardon himself 0.53 Government regulation of business is necessary to protect the public interest. -0.67 We should not think so much in international terms; concentrate more on our problems 0.40 Poor people today have it easy because they can get government benefits 0.59 Business corporations make too much profit -0.71 The U.S. will need to use military force to resolve the situation with North Korea -0.20 A zero-tolerance policy for sexual harassment is essential for social change -0.71 Islam encourages violence more than other faiths 0.53 Global warming will pose a serious threat to me or my way of life in my lifetime -0.80 The FBI’s investigation on Russia-Trump collusion only harms the U.S -0.75 DREAMers should leave the country and apply for citizenship like everyone else 0.60 Standardized loadings for first principal component based upon correlation matrix (with varimax rotation). First principal component explains 68% of the variance on all 13 items.

We use three items to measure attitudes about immigration: the policy trade-off regarding DACA (trade-off B), the policy trade-off regarding the migrant caravan (trade-off E), and the threat perception about undocumented immigrants taking jobs away from American citizens. We apply the same factor analysis scaling method as above, but using Wave 3 responses in

10 the control group instead of Wave 2 responses, which were not available. As shown in Table 5, these three items are highly correlated, with a single principal component (which we label “pro- immigration attitudes”) explaining 57% of the variance.

Table 5: PCA loadings: Pro-immigration attitudes scale

Survey item Loading DREAMers should leave the country and apply for citizenship like everyone else -0.87 The U.S. should turn away migrants in the caravan and actively deter them from -0.82 entry Not concerned about undocumented immigrants taking jobs away from U.S. citi- 0.41 zens Standardized loadings for ﬁrst principal component based upon correlation matrix (with varimax rotation). First principal component explains 57% of the variance on the three items.

1.4.2 Agenda-setting effects To estimate agenda-setting effects, we first identify which issues are deemed as most important by Republicans but not Democrats, and vice versa, by computing an index of partisan asymmetry in topic importance for each issue. We build this index for each issue included in the “most important problem” question in Waves 2 and 3 (responses are aggregated to improve our estimates), and then for each issue we subtract the proportion of self-identified Republicans who selected that problem from the proportion of self-identified Democrats who selected it (including leaners). Figure 5 shows the results of this calculation: issues with high scores are those that most concern Democrats but not Republicans (e.g., inequality, women’s rights, racism), whereas issues with low scores concern Republicans more than Democrats (e.g., terrorism, fake news, immigration). The outcome variable in our analysis measuring agenda-setting effects will be the average partisan asymmetry score of the problems that a given respondent marked as important in Wave 4. By taking the average, we control for the fact that some respondents may be likely to mark more items than others.

1.4.3 News browsing behavior Pre-treatment Pulse measures are computed using data from the month of September 2018 and up to the point that respondents take Wave 3. News sources are identiﬁed based on the list of top 500 domains shared on Facebook and compiled in Bakshy et al. (2015). Visits to news sources will be measured as the (logged) individual-level count of visits to URLs whose domain is included in this list. We classify domains as liberal or conservative based on their audience ideology score (negative for liberal, positive for conservative), which was computed by Bakshy et al. (2015) based on the self-reported political identity of Facebook users that shared those URLs on Facebook.

11 Figure 5: Partisan asymmetry in topic importance

Donald Trump and his administration Inequality Women's rights Racism Gun control Alt−right movement Relationship with Western countries Health care Political polarization Black Lives Matter Free speech Economy/unemployment Identity politics Relationship with North Korea Morality and values Crime Intl trade imbalances Immigration Fake news Islam Terrorism −0.25 0.00 0.25 0.50 0.75 Partisan asymmetry in agenda setting, by topic

Visits to conservative or liberal news sources are measured as logged count of URLs visited from each of the two groups described above, with an additional control for the logged total number of visits. Duplicate visits to webpages are not counted if they are successive (i.e., a page that was reloaded after ﬁrst opening it). URLs are cleaned of referrer information and other parameters before de-duplication. Pulse mobile visit data is included in our calculations, although we acknowledge that the data may have less coverage in our sample.

1.4.4 Media trust The first set of media trust items asked respondents to report how much they trust Huffington Post and Fox News when it comes to reporting the news about government and politics. The outcome variable was recoded from the categorical responses (“A great deal”, “a fair amount”, “not very much”, and “not at all”) to numeric scores (4 to 1). We operationalize trust in mainstream sources using the survey item in Wave 4, “How much trust and confidence do you have in the press when it comes to reporting the news about government and politics fully, accurately, and fairly?” The outcome variable was recoded from the categorical responses (“A great deal”, “a fair amount”, “not very much”, and “not at all”) to numeric scores (4 to 1). Finally, we measure perceived media slant using the item in Wave 4, “In presenting the news dealing with political and social issues, do you think that news organizations deal fairly with all sides, or do they tend to favor one side?” We subtract 3 from the numerical responses (which range from 1, “Tend to favor the liberal side,” to 5, “Tend to favor the conservative side”) so that perceptions of liberal slant are negative and perceptions of conservative slant are positive.

12 2 Descriptive statistics

Table 6: Characteristics of experimental sample (N = 1, 098)

Level N % Party ID Democrat 416 37.9 Republican 285 26.0 Independent 334 30.4 Gender Male 535 48.7 Female 563 51.3 Race White 896 81.6 Black 84 7.7 Hispanic 49 4.5 Asian / other 69 6.3 Education level No HS 17 1.5 High school graduate 162 14.8 Some college 247 22.5 2-year 157 14.3 4-year 283 25.8 Post-grad 232 21.1 Age group 18-29 58 5.3 30-44 198 18.0 45-59 294 26.8 60+ 548 49.9

2.1 Covariate balance The tables below report covariate balance between treatment and control groups. Each row is a weighted difference in means and associated two-sided p-value (computed from HC2 robust standard errors) for each demographic or behavioral attribute. We use weighted differences in means to match our analytic approach elsewhere and to take into account the blocked randomization by pre-treatment browser use. For the Fox News treatment group vs. control (Table 7), none of the differences are statistically signiﬁcant. For the HuffPost treatment group vs. control (Table 8), three characteristics (age, gender, and ideology) have differences that cross the threshold of statistical signiﬁcance at the 95% level. Looking at the total number of comparisons across both treatments, however, this is within the range of what might be expected due to chance. We address potential confounding issues by including these variables for possible selection in all covariate-adjusted models.

13 Table 7: Balance: Fox News treatment vs. Control

Covariate Difference (T–C) p Party ID 0.080 0.616 Age -0.956 0.374 Prop. female 0.034 0.352 Education -0.025 0.806 Ideology -0.038 0.694 Income -0.155 0.524 TV news freq. 0.112 0.447 Print newspaper freq. 0.155 0.279 Radio news freq. 0.286 0.071 Internet news freq. 0.015 0.902 Discussion freq. 0.043 0.740 Log news visits -0.037 0.824 Media diet slant 0.014 0.261

Differences and p-values from the weighted average of within-block differences-in-means, where blocks are de- ﬁned by pre-treatment web browser.

Table 8: Balance: HuffPost treatment vs. Control

Covariate Difference (T–C) p Party ID -0.242 0.131 Age -2.295 0.035 Prop. female 0.073 0.048 Education -0.111 0.294 Ideology -0.208 0.034 Income -0.098 0.694 TV news freq. -0.023 0.877 Print newspaper freq. 0.012 0.935 Radio news freq. 0.013 0.934 Internet news freq. -0.073 0.568 Discussion freq. -0.011 0.933 Log news visits 0.084 0.617 Media diet slant -0.018 0.153

Differences and p-values from the weighted average of within-block differences-in-means, where blocks are de- ﬁned by pre-treatment web browser.

14 2.2 Compliance A key challenge in our study is that we may suffer from two-sided noncompliance. On one hand, some respondents may already be heavy consumers of either Fox News or HuffPost, in which case receiving the encouragement is unlikely to exogenously increase their consumption of these outlets — they are “always-takers.” On the other, and perhaps more worryingly for our study, some respondents may agree to join the treatment group but then decline to follow the instructions or switch back their browser homepage after a few days — they would be “never- takers.” This section offers a summary of the extent to which compliance with our encouragement could be a concern. We seek to answer three different questions: (1) what proportion of subjects complied with the treatment? (2) are there any differences in compliance across treatment groups and treatment implementations or over time? (3) are there any differences regarding which subjects are more likely to comply with the treatment? To answer the first question, we use our pre-registered definition of “treated,” detailed above. Based on this definition, we find that the share of always-takers is 1.3% for the HuffPost treatment and 4.6% for the Fox News treatment. The proportion of treated respondents in each treatment group is 30.3% and 33.6%, respectively. Based on this, we estimate that the compliance rate was 28.7% for the HuffPost treatment and 29.0% for the Fox News treatment. In other words, our encouragement resulted in a clear exogenous increase in exposure to news from these outlets. Figure 6 offers a visual way to measure compliance in our sample. Here, we display the absolute change in the number of visits to Fox News and HuffPost in the six weeks after our treatment compared to the number of visits to the same outlet in the pre-treatment period. We find a clear difference between the treatment groups (in red or green) and the control groups (in gray): 57% of the respondents in the Fox News treatment group increased the frequency of their visits to this outlet with respect to the pre-treatment values and 50% of the respondents in the HuffPost treatment group increased their visits. In contrast, the graph does not show any meaningful change in visits for the control group. We also observe compliance even among users that had not visited these outlets before: Among respondents with 0 visits prior to the encouragement, 35% of those in the Fox News group visited this outlet for the first time as a result of our encouragement, and 23% of those in the HuffPost treatment group did so. In addition, the treatment was strong in nature for many of our respondents: 35% of the Fox News group and 30% of the HuffPost group increased the number of visits to these outlets more than tenfold compared to pre-treatment values. The second question we seek to answer is how compliance varied across different treatment dimensions. First, we explore with greater granularity how compliance varied across treatment groups and over time. Figures 7 and 8 clearly shows how the increased exposure is clearly due to our encouragement (denoted as a vertical line in both plots), how there is no spillover across treatment groups, and how the effects are durable nearly two months after the encouragement, although with some decay. (Note that the gap in late November corresponds to two days for

15 Figure 6: Compliance: Visualizing the Strength of Treatment

FoxNews HuffPost

400 400

200 200

0 0 Change in visits to news domain Change in visits to news

−200 −200 Respondents (from lowest to highest change in visit count)

Treatment group Control FoxNews HuffPost

Note: Respondents for whom change was above 1,000 visits (19 for Fox News, 1 for HuffPost) or below -200 visits (5 for Fox News) are considered outliers and excluded from the graph to facilitate its interpretation. which we do not have web tracking data.) Figures 9 and 10 help us understand how the different encouragements we implemented (changing default browser homepage, subscribing to a newsletter, and following the Facebook page of each outlet) may have contributed separately to the exogenous increase we document in the main text. As shown in Figure 9, we also ﬁnd an increase in visits to Fox News and Huff- Post from respondents’ mobile devices. This suggests that it was not only the browser homepage change that had any effect, since our encouragement only referred to respondents’ desktop com- puters. Similarly, Figure 10 also shows an increase in visits to facebook.com/FoxNews and facebook.com/HuffPost after our encouragement, which again suggests that our Facebook encouragement may have also contributed to increasing respondents’ exposure to content from these two sites. We also tracked clicks to the links we provided to the email newsletter sign-up pages, although we could not determine whether or not respondents actually registered to receive emails. According to this measure, 76% in the Fox News group and 78% in the HuffPost group clicked to sign up. Our third and ﬁnal question is how individual-level characteristics may have affected respondents’ levels of compliance with the encouragement. It is important to understand who complied with the encouragement since it represents the subgroup that the CACE estimates are informative about. We approach this question in two different ways. First, Figure 11 shows

16 Figure 7: Compliance: Fox News treatment

Change in visits to FoxNews.com per day after treatment

0.4 Control group 0.3

0.2

0.1

0.0

0.4 Fox News group

0.3

0.2

0.1

0.0

0.4 HuffPost group

0.3

0.2

0.1

0.0

Average count of FoxNews.com visits per day (log; 95% CIs) visits per day count of FoxNews.com Average Sep Oct Nov

Treatment group Control group Fox News group HuffPost group

the coefficients of two multivariate regressions (one for each of the two encouragement groups) where the dependent variable is log visits to the assigned news site, and independent variables include pre-treatment visits to the two sites as well as other covariates that may plausibly predict treatment uptake. Not surprisingly, in both cases we find that respondents that already visited many news sites before the encouragement (and thus were probably more interested in news) were also more likely to take up the treatment. For the Fox News encouragement, we find that ideology seems to be an important predictor: conservative respondents increased their visits to Fox News twice as much as liberals. For the HuffPost encouragement, we do not find a similar effect for ideology, but we do find that treatment uptake is higher among those that were already frequent visitors to this site. Second, we implement the profiling method suggested by Marbach and Hangartner (2020) for the principal strata. This allows us to separate compliers from always-takers and never- takers, who might contaminate the previous analysis. Figure 12 reports point estimates along with 83%/95% bootstrap confidence intervals on various covariates for the entire sample, compliers, always-takers, and never-takers. Compared to the entire sample, compliers in both treatment groups visited news sites prior to encouragement and reported very high political interest more often. There are no substantively large education, age, or income differences. Interest- ingly, while compliers with the Fox News encouragement tend to be more conservative than the overall sample average, the same applies for compliers with the HuffPost encouragement, who also tend to be less liberal than the overall sample. The ideological differences are much more pronounced in the always-taker groups, in the expected direction: The share of FoxNews

17 Figure 8: Compliance: HuffPost treatment

Change in visits to HuffPost.com per day after treatment Control group 0.2

0.1

0.0 Fox News group 0.2

0.1

0.0 HuffPost group 0.2

0.1

0.0

Average count of HuffingPost.com visits per day (log; 95% CIs) visits per day count of HuffingPost.com Average Sep Oct Nov

Treatment group Control group Fox News group HuffPost group

always-takers with liberal ideology is estimated zero but 60% for HuffPost always-takers. Con- versely, the share of FoxNews always-takers with conservative ideology is estimated 90% and zero for HuffPost always-takers. An important implication of these findings is that our encouragement did not just deliver more conservative (liberal) news content to conservative (liberal) minds, but also induced exposure to partisan news for ideologically averse consumers. Impor- tantly, however, compliers tended to consume more news and reported higher levels of political interest in the first place. Our complier profiling analysis is limited to the characteristics that we are able to observe. Thus, while Figure 12 reassures in its conclusion that compliers did not consume substantially more partisan news than the sample as a whole, it does not speak to differences along unob- served dimensions. Specifically, our Pulse sample is composed of participants from a larger survey pool who opted in to passive metering — an additional selection step that could alter the sample composition. Though Guess et al. (2020) document that attitudes related to online privacy do not appear to be strongly related to participation in Pulse (as compared to the general survey pool), it is possible that attributes such as these could affect both selection into the Pulse sample and compliance with our treatments. For this reason, the compliers in our experiment can be interpreted as the subset among Pulse participants who would visit Fox or HuffPost if and only if encouraged to do so. While this is not an insignificant share of that sample, we do not claim that this group is representative of the general population.

18 Figure 9: Compliance: mobile-only visits

Visits to FoxNews.com Visits to HuffingtonPost.com 0.08

0.09 Control group 0.06 0.06 0.04 0.03 0.02 0.00 0.00

0.08 0.09 Fox News group 0.06 0.06 0.04 0.03 0.02 0.00 0.00

0.08 0.09 HuffPost group 0.06 0.06 0.04 0.03 0.02

Average count of total web visits per day (95% CIs, log) (95% CIs, visits per day count of total web Average 0.00 0.00 −4 −3 −2 −1 1 2 3 4 5 6 7 −4 −3 −2 −1 1 2 3 4 5 6 7

Weeks before and after treatment was administered

Figure 10: Compliance: Fox News or HuffPost Facebook pages

% of panelists visiting Facebook pages URLs each day, by treatment group

facebook.com/FoxNews facebook.com/HuffPost

● 2.00%

1.50%

●

● ● ● ● ● ● 1.00% ● ● ● ● ● ●

● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ●●● ● ● ● ● ● ● ● ●● ● ●● ●● ●● ● ●● ●● ●● ● ● ● ●● ● ●● ● ● ● ●●● 0.50% ●● ● ●● ● ● ● ● ● ● ●● ●● ● ● ●

0.00% ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● Sep Oct Nov Dec Sep Oct Nov Dec

Treatment group ● Control ● Fox News ● HuffPost

19 Figure 11: Individual-level predictors of treatment uptake

DV = log visits to Fox News/HuffPost, during 4 weeks after encouragement

Fox News treatment group HuffPost treatment group

Intercept ● ●

Pre−treatment visits ● ● to news sites (log)

Pre−treatment visits ● ● to Fox News (log)

Pre−treatment visits ● ● to HuffPost (log)

Ideology = Liberal ● ●

Ideology = Conservative ● ●

Age >= 60 ● ●

Gender = female ● ●

Educ: HS or less ● ●

Educ: 4−year college or more ● ●

Political interest = very high ● ●

Above median hh income ● ●

−0.5 0.0 0.5 1.0 −0.5 0.0 0.5 1.0 Estimated % increase in visits during four weeks after encouragement

20 Figure 12: Mean estimates and 68%/95% bootstrap conﬁdence intervals on various covariates for the principal strata, following the proﬁling method by Marbach and Hangartner (2020).

21 3 Full results

In this section we report the results of testing each of our pre-registered hypotheses. For each of them, we report four quantities of interest:

1. The unadjusted difference in means (Intent-To-Treat estimate or ITT) in the outcome variable between treatment and control groups, as well as the standard error (in parentheses) and a 95% conﬁdence interval (in square brackets).

2. The covariate-adjusted ITT estimate computed using the Lin (2013) saturated regression approach, as well as the standard error (in parentheses) and a 95% conﬁdence interval (in square brackets).

3. The Complier Average Causal Effect (CACE) estimates, computed using the instrumental variables framework where compliance (measured using the criteria described in Sec- tion 1.3) is instrumented with the treatment assignment. We also report the standard error (in parentheses) and a 95% conﬁdence interval (in square brackets).

4. The Minimum Detectable Effect (MDE) for the covariate-adjusted ITT estimate, which we compute based on the % of variance reduced by the covariate adjustment assuming a test with 80% power for a level of signiﬁcance of 0.05. We also report in parentheses the equivalent Cohen’s d, computed by dividing the MDE by the standard deviation of the outcome variable. When interpreting this value, we follow the standard convention of assuming that d ≤ 0.20 corresponds to a small effect.

For our research questions regarding heterogeneous effects — for which we didn’t have clear expectations — we report the coefficient’s sign and significance (not significant, positive, or negative) for the interaction effect between treatment indicator and moderator (e.g., ideology). These interactive models are computed using the covariate-adjusted ITT model as a baseline and then adding the interaction as an additional term. For models where we find interaction effects that are statistically significant, we graphically display the estimated marginal effects for each value of the moderator. The tables on the next two pages summarize the results of each pre-specified hypothesis test (26 in Study 1, 24 in Study 2, excluding research questions), according to the covariate-adjusted ITT estimate. “Support” indicates that the null hypothesis of no effect was rejected at the 0.05 level (two-sided, no adjustment for multiple comparisons).

22 Table 9

Hypothesis Specific outcome Treatment Study Support? Issue Opinions H1 Issue scale (conservatism) Fox News Study 1 Issue Opinions H3 Immigration issue scale Fox News Study 1 News Browsing H1 Conservative news visits Fox News Study 1 ! Social Media H1 Conservative news shares Fox News Study 1 Social Media H3 Conservative news follows Fox News Study 1 Affective Polarization H2a Rate Dems Fox News Study 1 Affective Polarization H2b Rate Reps Fox News Study 1 Affective Polarization H2c Rate Trump supporters Fox News Study 1 Perceived Polarization H1a Fox News Study 1 Agenda Setting H1 Issue importance index Fox News Study 1 Elite Approval H1a Trump approval Fox News Study 1 Elite Approval H1b Congress (Rep) Fox News Study 1 Elite Approval H1c Congress (Dem) Fox News Study 1 Media Trust H1a Trust in Fox News Fox News Study 2 Media Trust H1b Trust in HuffPost Fox News Study 2 ! Media Trust H3a Perceived bias Fox News Study 2 Media Trust H4a Mainstream media trust Fox News Study 2 ! Factual Knowledge H1 % foreign born Fox News Study 2 Election Prediction H1a Predict GOP win House Fox News Study 2 Election Prediction H1b Predict GOP win vote share Fox News Study 2 Election Prediction H1c Predict GOP win district Fox News Study 2 Mail-Bombing H1a False flag Fox News Study 2 Mail-Bombing H1b False flag (vs. HuffPost) Fox News Study 2 Mail-Bombing H2a Media bias responsible Fox News Study 2 Mail-Bombing H2b Media bias responsible (vs. HuffPost) Fox News Study 2 Mail-Bombing H3a Trump responsible Fox News Study 2 Mail-Bombing H3b Trump responsible (vs. HuffPost) Fox News Study 2

23 Table 10

Hypothesis Speciﬁc outcome Treatment Study Support? Issue Opinions H2 Issue scale (conservatism) HuffPost Study 1 Issue Opinions H4 Immigration issue scale HuffPost Study 1 News Browsing H2 Liberal news visits HuffPost Study 1 Social Media H2 Liberal news shares HuffPost Study 1 Social Media H4 Liberal news follows HuffPost Study 1 Affective Polarization H1a Rate Dems HuffPost Study 1 Affective Polarization H1b Rate Reps HuffPost Study 1 ! Affective Polarization H1c Rate Trump supporters HuffPost Study 1 Perceived Polarization H1b HuffPost Study 1 Agenda Setting H2 Issue importance index HuffPost Study 1 Elite Approval H2a Trump approval HuffPost Study 1 Elite Approval H2b Congress (Rep) HuffPost Study 1 Elite Approval H2c Congress (Dem) HuffPost Study 1 Media Trust H2a Trust in HuffPost HuffPost Study 2 Media Trust H2b Trust in Fox News HuffPost Study 2 Media Trust H3b Perceived bias HuffPost Study 2 Media Trust H4b Mainstream media trust HuffPost Study 2 Factual Knowledge H2 % foreign born HuffPost Study 2 Election Prediction H2a Predict Dems win House HuffPost Study 2 Election Prediction H2b Predict Dems win vote share HuffPost Study 2 Election Prediction H2c Predict Dems win district HuffPost Study 2

24 3.1 Issue opinions Pre-registered hypotheses:

• Issue Opinions H1: Subjects assigned to the Fox News treatment will provide more conservative responses on average to questions about opinions on domestic and foreign issues, as well as policy trade-offs, than subjects assigned to the control group.

• Issue Opinions H2: Subjects assigned to the HuffPost treatment will provide more liberal responses on average to questions about opinions on domestic and foreign issues, as well as policy trade-offs, than subjects assigned to the control group.

• Issue Opinions H3: Subjects assigned to the Fox News treatment will report less pro- immigration attitudes than subjects assigned to the control group.

• Issue Opinions H4: Subjects assigned to the HuffPost treatment will not report less pro- immigration attitudes than subjects assigned to the control group.

• Issue Opinions RQ1: We will test for heterogeneous effects in the above hypotheses by self-reported party and ideology (including leaners). We do not generally expect to ﬁnd “backﬁre” effects (positive effects for some subgroups but negative for others) but are agnostic about potential differential updating.

Table 11: Results: issue opinions hypotheses 1–4

Outcome Unadjusted ITT Adjusted ITT CACE MDE H1: Conservatism 0.025 (0.086) -0.044 (0.036) -0.130 (0.165) 0.072 (FoxNews) [-0.145, 0.194] [-0.114, 0.027] [-0.454, 0.193] (d=0.070) H2: Conservatism -0.110 (0.086) -0.035 (0.036) -0.066 (0.140) 0.072 (HuffPost) [-0.278, 0.058] [-0.106, 0.036] [-0.341, 0.209] (d=0.070) H3: Pro-immigration 0.073 (0.077) 0.032 (0.048) 0.282 (0.210) 0.097 (FoxNews) [-0.078, 0.224] [-0.063, 0.126] [-0.130, 0.695] (d=0.095) H4: Pro-immigration 0.227 (0.076) 0.072 (0.049) 0.480 (0.221) 0.097 (HuffPost) [0.078, 0.377] [-0.024, 0.167] [0.045, 0.915] (d=0.096)

Each column reports: (1) unadjusted difference-in-means estimate, (2) covariate-adjusted ITT estimate, (3) Com- plier Average Causal Effect estimate using an IV framework, and (4) Minimum Detectable Effect assuming power=0.80 and α=0.05 in the covariate-adjusted ITT estimate. Standard errors are reported in parentheses and 95% CIs are reported in square brackets.

25 Table 12: Results: issue opinions RQ1

Outcome Party ID Ideology H1: Conservatism (Fox News) n.s. n.s. H2: Conservatism (HuffPost) n.s. n.s. H3: Pro-immigration (Fox News) n.s. n.s. H4: Pro-immigration (HuffPost) n.s. n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

26 3.2 Subsequent news browsing behaviors Pre-registered hypotheses:

• News Browsing H1: Subjects assigned to the Fox News treatment will be more likely to visit conservative news sources than subjects assigned to the control group.

• News Browsing H2: Subjects assigned to the HuffPost treatment will be more likely to visit liberal news sources than subjects assigned to the control group.

• News Browsing RQ1: We will test for heterogeneous effects by self-reported party and ideology (including leaners). We do not expect to ﬁnd “backﬁre” effects (positive effects for some subgroups but negative for others) but are agnostic about potential differential magnitudes in effects.

• News Browsing RQ2: We will test for heterogeneous effects by pre-treatment news habits (“patient preference”). We might expect smaller effects for those who already used Huff- Post or Fox News according to pre-treatment Pulse data.

Table 13: Results: News browsing hypotheses 1–2

Outcome Unadjusted ITT Adjusted ITT CACE MDE 1 week after treatment H1: Cons. news 0.418 (0.137) 0.404 (0.079) 1.448 (0.283) 0.156 (FoxNews) [0.149, 0.687] [0.248, 0.560] [0.892, 2.005] (d=0.105) H2: Lib. news 0.038 (0.147) -0.149 (0.082) -0.446 (0.307) 0.156 (HuffPost) [-0.252, 0.328] [-0.309, 0.012] [-1.050, 0.158] (d=0.097) 4 weeks after treatment H1: Cons. news 0.454 (0.167) 0.501 (0.096) 1.769 (0.336) 0.185 (FoxNews) [0.126, 0.781] [0.311, 0.690] [1.109, 2.428] (d=0.100) H2: Lib. news -0.009 (0.169) -0.246 (0.086) -0.899 (0.337) 0.174 (HuffPost) [-0.342, 0.323] [-0.415, -0.078] [-1.562, -0.236] (d=0.091) 6 weeks after treatment H1: Cons. news 0.454 (0.173) 0.510 (0.100) 1.792 (0.346) 0.196 (FoxNews) [0.115, 0.793] [0.313, 0.706] [1.113, 2.472] (d=0.101) H2: Lib. news 0.043 (0.172) -0.184 (0.092) -0.736 (0.352) 0.187 (HuffPost) [-0.295, 0.382] [-0.365, -0.003] [-1.429, -0.044] (d=0.095)

27 Table 14: Results: News browsing RQ1 & RQ2 (one week after treatment)

Outcome Party ID Ideology Pre-treat. DV H1: Cons. news (Fox News) n.s. n.s. n.s. H2: Lib. news (HuffPost) n.s. n.s. n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

28 3.3 Subsequent social media behaviors Pre-registered hypotheses:

• Social Media H1: Subjects assigned to the Fox News treatment will be more likely to share URLs from conservative news sources than subjects assigned to the control group.

• Social Media H2: Subjects assigned to the HuffPost treatment will be more likely to share URLs from liberal news sources than subjects assigned to the control group.

• Social Media H3: Subjects assigned to the Fox News treatment will be more likely to follow conservative news sources on Twitter than subjects assigned to the control group.

• Social Media H4: Subjects assigned to the HuffPost treatment will be more likely to follow liberal news sources on Twitter than subjects assigned to the control group.

Table 15: Results: social media hypotheses 1–4

Outcome Unadjusted ITT Adjusted ITT CACE MDE H1: Tweets w/cons. links -0.217 (0.175) 0.019 (0.086) 0.293 (0.496) 0.144 (FoxNews) [-0.564, 0.131] [-0.151, 0.190] [-0.697, 1.282] (d=0.159) H2: tweets w/lib. links 0.013 (0.191) 0.059 (0.068) 0.473 (0.293) 0.178 (HuffPost) [-0.364, 0.390] [-0.076, 0.194] [-0.108, 1.054] (d=0.173) H3: follow cons. media -0.063 (0.084) -0.066 (0.057) -0.390 (0.327) 0.114 (FoxNews) [-0.229, 0.103] [-0.178, 0.047] [-1.036, 0.257] (d=0.180) H4: follow lib. media 0.212 (0.111) 0.020 (0.067) -0.322 (0.314) 0.131 (HuffPost) [-0.006, 0.430] [-0.113, 0.152] [-0.943, 0.300] (d=0.149)

29 3.4 Affective polarization Pre-registered hypotheses:

• Affective Polarization H1: Subjects in the HuffPost treatment will rate Democrats more highly and Republicans and Trump supporters less highly than those in the control group.

• Affective Polarization H2: Subjects in the Fox News treatment will rate Democrats less highly and Republicans and Trump supporters more highly than those in the control group.

• Affective Polarization RQ1: We will test for heterogeneous effects by self-reported party and ideology (including leaners). We do not generally expect to ﬁnd “backﬁre” effects (positive effects for some subgroups but negative for others) but are agnostic about potential differential magnitudes in effects.

Table 16: Results: affective polarization hypotheses 1–2

Outcome Unadjusted ITT Adjusted ITT CACE MDE H1a: Dem. thermometer 3.505 (2.660) -0.521 (1.032) -1.379 (4.536) 2.033 (HuffPost) [-1.717, 8.728] [-2.547, 1.506] [-10.293, 7.535] (d=0.058) H1b: Rep. thermometer -4.420 (2.642) 2.727 (1.111) 11.925 (4.826) 2.130 (HuffPost) [-9.607, 0.766] [0.545, 4.909] [2.440, 21.410] (d=0.061) H1c: Trump distance 0.358 (0.169) 0.073 (0.105) 0.476 (0.450) 0.207 (HuffPost) [0.026, 0.689] [-0.133, 0.279] [-0.408, 1.361] (d=0.091) H2a: Dem. thermometer -1.132 (2.687) -0.531 (1.071) -6.797 (4.598) 2.029 (FoxNews) [-6.409, 4.144] [-2.633, 1.571] [-15.831, 2.238] (d=0.058) H2b: Rep. thermometer -0.355 (2.687) 1.361 (1.042) 4.798 (4.612) 2.119 (FoxNews) [-5.630, 4.921] [-0.684, 3.407] [-4.266, 13.862] (d=0.061) H2c: Trump distance -0.048 (0.176) -0.019 (0.108) -0.055 (0.455) 0.206 (FoxNews) [-0.392, 0.297] [-0.231, 0.193] [-0.949, 0.839] (d=0.090)

30 Table 17: Results: Affective polarization RQ1

Outcome Party ID Ideology H1a: Dem. thermometer (HuffPost) n.s. n.s. H1b: Rep. thermometer (HuffPost) n.s. n.s. H1c: Trump distance (HuffPost) n.s. n.s. H2a: Dem. thermometer (FoxNews) n.s. n.s. H2b: Rep. thermometer (FoxNews) n.s. n.s. H2c: Trump distance (FoxNews) n.s. n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

31 3.5 Perceived polarization Pre-registered hypotheses:

• Perceived Polarization H1: Subjects assigned to either treatment condition will perceive more political polarization than those in the control group.

• Perceived Polarization RQ1: We will test for heterogeneous effects by self-reported party (including leaners).

Table 18: Results: perceived polarization hypothesis 1

Outcome Unadjusted ITT Adjusted ITT CACE MDE H1a: Perceived polarization 0.086 (0.061) 0.086 (0.061) 0.339 (0.254) 0.120 (FoxNews) [-0.033, 0.205] [-0.033, 0.205] [-0.161, 0.839] (d=0.151) H1b: Perceived polarization 0.068 (0.060) 0.068 (0.060) 0.259 (0.265) 0.120 (HuffPost) [-0.049, 0.186] [-0.049, 0.186] [-0.261, 0.780] (d=0.151)

Table 19: Results: Perceived polarization RQ1

Outcome Party ID H1a: Perceived polariz. (HuffPost) n.s. H1b: Perceived polariz. (HuffPost) n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

32 3.6 Agenda-setting Pre-registered hypotheses:

• Agenda Setting H1: Subjects assigned to the Fox News treatment will tend to rate as important those issues that Republican respondents are more likely than Democrats to identify as important, compared to the control group.

• Agenda Setting H2: Subjects assigned to the HuffPost treatment will tend to rate as important those issues that Democratic respondents are more likely than Republicans to identify as important, compared to the control group.

• Agenda Setting RQ1: We will test for heterogeneous effects by self-reported party and ideology (including leaners).

• Agenda Setting RQ2: We will test for heterogeneous effects by pre-treatment news habits (“patient preference”). We might expect smaller effects for those who already used Huff- Post or Fox News according to pre-treatment Pulse data.

Table 20: Results: agenda setting hypotheses 1–2

Outcome Unadjusted ITT Adjusted ITT CACE MDE H1: Agenda setting 0.003 (0.016) 0.006 (0.009) 0.031 (0.036) 0.017 (FoxNews) [-0.028, 0.034] [-0.011, 0.024] [-0.040, 0.101] (d=0.083) H2: Agenda setting 0.033 (0.016) 0.007 (0.009) 0.040 (0.037) 0.017 (HuffPost) [0.003, 0.064] [-0.010, 0.025] [-0.032, 0.113] (d=0.083)

Table 21: Results: Agenda setting RQ1 & RQ2

Outcome Party ID Ideology Pre-treatment news H1: Agenda setting (FoxNews) n.s. n.s. n.s. H2: Agenda setting (HuffPost) n.s. n.s. n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

33 3.7 Approval of President/Congress Pre-registered hypotheses:

• Elite Approval H1: Subjects assigned to the Fox News treatment will be more supportive of President Trump and the Republican Party and less supportive of the Democratic Party than subjects assigned to the control condition.

• Elite Approval H2: Subjects assigned to the HuffPost treatment will be less supportive of President Trump and the Republican Party and more supportive of the Democratic Party than subjects assigned to the control condition.

• Elite Approval RQ1: We will test for heterogeneous effects by self-reported party and ideology (including leaners).

Table 22: Results: Elite approval hypotheses 1–2

Outcome Unadjusted ITT Adjusted ITT CACE MDE H1a: Pres. approval 0.009 (0.135) 0.038 (0.034) 0.032 (0.147) 0.068 (FoxNews) [-0.256, 0.274] [-0.029, 0.105] [-0.258, 0.321] (d=0.039) H1b: Rep. Cong. pref. -0.001 (0.036) 0.001 (0.014) 0.010 (0.043) 0.028 (FoxNews) [-0.071, 0.070] [-0.026, 0.028] [-0.075, 0.095] (d=0.058) H1c: Dem. Cong. pref. 0.005 (0.037) 0.007 (0.020) 0.002 (0.065) 0.039 (FoxNews) [-0.067, 0.077] [-0.031, 0.046] [-0.125, 0.129] (d=0.079) H2a: Pres. approval -0.222 (0.132) 0.044 (0.035) 0.077 (0.156) 0.069 (HuffPost) [-0.481, 0.037] [-0.025, 0.114] [-0.229, 0.383] (d=0.039) H2b: Rep. Cong. pref. -0.054 (0.035) -0.003 (0.015) 0.006 (0.054) 0.028 (HuffPost) [-0.123, 0.015] [-0.032, 0.026] [-0.101, 0.113] (d=0.058) H2c: Dem. Cong. pref. 0.051 (0.036) -0.003 (0.020) 0.015 (0.070) 0.039 (HuffPost) [-0.021, 0.122] [-0.042, 0.037] [-0.122, 0.152] (d=0.079)

34 Table 23: Results: Elite approval RQ1

Outcome Party ID Ideology H1a: Pres. approval (FoxNews) n.s. n.s. H1b: Rep. Cong. pref. (FoxNews) n.s. n.s. H1c: Dem. Cong. pref. (FoxNews) n.s. n.s. H2a: Pres. approval (HuffPost) n.s. n.s. H2b: Rep. Cong. pref. (HuffPost) n.s. n.s. H2c: Dem. Cong. pref. (HuffPost) n.s. n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

35 3.8 Voting behavior Pre-registered hypotheses:

• Voting Behavior RQ1: We will test whether subjects assigned to either of the treatment conditions were more likely to exhibit higher turnout than subjects assigned to the control group.

Table 24: Results: Voting behavior RQ1

Outcome Unadjusted ITT Adjusted ITT CACE MDE RQ1a: Turnout -0.036 (0.028) -0.031 (0.024) -0.120 (0.099) 0.047 (FoxNews) [-0.091, 0.018] [-0.079, 0.017] [-0.315, 0.076] (d=0.129) RQ1b: Turnout 0.036 (0.025) 0.026 (0.022) 0.102 (0.090) 0.047 (HuffPost) [-0.014, 0.085] [-0.018, 0.070] [-0.075, 0.279] (d=0.129)

36 3.9 Media trust Pre-registered hypotheses:

• Media Trust H1a: Subjects assigned to the Fox News treatment will have more trust in Fox News than those in the control group.

• Media Trust H1b: Subjects assigned to the Fox News treatment will not have more trust in HuffPost than those in the control group.

• Media Trust H2a: Subjects assigned to the HuffPost treatment will have more trust in HuffPost than those in the control group.

• Media Trust H2b: Subjects assigned to the HuffPost treatment will not have more trust in Fox News than those in the control group.

• Media Trust H3a: Subjects assigned to the Fox News treatment will be more likely to believe that news organizations tend to favor liberals than subjects assigned to control.

• Media Trust H3b: Subjects assigned to the HuffPost treatment will be more likely to believe that news organizations tend to favor conservatives than subjects assigned to control.

• Media Trust RQ1: We will test for heterogeneous effects in the above hypotheses by self-reported party and ideology (including leaners). We do not expect to ﬁnd “backﬁre” effects (positive effects for some subgroups but negative for others) but are agnostic about potential differential magnitudes in effects.

• Media Trust H4: Subjects assigned to both treatments will have less trust in mainstream sources than those in the control group.

• Media Trust RQ2: Will the magnitude of the effect in H3 and H4 be greater for subjects assigned to Fox News (vs. control) than for those assigned to HuffPost (vs. control)?

37 Table 25: Results: Media trust hypotheses 1–3

Outcome Unadjusted ITT Adjusted ITT CACE MDE H1a: Trust in Fox News 0.018 (0.084) 0.000 (0.060) -0.058 (0.260) 0.118 (FoxNews) [-0.146, 0.182] [-0.117, 0.118] [-0.569, 0.454] (d=0.108) H1b: Trust in HuffPost -0.131 (0.070) -0.133 (0.058) -0.815 (0.272) 0.112 (FoxNews) [-0.268, 0.007] [-0.246, -0.019] [-1.349, -0.280] (d=0.121) H2a: Trust in HuffPost 0.099 (0.071) -0.011 (0.057) -0.128 (0.248) 0.113 (HuffPost) [-0.040, 0.239] [-0.124, 0.102] [-0.615, 0.359] (d=0.122) H2b: Trust in Fox News -0.200 (0.082) -0.061 (0.061) -0.239 (0.266) 0.118 (HuffPost) [-0.361, -0.040] [-0.181, 0.058] [-0.762, 0.284] (d=0.109) H3a: Liberal media bias 0.051 (0.085) 0.057 (0.073) 0.191 (0.316) 0.146 (FoxNews) [-0.115, 0.218] [-0.087, 0.201] [-0.431, 0.812] (d=0.130) H3b: Liberal media bias 0.154 (0.084) 0.053 (0.074) 0.225 (0.327) 0.147 (HuffPost) [-0.012, 0.319] [-0.093, 0.199] [-0.418, 0.869] (d=0.130) H4a: Media trust -0.118 (0.071) -0.125 (0.053) -0.538 (0.232) 0.106 (FoxNews) [-0.257, 0.021] [-0.229, -0.022] [-0.994, -0.082] (d=0.112) H4b: Media trust 0.026 (0.072) -0.084 (0.054) -0.378 (0.223) 0.106 (HuffPost) [-0.116, 0.168] [-0.189, 0.022] [-0.817, 0.060] (d=0.113)

Table 26: Results: Media trust RQ1

Outcome Party ID Ideology H1a: Trust in Fox News (FoxNews) n.s. n.s. H1b: Trust in HuffPost (FoxNews) n.s. n.s. H2a: Trust in HuffPost (HuffPost) n.s. n.s. H2b: Trust in Fox News (HuffPost) n.s. n.s. H3a: Liberal media bias (FoxNews) + − H3b: Liberal media bias (HuffPost) n.s. n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

38 Figure 13: Estimated marginal effects of Fox News treatment on media bias (perception that media tends to favor the liberal side), by self-reported ideology and party ID.

interaction with party ID Interaction with ideology

0.5 ● ●

● ● ●

● ●

0.0 ●

● ●

● ● Estimated marginal effect (95% CI) Estimated marginal effect −0.5

Strong Not very Lean Independent Lean Not very Strong Very Liberal Moderate Conservative Very Democrat strong Democrat Republican strong Republican liberal conservative Democrat Republican Self−reported party ID or ideology

39 3.10 Knowledge Pre-registered hypotheses:

• Factual Knowledge H1: Subjects assigned to the Fox News condition will be more likely to overestimate the percent of the U.S. population that is foreign-born than those assigned to HuffPost or to control.

• Factual Knowledge H2: Subjects assigned to the HuffPost condition will be more likely to overestimate the unemployment rate than those assigned to Fox News or to control.

• Event Knowledge RQ1: Did subjects assigned to Fox News or HuffPost exhibit greater or less belief accuracy on questions about news reception of recent events compared to those assigned to control?

• Event and Factual Knowledge RQ2: We will test for heterogeneous effects in the above hypotheses by self-reported party and ideology (including leaners). We do not generally expect to ﬁnd “backﬁre” effects (positive effects for some subgroups but negative for others) but are agnostic about potential differential learning.

Following the pre-analysis plan, we employed Mokken scaling (Van Schuur 2003) to identify a subset of event items that measure the same underlying concept of event knowledge. As a result, three out of eight pre-treatment items from Wave 2 as well as three out of six post- treatment items from Wave 4 were selected. The outcome variable was then calculated as total number of correct answers divided by total number of questions asked in these subsets. The response items were: “Donald Trump and Kim Jong Un met in Singapore”, “Kim Kardashian met with President Trump in the Oval Ofﬁce”, “The trial against former Trump campaign chairman Paul Manafort began” (all Wave 2), “A caravan of Central American migrants crossed into Mexico with the eventual goal of reaching the United States”, “A Russian national was charged with attempting to interfere in the 2018 U.S. midterm elections”, and “Kanye West met in the Oval Ofﬁce with President Donald Trump” (all Wave 4).

40 Table 27: Results: Factual/event knowledge hypotheses 1–2 & RQ2

Outcome Unadjusted ITT Adjusted ITT CACE MDE H1: % foreign born -0.656 (1.393) 0.656 (1.393) -1.352 (5.553) 2.804 (FoxNews) [-3.391, 2.079] [-2.079, 3.391] [-12.264, 9.560] (d=0.152) H2: % unemployment 0.000 (1.319) 1.226 (1.238) 8.829 (5.663) 2.284 (HuffPost) [-2.596, 2.596] [-1.205, 3.657] [-2.298, 19.957] (d=0.152) RQ1a: Event knowledge 0.012 (0.005) 0.010 (0.005) 0.044 (0.020) 0.010 (HuffPost) [0.002, 0.022] [0.000, 0.020] [0.006, 0.083] (d=0.146) RQ1b: Event knowledge 0.016 (0.005) 0.017 (0.005) 0.059 (0.019) 0.010 (FoxNews) [0.005, 0.026] [0.007, 0.027] [0.022, 0.096] (d=0.146)

Table 28: Results: Knowledge RQ2

Outcome Party ID Ideology H1: % foreign born (FoxNews) n.s. n.s. H2: % unemployment (HuffPost) n.s. n.s. RQ1a: Event knowledge (HuffPost) n.s. n.s. RQ1b: Event knowledge (FoxNews) n.s. n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

41 3.11 Election prediction Pre-registered hypotheses:

• Election Prediction H1: Subjects assigned to the Fox News treatment will be more likely (a) to predict the Republican Party as winning the control of the House of Representatives, (b) to predict a higher national two-party vote share for the Republican Party, and (c) to predict the Republican candidate to win the race for the House seat in their Congressional district than subjects assigned to the control group.

• Election Prediction H2: Subjects assigned to the HuffPost treatment will be more likely (a) to predict the Democratic Party as as winning the control of the House of Represen- tatives, (b) to predict a higher national two-party vote share for the Democratic Party, and (c) to predict the Democratic candidate to win the race for the House seat in their Congressional district than subjects assigned to the control group.

• Election Prediction H3: Pre-treatment Republican partisans in the Fox News treatment group and pre-treatment Democratic partisans in the HuffPost treatment group will report a higher average certainty in their predictions than subjects assigned to the control group.

• Election Prediction H4: Pre-treatment Republican partisans in the HuffPost treatment group and pre-treatment Democratic partisans in the Fox News treatment group will report a lower average certainty in their predictions than subjects assigned to the control group.

42 Table 29: Results: Election prediction hypotheses 1–2

Outcome Unadjusted ITT Adjusted ITT CACE MDE H1a: GOP House winner -0.038 (0.038) -0.042 (0.031) -0.078 (0.138) 0.061 (FoxNews) [-0.112, 0.037] [-0.103, 0.020] [-0.349, 0.192] (d=0.122) H1b: GOP House vote share -0.754 (1.262) -1.151 (1.106) -0.690 (4.453) 2.126 (FoxNews) [-3.232, 1.725] [-3.324, 1.022] [-9.445, 8.065] (d=0.144) H1c: GOP House district winner -0.001 (0.038) -0.012 (0.031) -0.018 (0.144) 0.062 (FoxNews) [-0.075, 0.074] [-0.073, 0.050] [-0.300, 0.264] (d=0.124) H2a: DEM House winner 0.075 (0.037) 0.022 (0.031) 0.050 (0.135) 0.061 (HuffPost) [0.002, 0.149] [-0.038, 0.083] [-0.214, 0.314] (d=0.122) H2b: DEM House vote share 1.138 (1.209) 0.451 (1.068) 3.818 (4.447) 2.111 (HuffPost) [-1.237, 3.512] [-1.646, 2.548] [-4.924, 12.560] (d=0.143) H2c: DEM House district winner 0.023 (0.038) -0.023 (0.032) -0.024 (0.150) 0.063 (HuffPost) [-0.051, 0.097] [-0.086, 0.040] [-0.318, 0.271] (d=0.126)

Each column reports: (1) unadjusted difference-in-means estimate, (2) covariate-adjusted ITT estimate (not reported here due to data sparsity), (3) Complier Average Causal Effect estimate using an IV framework, and (4) Minimum Detectable Effect assuming power=0.80 and α=0.05 in the covariate-adjusted ITT estimate. Standard errors are reported in parentheses and 95% CIs are reported in square brackets.

Table 30: Results: Election prediction hypotheses 3–4

Outcome Unadjusted ITT Adjusted ITT CACE MDE H3: Prediction certainty 0.073 (0.175) 0.073 (0.175) -1.017 (1.452) 0.351 (Treatment) [-0.270, 0.416] [-0.270, 0.416] [-3.871, 1.837] (d=0.153) H4: Prediction certainty 0.136 (0.178) 0.136 (0.178) -0.558 (1.826) 0.363 (Treatment) [-0.215, 0.486] [-0.215, 0.486] [-4.147, 3.031] (d=0.158)

43 3.12 Mail-bombing incident We included a question in Wave 4 to test whether differential exposure to partisan media would affect how respondents perceived and assigned blame for a controversial political event of uncertain cause. We used the case of a mail-bombing spree in which a series of mail bombs were sent to prominent Democrats and critics of President Trump in the fall of 2018. The inci- dents spurred false-ﬂag conspiracy theories that the bombs were sent to cast blame on Trump supporters. Our question asked: “Recently, prominent Democrats and critics of President Trump, including Barack Obama, Hillary Clinton, George Soros, and Robert De Niro, were the apparent targets of a mail-bombing spree. Below are several explanations for the mail bombs with which some people agree while others do not. How about you? Please select the responses that come closest to your views.” Pre-registered hypotheses:

• Mail-Bombing H1: Subjects assigned to the Fox News treatment will be more likely to think the “mail-bombing” incident targeting Democrats was a “false ﬂag” event organized by liberals than those assigned to the control group and to the HuffPost treatment.

• Mail-Bombing H2: Subjects assigned to the Fox News treatment will be more likely to think mainstream media’s bias was responsible for the “mail-bombing” incident than those assigned to the control group and to the HuffPost treatment.

• Mail-Bombing H3: Subjects assigned to the Fox News treatment will be less likely to think Trump’s ﬁery rhetoric was responsible for the “mail-bombing” incident than those assigned to the control group and to the HuffPost treatment.

44 Table 31: Results: Mail-bombing incident hypotheses 1–3

Outcome Unadjusted ITT Adjusted ITT CACE MDE H1a: Mail-bombing is a false ﬂag -0.021 (0.105) 0.008 (0.089) 0.392 (0.394) 0.170 (FoxNews) [-0.226, 0.184] [-0.168, 0.183] [-0.381, 1.165] (d=0.126) H1b: Mail-bombing is a false ﬂag 0.098 (0.102) 0.008 (0.089) 8.121 (12.236) 0.170 (FoxNews) [-0.103, 0.299] [-0.168, 0.183] [-15.924, 32.167] (d=0.126) H2a: Media is accountable -0.018 (0.111) -0.036 (0.089) 0.192 (0.387) 0.177 (FoxNews) [-0.235, 0.200] [-0.211, 0.139] [-0.569, 0.952] (d=0.121) H2b: Media is accountable 0.188 (0.111) -0.036 (0.089) 3.365 (7.527) 0.177 (FoxNews) [-0.030, 0.405] [-0.211, 0.139] [-11.427, 18.157] (d=0.121) H3a: Trump is accountable -0.035 (0.129) -0.027 (0.082) -0.078 (0.355) 0.161 (FoxNews) [-0.288, 0.218] [-0.188, 0.134] [-0.775, 0.620] (d=0.095) H3b: Trump is accountable -0.140 (0.130) -0.027 (0.082) -1.363 (6.376) 0.161 (FoxNews) [-0.394, 0.114] [-0.188, 0.134] [-13.893, 11.167] (d=0.095)

45 4 Additional results: Wave 5

In this section we report the results of testing our hypotheses on Wave 5 outcomes. Our pre- analysis plan indicated that we would do this for Wave 5 where available. We follow the structure of the previous section in reporting the results.

4.1 Issue opinions

Table 32: Results: Issue opinions hypotheses 1–4 (W5)

Outcome Unadjusted ITT Adjusted ITT CACE MDE H1: Conservatism -0.020 (0.091) -0.055 (0.038) -0.197 (0.176) 0.075 (FoxNews) [-0.199, 0.158] [-0.130, 0.019] [-0.543, 0.149] (d=0.074) H2: Conservatism 0.024 (0.088) -0.037 (0.036) -0.183 (0.163) 0.073 (HuffPost) [-0.149, 0.198] [-0.107, 0.033] [-0.503, 0.137] (d=0.072) H3: Pro-immigration 0.042 (0.077) -0.004 (0.034) 0.169 (0.148) 0.068 (FoxNews) [-0.108, 0.193] [-0.072, 0.063] [-0.123, 0.460] (d=0.068) H4: Pro-immigration 0.177 (0.075) 0.012 (0.035) 0.227 (0.167) 0.068 (HuffPost) [0.029, 0.325] [-0.056, 0.080] [-0.101, 0.555] (d=0.068)

Table 33: Results: Issue opinions RQ1 (W5)

Outcome Party ID Ideology H1: Conservatism (Fox News) n.s. n.s. H2: Conservatism (HuffPost) n.s. n.s. H3: Pro-immigration (Fox News) n.s. n.s. H4: Pro-immigration (HuffPost) n.s. n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

46 4.2 Agenda-setting

Table 34: Results: Agenda-setting hypotheses 1–2 (W5)

Outcome Unadjusted ITT Adjusted ITT CACE MDE H1: Agenda setting -0.004 (0.016) 0.004 (0.009) 0.009 (0.036) 0.016 (FoxNews) [-0.035, 0.027] [-0.013, 0.021] [-0.063, 0.081] (d=0.080) H2: Agenda setting 0.025 (0.015) 0.006 (0.008) 0.001 (0.037) 0.016 (HuffPost) [-0.006, 0.055] [-0.010, 0.022] [-0.072, 0.074] (d=0.080)

Table 35: Results: Agenda setting RQ1 & RQ2 (W5)

Outcome Party ID Ideology Pre-treatment news H1: Agenda setting (FoxNews) n.s. n.s. n.s. H2: Agenda setting (HuffPost) n.s. n.s. n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

47 4.3 Approval of President

Table 36: Results: Elite approval hypotheses 1–2 (W5)

Outcome Unadjusted ITT Adjusted ITT CACE MDE H1a: Pres. approval -0.028 (0.135) 0.015 (0.038) -0.068 (0.157) 0.075 (FoxNews) [-0.293, 0.237] [-0.059, 0.090] [-0.376, 0.240] (d=0.043) H2a: Pres. approval -0.193 (0.131) 0.024 (0.039) -0.008 (0.182) 0.075 (HuffPost) [-0.450, 0.064] [-0.053, 0.100] [-0.366, 0.351] (d=0.043)

Table 37: Results: Elite approval RQ1 (W5)

Outcome Party ID Ideology H1a: Pres. approval (FoxNews) n.s. n.s. H2a: Pres. approval (HuffPost) n.s. n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

48 4.4 Media trust

Table 38: Results: Media trust hypotheses 1–2 (W5)

Outcome Unadjusted ITT Adjusted ITT CACE MDE H1a: Trust in Fox News -0.044 (0.084) -0.058 (0.061) -0.192 (0.268) 0.121 (FoxNews) [-0.209, 0.122] [-0.178, 0.063] [-0.718, 0.334] (d=0.110) H1b: Trust in HuffPost -0.042 (0.071) -0.039 (0.059) -0.183 (0.254) 0.117 (FoxNews) [-0.182, 0.098] [-0.154, 0.077] [-0.682, 0.316] (d=0.125) H2a: Trust in HuffPost 0.110 (0.073) 0.018 (0.061) -0.094 (0.282) 0.117 (HuffPost) [-0.032, 0.253] [-0.102, 0.138] [-0.647, 0.460] (d=0.125) H2b: Trust in Fox News -0.165 (0.084) -0.042 (0.061) -0.205 (0.288) 0.121 (HuffPost) [-0.330, -0.001] [-0.162, 0.078] [-0.772, 0.362] (d=0.110)

Table 39: Results: Media trust RQ1 (W5)

Outcome Party ID Ideology H1a: Trust in Fox News (FoxNews) n.s. n.s. H1b: Trust in HuffPost (FoxNews) + n.s. H2a: Trust in HuffPost (HuffPost) n.s. n.s. H2b: Trust in Fox News (HuffPost) n.s. n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

49 Figure 14: Estimated marginal effects of Fox News treatment on trust in HuffPost (Wave 5), by self-reported party ID.

0.3 ●

●

0.0 ●

●

● −0.3 Estimated marginal effect (95% CI) Estimated marginal effect

−0.6 Strong Not very Lean Independent Lean Not very Strong Democrat strong Democrat Republican strong Republican Democrat Republican Self−reported party ID

50 4.5 Knowledge Following the pre-analysis plan, we employed Mokken scaling (Van Schuur 2003) to identify a subset of event items that measure the same underlying concept of event knowledge. As a result, three out of eight pre-treatment items from Wave 2 and four out of six post-treatment items from Wave 5 were selected. The outcome variable was then calculated as total number of correct answers divided by total number of questions asked in these subsets. The response items were: “Donald Trump and Kim Jong Un met in Singapore”, “Kim Kardashian met with President Trump in the Oval Ofﬁce”, “The trial against former Trump campaign chairman Paul Manafort began” (all Wave 2), “A wildﬁre in California killed more than 80 people”, “Mark Zuckerberg stepped down from his role as Facebook’s CEO”, “Theresa May (UK’s then-prime minister) reached an agreement with European Union authorities about UK’s decision to leave the European Union”, and “U.S. border agents used tear gas against migrants who were trying to cross the border from Mexico” (all Wave 5).

Table 40: Results: Factual/event knowledge RQ1 (W5)

Outcome Unadjusted ITT Adjusted ITT CACE MDE RQ1a: Event knowledge 0.045 (0.055) 0.025 (0.052) 0.159 (0.232) 0.106 (HuffPost) [-0.064, 0.153] [-0.078, 0.128] [-0.296, 0.614] (d=0.142) RQ1b: Event knowledge -0.021 (0.059) -0.037 (0.055) -0.149 (0.218) 0.107 (FoxNews) [-0.136, 0.095] [-0.146, 0.071] [-0.578, 0.280] (d=0.143)

Table 41: Results: Knowledge RQ2 (W5)

Outcome Party ID Ideology RQ1a: Event knowledge (HuffPost) n.s. n.s. RQ1b: Event knowledge (FoxNews) n.s. n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

51 5 Additional results: Waves 7 and 8

In this section we report the results of testing our hypotheses on Wave 7 and 8 outcomes, where available. These analyses were not speciﬁed in our pre-analysis plan, which we indicate in the section below.

5.1 Issue opinions

Table 42: Results: Issue opinions hypotheses 1–4 (W7)

Outcome Unadjusted ITT Adjusted ITT CACE MDE H1: Conservatism -0.007 (0.087) -0.017 (0.039) -0.121 (0.178) 0.076 (FoxNews) [-0.178, 0.164] [-0.093, 0.059] [-0.471, 0.230] (d=0.076) H2: Conservatism -0.037 (0.088) 0.027 (0.038) 0.041 (0.162) 0.077 (HuffPost) [-0.210, 0.135] [-0.048, 0.102] [-0.278, 0.361] (d=0.077) H3: Pro-immigration 0.036 (0.079) -0.014 (0.034) 0.020 (0.152) 0.067 (FoxNews) [-0.119, 0.191] [-0.082, 0.053] [-0.280, 0.319] (d=0.068) H4: Pro-immigration 0.135 (0.078) -0.020 (0.035) -0.103 (0.160) 0.067 (HuffPost) [-0.018, 0.288] [-0.088, 0.048] [-0.417, 0.212] (d=0.068)

52 5.2 Agenda-setting

Table 43: Results: Agenda-setting hypotheses 1–2 (W8)

Outcome Unadjusted ITT Adjusted ITT CACE MDE H1: Agenda setting 0.010 (0.016) 0.013 (0.009) 0.091 (0.044) 0.017 (FoxNews) [-0.022, 0.042] [-0.005, 0.030] [0.005, 0.178] (d=0.085) H2: Agenda setting 0.029 (0.016) 0.005 (0.009) 0.028 (0.039) 0.018 (HuffPost) [-0.003, 0.061] [-0.013, 0.023] [-0.050, 0.105] (d=0.086)

5.3 Media trust

Table 44: Results: Media trust hypothesis 4 (W7)

Outcome Unadjusted ITT Adjusted ITT CACE MDE H4a: Media trust (W7) -0.099 (0.079) -0.132 (0.058) -0.562 (0.261) 0.115 (FoxNews) [-0.255, 0.056] [-0.245, -0.019] [-1.074, -0.049] (d=0.115) H4b: Media trust (W7) -0.049 (0.079) -0.165 (0.059) -0.721 (0.275) 0.115 (HuffPost) [-0.204, 0.106] [-0.281, -0.048] [-1.262, -0.180] (d=0.115)

53 Table 45: Results: Media trust hypothesis 4 (W8)

Outcome Unadjusted ITT Adjusted ITT CACE MDE H4a: Media trust (W8) -0.082 (0.082) -0.091 (0.057) -0.512 (0.267) 0.114 (FoxNews) [-0.243, 0.080] [-0.203, 0.022] [-1.036, 0.013] (d=0.112) H4b: Media trust (W8) 0.001 (0.082) -0.105 (0.060) -0.565 (0.267) 0.115 (HuffPost) [-0.160, 0.163] [-0.223, 0.014] [-1.091, -0.040] (d=0.113)

Figure 15: Mean estimates and 95% conﬁdence intervals on the effect of the treatments on mainstream media trust across survey waves (ITT in gray; CACE in black).

Media trust

Wave 4 Wave 7 Wave 8

0.0

−0.4

−0.8

−1.2

Fox News HuffPost Fox News HuffPost Fox News HuffPost

54 6 Additional results: Comparing two treatment groups

In this section, we report the results of additional (not pre-registered) statistical tests of the hypotheses and research questions. In our pre-analysis plan, we proposed to compare one of the two treatment groups (i.e., Fox News or HuffPost) with the control group in most tests (see Sec- tion 3). However, that approach may be too conservative if the experimental effects are not large enough to distinguish effects between a treatment group and the control group but are enough to distinguish the effects between the two treatment groups. Thus, we ran additional analyses to examine whether there are meaningful differences between the two treatment groups. In the following tests, (1) we treated the HuffPost condition as a treatment group and the Fox News condition as a control group and excluded the original “Control” condition from the analyses. (2) We merged multiple tests into a single test whenever possible. Previously, the hypotheses and research questions require multiple tests to compare the difference between the HuffPost condition and the control condition and the difference between the Fox News condition and the control condition separately. The new tests in this section are reduced into a single test that compare the HuffPost condition and the Fox News condition. (3) We revised some of the hypotheses and research questions so we can accommodate these changes in testing. (4) We report two quantities of interest for each of them:

2. The covariate-adjusted ITT estimate computed using the Lin (2013) saturated regression approach, as well as the standard error (in parentheses) and a 95% conﬁdence interval (in square brackets).

55 Table 46

Hypothesis Speciﬁc outcome Study Support? Issue Opinions H1 & H2 Issue scale (conservatism) Study 1 Issue Opinions H3 & H4 Immigration issue scale Study 1 News Browsing H1 Conservative news visits (1 week) Study 1 ! News Browsing H2 Liberal news visits (1 week) Study 1 News Browsing H1 Conservative news visits (4 weeks) Study 1 ! News Browsing H2 Liberal news visits (4 weeks) Study 1 News Browsing H1 Conservative news visits (6 weeks) Study 1 ! News Browsing H2 Liberal news visits (6 weeks) Study 1 Social Media H1 Conservative news shares Study 1 Social Media H2 Liberal news shares Study 1 Social Media H3 Conservative news follows Study 1 Social Media H4 Liberal news follows Study 1 Affective Polarization H1a & H2a Rate Dems Study 1 Affective Polarization H1b & H2b Rate Reps Study 1 ! Affective Polarization H1c & H2c Rate Trump supporters Study 1 Perceived Polarization H1a & H1b Study 1 Agenda Setting H1 & H2 Issue importance index Study 1 Elite Approval H1a & H2a Trump approval Study 1 Elite Approval H1b & H2b Congress (Rep) Study 1 Elite Approval H1c & H2c Congress (Dem) Study 1 Media Trust H1a & H2a Trust HuffPost Study 2 ! Media Trust H1b & H2b Trust in Fox News Study 2 Media Trust H3a & H3b Perceived bias Study 2 Media Trust H4a & H4b Mainstream media trust Study 2 Media Trust (week 7) H4a & H4b Mainstream media trust Study 2 Factual Knowledge H1 % Foreign born Study 2 Factual Knowledge H2 % Unemployment Study 2 Election Prediction H1a & H2a Predict Dems win House Study 2 Election Prediction H1b & H2b Predict Dems win vote share Study 2 Election Prediction H1c & H2c Predict Dems win district Study 2 Mail-Bombing H1a & H1b False ﬂag Study 2 Mail-Bombing H1a & H1b Media bias responsible Study 2 Mail-Bombing H1a & H1b Trump responsible Study 2

56 6.1 Issue opinions The new results are consistent with the pre-registered hypotheses discussed in Section 3.1.

Table 47: Results: issue opinions hypotheses 1–4

Outcome Unadjusted ITT Adjusted ITT H1 & H2: Conservatism -0.123 (0.086) 0.008 (0.037) (HuffPost vs. FoxNews) [-0.292, 0.046] [-0.064, 0.080] H3 & H4: Pro-immigration 0.149 (0.076) 0.040 (0.051) (HuffPost vs. FoxNews) [0.000, 0.297] [-0.060, 0.139]

Each column reports: (1) unadjusted difference-in-means estimate and (2) covariate-adjusted ITT estimate. Stan- dard errors are reported in parentheses and 95% CIs are reported in square brackets.

Table 48: Results: issue opinions RQ1

Outcome Party ID Ideology H1 & H2: Conservatism n.s. n.s. H3 & H4: Pro-immigration n.s. n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

57 6.2 Subsequent news browsing behaviors The participants in the HuffPost condition are less likely to use conservative news sites than the participants in the Fox News condition after 1 week, 4 weeks, and 6 weeks after the intervention. The new results are consistent with the results of the pre-registered hypotheses and research questions discussed in Section 3.2.

Table 49: Results: news browsing

Outcome Unadjusted ITT Adjusted ITT 1 week after treatment H1: Cons. news (1 week) -0.558 (0.130) -0.473 (0.082) (HuffPost vs. FoxNews) [-0.813, -0.303] [-0.635, -0.311] H2: Lib. news (1 week) 0.147 (0.141) -0.041 (0.078) (HuffPost vs. FoxNews) [-0.131, 0.424] [-0.193, 0.111] 4 week after treatment H1: Cons. news (4 weeks) -0.585 (0.160) -0.578 (0.094) (HuffPost vs. FoxNews) [-0.899, -0.271] [-0.763, -0.393] H2: Lib. news (4 weeks) 0.155 (0.169) -0.105 (0.088) (HuffPost vs. FoxNews) [-0.176, 0.487] [-0.277, 0.068] 6 week after treatment H1: Cons. news (6 weeks) -0.622 (0.168) -0.618 (0.100) (HuffPost vs. FoxNews) [-0.953, -0.292] [-0.815, -0.420] H2: Lib. news (6 weeks) 0.227 (0.174) -0.042 (0.095) (HuffPost vs. FoxNews) [-0.114, 0.568] [-0.229, 0.146]

Each column reports: (1) unadjusted difference-in-means estimate and (2) covariate-adjusted ITT estimate. Stan- dard errors are reported in parentheses and 95% CIs are reported in square brackets.

Table 50: Results: News browsing RQ1 & RQ2 (one week after treatment)

Outcome Party ID Ideology Pre-treat. HuffPost Pre-treat. FoxNews H1: Cons. news n.s. + n.s. n.s. H2: Lib. news n.s. n.s. n.s. n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

58 6.3 Subsequent social media behaviors The new results are consistent with the results of the pre-registered hypotheses and research questions discussed in Section 3.3.

Table 51: Results: social media hypotheses 1–4

Outcome Unadjusted ITT Adjusted ITT H1: Tweets w/cons. links 0.109 (0.149) 0.056 (0.078) (HuffPost vs. FoxNews) [-0.186, 0.404] [-0.099, 0.211] H2: Tweets w/lib. links 0.177 (0.171) 0.049 (0.106) (HuffPost vs. FoxNews) [-0.161, 0.515] [-0.160, 0.258] H3: Follow cons. media 0.003 (0.081) 0.089 (0.061) (HuffPost vs. FoxNews) [-0.158, 0.163] [-0.031, 0.210] H4: Follow lib. media 0.012 (0.120) -0.089 (0.072) (HuffPost vs. FoxNews) [-0.223, 0.248] [-0.230, 0.053]

Each column reports: (1) unadjusted difference-in-means estimate, (2) covariate-adjusted ITT estimate. Standard errors are reported in parentheses and 95% CIs are reported in square brackets.

59 6.4 Affective polarization The new results are consistent with the results of the pre-registered hypotheses and research questions discussed in Section 3.4.

Table 52: Results: affective polarization hypotheses 1–2

Outcome Unadjusted ITT Adjusted ITT H1a & H2a: Dem. thermometer 4.494 (2.689) -0.067 (0.979) (HuffPost vs. FoxNews) [-0.785, 9.773] [-1.990, 1.855] H1b & H2b: Rep. thermometer -3.960 (2.613) 1.178 (1.030) (HuffPost vs. FoxNews) [-9.091, 1.171] [-0.845, 3.201] H1c & H2c: Trump distance 0.401 (0.171) 0.104 (0.100) (HuffPost vs. FoxNews) [0.066, 0.737] [-0.093, 0.301]

Each column reports: (1) unadjusted difference-in-means estimate, (2) covariate-adjusted ITT estimate. Standard errors are reported in parentheses and 95% CIs are reported in square brackets.

Table 53: Results: Affective polarization RQ1

Outcome Party ID Ideology H1a & H2a: Dem. thermometer (HuffPost vs. FoxNews) n.s. n.s. H1b & H2b: Rep. thermometer (HuffPost vs. FoxNews) n.s. n.s. H1c & H2c: Trump distance (HuffPost vs. FoxNews) n.s. n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

60 6.5 Perceived polarization The new results are consistent with the results of the pre-registered hypotheses and research questions discussed in Section 3.5.

Table 54: Results: Perceived polarization hypotheses 1

Outcome Unadjusted ITT Adjusted ITT H1a & H1b: Perceived polarization -0.018 (0.062) -0.018 (0.062) (HuffPost vs. FoxNews) [-0.141, 0.104] [-0.141, 0.104]

Each column reports: (1) unadjusted difference-in-means estimate, (2) covariate-adjusted ITT estimate. Standard errors are reported in parentheses and 95% CIs are reported in square brackets.

Table 55: Results: Perceived polarization RQ1

Outcome Party ID Ideology H1a & H2a: Perceived polarization (HuffPost vs. FoxNews) n.s. n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

61 6.6 Agenda-setting The new results are consistent with the results of the pre-registered hypotheses and research questions discussed in Section 3.6.

Table 56: Results: agenda setting hypotheses 1–2

Outcome Unadjusted ITT Adjusted ITT H1 & H2: Agenda setting 0.030 (0.016) -0.000 (0.008) (HuffPost vs. FoxNews) [-0.001, 0.061] [-0.017, 0.016]

Each column reports: (1) unadjusted difference-in-means estimate, (2) covariate-adjusted ITT estimate. Standard errors are reported in parentheses and 95% CIs are reported in square brackets.

Table 57: Results: Agenda setting RQ1 & RQ2

Outcome Party ID Ideology Pre-treat. HuffPost Pre-treat. FoxNews H1 & H2: Agenda setting (HuffPost vs. FoxNews) n.s. n.s. n.s. n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

62 6.7 Approval of President/Congress The new results are consistent with the results of the pre-registered hypotheses and research questions discussed in Section 3.7.

Table 58: Results: Elite approval hypotheses 1–2

Outcome Unadjusted ITT Adjusted ITT H1a & H2a: Pres. approval -0.224 (0.133) 0.003 (0.035) (HuffPost vs. FoxNews) [-0.485, 0.037] [-0.066, 0.071] H1b & H2b: Rep. Cong. pref. -0.052 (0.036) -0.002 (0.011) (HuffPost vs. FoxNews) [-0.122, 0.018] [-0.024, 0.020] H1c & H2c: Dem. Cong. pref. 0.043 (0.037) -0.012 (0.017) (HuffPost vs. FoxNews) [-0.029, 0.116] [-0.046, 0.021]

Each column reports: (1) unadjusted difference-in-means estimate, (2) covariate-adjusted ITT estimate. Standard errors are reported in parentheses and 95% CIs are reported in square brackets.

Table 59: Results: Elite approval RQ1

Outcome Party ID Ideology H1a & H2a: Pres. approval (HuffPost vs. FoxNews) n.s. n.s. H1b & H2b: Rep. Cong. pref. (HuffPost vs. FoxNews) n.s. n.s. H1c & H2c: Dem. Cong. pref. (HuffPost vs. FoxNews) n.s. n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

63 6.8 Voting behavior The new results suggest that the subjects in the HuffPost condition are more likely to cast a vote than the participants in the Fox News condition. The original results discussed in Section 3.8 did not ﬁnd any statistically signiﬁcant difference in voting behavior.

Table 60: Results: Voting behavior RQ1

Outcome Unadjusted ITT Adjusted ITT RQ1a & RQ1b: Turnout 0.073 (0.027) 0.058 (0.024) (HuffPost vs. FoxNews) [0.020, 0.125] [0.010, 0.105]

Each column reports: (1) unadjusted difference-in-means estimate, (2) covariate-adjusted ITT estimate. Standard errors are reported in parentheses and 95% CIs are reported in square brackets.

64 6.9 Media trust The new results suggest that the subjects in the HuffPost condition show higher levels of trust in the HuffPost than the participants in the Fox News condition. However, the negative impact of the Fox News on trust in media, one of the key effects that we found in Section 3.9, was no longer statistically signiﬁcant. This is because the participants in the both two treatment conditions are more likely to have lower media trust, so there is no statistically signiﬁcant differences between these two groups. The tests of research questions show consistent patterns that we found in Section 3.9.

Table 61: Results: Media trust hypotheses 1–3

Outcome Unadjusted ITT Adjusted ITT H1a & H2a: Trust in Fox News -0.213 (0.081) -0.097 (0.060) (HuffPost vs. FoxNews) [-0.372, -0.054] [-0.215, 0.022] H1b & H2b: Trust in HuffPost 0.222 (0.069) 0.132 (0.056) (HuffPost vs. FoxNews) [0.087, 0.357] [0.023, 0.242] H3a & H3b: Liberal media bias 0.095 (0.086) -0.021 (0.074) (HuffPost vs. FoxNews) [-0.074, 0.264] [-0.167, 0.126] H4a: Media trust 0.138 (0.071) 0.035 (0.053) (HuffPost vs. FoxNews) [-0.002, 0.279] [-0.069, 0.139] H4a: Media trust (w7) 0.047 (0.077) -0.046 (0.056) (HuffPost vs. FoxNews) [-0.105, 0.199] [-0.156, 0.064]

Each column reports: (1) unadjusted difference-in-means estimate, (2) covariate-adjusted ITT estimate. Standard errors are reported in parentheses and 95% CIs are reported in square brackets.

Table 62: Results: Media trust RQ1

Outcome Party ID Ideology H1a: Trust in Fox News (HuffPost vs. FoxNews) n.s. n.s. H2a: Trust in HuffPost (HuffPost vs. FoxNews) − n.s. H3a: Liberal media bias (HuffPost vs. FoxNews) − +

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

65 6.10 Knowledge The new results suggest that there is no statistically signiﬁcant differences between the participants in the HuffPost condition and the participants in the Fox News condition. These results are because of the fact that the participants in both groups have higher levels of event knowledge compared to the participants in the control group.

Table 63: Results: Factual/event knowledge hypotheses 1–2 & RQ2

Outcome Unadjusted ITT Adjusted ITT H1: % Foreign born 0.656 (1.393) 0.656 (1.393) (HuffPost vs. FoxNews) [-2.079, 3.391] [-2.079, 3.391] H2: % unemployment 1.226 (1.238) 1.226 (1.238) (HuffPost vs. FoxNews) [-1.205, 3.657] [-1.205, 3.657] RQ1a: Event knowledge -0.003 (0.005) -0.006 (0.005) (HuffPost vs. FoxNews) [-0.013, 0.007] [-0.016, 0.003] RQ1b: Immigration event knowledge -0.022 (0.025) -0.022 (0.025) (HuffPost vs. FoxNews) [-0.071, 0.027] [-0.071, 0.027]

Each column reports: (1) unadjusted difference-in-means estimate, (2) covariate-adjusted ITT estimate. Standard errors are reported in parentheses and 95% CIs are reported in square brackets.

Table 64: Results: Knowledge RQ2

Outcome Party ID Ideology H1: % foreign born n.s. n.s. H2: % unemployment n.s. n.s. RQ1a: Event knowledge n.s. n.s. RQ1b: Event knowledge n.s. n.s.

Coefﬁcient sign (not signiﬁcant, positive, negative) for interaction effect between treatment indicator and moderator (in column). Effects are computed in the covariate-adjusted ITT model.

66 6.11 Election prediction In order to compare the two treatment groups, we revised the pre-registered hypotheses as follows: Pre-registered hypotheses: • Election Prediction H3: Pre-treatment Republican partisans in the Fox News treatment group and pre-treatment Democratic partisans in the HuffPost treatment group will report a higher average certainty in their predictions than subjects assigned to the control group. • Election Prediction H4: Pre-treatment Republican partisans in the HuffPost treatment group and pre-treatment Democratic partisans in the Fox News treatment group will report a lower average certainty in their predictions than subjects assigned to the control group. Revised hypotheses: • Election Prediction H3:Pre-treatment Democratic partisans in the HuffPost treatment group will report a higher average certainty in their predictions than pre-treatment Re- publican partisans in the Fox News treatment group. • Election Prediction H4: Pre-treatment Republican partisans in the HuffPost treatment group will report a higher average certainty in their predictions than pre-treatment Demo- cratic partisans in the Fox News treatment group. The new results are consistent with the results of the pre-registered hypotheses and research questions discussed in Section 3.11.

Table 65: Results: Election prediction hypotheses 1–2

Outcome Unadjusted ITT Adjusted ITT H1a & H2a: GOP House winner -0.035 (0.038) 0.016 (0.031) (HuffPost vs. FoxNews) [-0.109, 0.040] [-0.044, 0.076] H1b & H2b: GOP House vote share -0.277 (1.202) 0.867 (1.049) (HuffPost vs. FoxNews) [-2.637, 2.083] [-1.193, 2.926] H1c & H2c: GOP House district winner -0.000 (0.038) 0.018 (0.031) (HuffPost vs. FoxNews) [-0.075, 0.075] [-0.044, 0.079] H3a & H3b: Prediction certainty -0.455 (0.246) -0.455 (0.246) (HuffPost vs. FoxNews) [-0.940, 0.029] [-0.940, 0.029] H4a & H4b: Prediction certainty 0.224 (0.268) 0.224 (0.268) (HuffPost vs. FoxNews) [-0.303, 0.752] [-0.303, 0.752]

Each column reports: (1) unadjusted difference-in-means estimate, (2) covariate-adjusted ITT estimate (not reported here due to data sparsity). Standard errors are reported in parentheses and 95% CIs are reported in square brackets.

67 6.12 Mail-bombing incident The new results are consistent with the results of the pre-registered hypotheses and research questions discussed in Section 3.12.

Table 66: Results: Mail-bombing incident hypotheses 1–3

Outcome Unadjusted ITT Adjusted ITT H1a & H1b: Mail-bombing is a false ﬂag 0.098 (0.102) -0.010 (0.085) (FoxNews vs. FoxNews) [-0.103, 0.299] [-0.178, 0.157] H2a & H2b: Media is accountable -0.188 (0.111) -0.037 (0.091) (HuffPost vs. FoxNews) [-0.405, 0.030] [-0.216, 0.143] H3a & H3b: Trump is accountable 0.140 (0.130) -0.094 (0.082) (HuffPost vs. FoxNews) [-0.114, 0.394] [-0.254, 0.067]

Each column reports: (1) unadjusted difference-in-means estimate, (2) covariate-adjusted ITT estimate. Standard errors are reported in parentheses and 95% CIs are reported in square brackets.

68 7 Additional results: Multiple testing adjustments

In this section we report the results of multiple testing adjustments of our pre-registered hypotheses. This is to address concerns about inflated Type I errors when running multiple hypothesis tests. These analyses were not specified in our pre-analysis plan. We apply the two- stage linear step-up procedure to control the false discovery rate as suggested by Benjamini et al. (2006). Figure 16 reports the results for the adjusted p-values of the ITT estimates and Figure 17 reports the results for the adjusted p-values of the CACE estimates. In both figures, the columns report the following:

• Unadjusted: The unadjusted p-values of the ITT/CACE estimates.

• Two-stage BH: The adjusted p-values using the two-stage step-up method by Benjamini et al. (2006), applied to the full vector of n = 52 values.

• Two-stage BH (Domain): The adjusted p-values using the two-stage step-up method by Benjamini et al. (2006), applied within each of the k = 12 hypothesis domains (Issue Opinions, News Browsing, Social Media, etc.).

• Two-stage BH (Study 1): The adjusted p-values using the two-stage step-up method by Benjamini et al. (2006), applied to hypotheses pre-registered under Study 1.

• Two-stage BH (Study 2): The adjusted p-values using the two-stage step-up method by Benjamini et al. (2006), applied to hypotheses pre-registered under Study 2.

69 Figure 16: Unadjusted and adjusted p-values associated with ITT estimates for pre-registered hypotheses.

Unadjusted Two−stage BH Two−stage BH Two−stage BH Two−stage BH (Domain) (Study 1) (Study 2) Issue Opinions H1 0.22 0.85 0.45 0.61 1.00 Issue Opinions H2 0.34 0.85 0.11 0.70 Issue Opinions H3 0.51 0.87 0.51 0.76 0.95 Issue Opinions H4 0.14 0.85 0.63 0.61 News Browsing H1 0.00 0.00 0.45 0.00 News Browsing H2 0.07 0.68 0.11 0.58 0.90 Social Media H1 0.82 0.95 0.45 0.90 Social Media H2 0.39 0.85 0.99 0.73 0.85 Social Media H3 0.25 0.85 0.00 0.61 Social Media H4 0.77 0.92 0.63 0.90 Affective Polarization H1a 0.61 0.91 0.63 0.82 0.80 Affective Polarization H1b 0.01 0.32 0.38 0.18 Affective Polarization H1c 0.49 0.85 0.64 0.76 0.75 Affective Polarization H2a 0.62 0.91 0.03 0.82 Affective Polarization H2b 0.19 0.85 0.82 0.61 Affective Polarization H2c 0.86 0.95 0.78 0.90 0.70 Perceived Polarization H1a 0.16 0.85 0.78 0.61 Perceived Polarization H1b 0.26 0.85 0.64 0.61 0.65 Agenda Setting H1 0.48 0.85 0.82 0.76 Agenda Setting H2 0.41 0.85 0.70 0.73 0.60 Elite Approval H1a 0.27 0.85 0.74 0.61 Elite Approval H1b 0.95 0.95 0.57 0.92 Elite Approval H1c 0.71 0.91 0.86 0.89 0.55 Elite Approval H2a 0.21 0.85 0.70 0.61 Elite Approval H2b 0.86 0.95 0.70 0.90 0.50 Elite Approval H2c 0.90 0.95 0.70 0.90 Media Trust H1a 0.99 0.97 0.74 0.99 Media Trust H1b 0.03 0.33 0.09 0.32 0.45 Media Trust H2a 0.91 0.95 0.70 0.97 Media Trust H2b 0.31 0.85 0.70 0.89 0.40 Media Trust H3a 0.44 0.85 0.74 0.89 Media Trust H3b 0.47 0.85 0.93 0.89 Media Trust H4a 0.02 0.32 0.26 0.32 0.35 Media Trust H4b 0.14 0.85 0.93 0.89

Factual Knowledge H1 0.64 0.91 0.26 0.89 0.30 Factual Knowledge H2 0.32 0.85 0.93 0.89 Election Prediction H1a 0.18 0.85 0.48 0.89 Election Prediction H1b 0.30 0.85 0.48 0.89 0.25 Election Prediction H1c 0.70 0.91 0.80 0.89 Election Prediction H2a 0.47 0.85 0.93 0.89 0.20 Election Prediction H2b 0.67 0.91 0.93 0.89 Election Prediction H2c 0.48 0.85 0.93 0.89 0.15 Prediction uncertainty H3 0.68 0.91 0.68 0.89 Prediction uncertainty H4 0.45 0.85 0.68 0.89 Mail−Bombing H1a 0.93 0.95 0.95 0.97 0.10 Mail−Bombing H1b 0.93 0.95 0.95 0.97 Mail−Bombing H2a 0.69 0.91 0.80 0.89 0.05 Mail−Bombing H2b 0.69 0.91 0.95 0.89 Mail−Bombing H3a 0.74 0.91 0.95 0.89 Mail−Bombing H3b 0.74 0.91 0.99 0.89 0.00

70 Figure 17: Unadjusted and adjusted p-values associated with CACE estimates for pre-registered hypotheses.

Unadjusted Two−stage BH Two−stage BH Two−stage BH Two−stage BH (Domain) (Study 1) (Study 2) Issue Opinions H1 0.43 0.91 0.57 0.67 1.00 Issue Opinions H2 0.64 0.92 0.03 0.84 Issue Opinions H3 0.18 0.75 0.36 0.57 0.95 Issue Opinions H4 0.03 0.30 0.64 0.25 News Browsing H1 0.00 0.00 0.64 0.00 News Browsing H2 0.15 0.72 0.07 0.57 0.90 Social Media H1 0.56 0.92 0.12 0.82 Social Media H2 0.11 0.72 0.68 0.57 0.85 Social Media H3 0.24 0.85 0.00 0.59 Social Media H4 0.31 0.85 0.64 0.59 Affective Polarization H1a 0.76 0.92 0.64 0.90 0.80 Affective Polarization H1b 0.01 0.23 0.21 0.17 Affective Polarization H1c 0.29 0.85 0.81 0.59 0.75 Affective Polarization H2a 0.14 0.72 0.07 0.57 Affective Polarization H2b 0.30 0.85 0.56 0.59 Affective Polarization H2c 0.90 0.92 0.41 0.92 0.70 Perceived Polarization H1a 0.18 0.75 0.41 0.57 Perceived Polarization H1b 0.33 0.85 0.24 0.59 0.65 Agenda Setting H1 0.39 0.88 0.41 0.66 Agenda Setting H2 0.27 0.85 0.90 0.59 0.60 Elite Approval H1a 0.83 0.92 0.42 0.90 Elite Approval H1b 0.81 0.92 0.45 0.90 Elite Approval H1c 0.97 0.95 0.90 0.94 0.55 Elite Approval H2a 0.62 0.92 0.90 0.84 Elite Approval H2b 0.92 0.92 0.90 0.92 0.50 Elite Approval H2c 0.83 0.92 0.90 0.90 Media Trust H1a 0.82 0.92 0.90 0.90 Media Trust H1b 0.00 0.10 0.08 0.09 0.45 Media Trust H2a 0.68 0.92 0.90 0.90 Media Trust H2b 0.37 0.88 0.90 0.90 0.40 Media Trust H3a 0.55 0.92 0.45 0.90 Media Trust H3b 0.49 0.92 0.83 0.90 Media Trust H4a 0.02 0.23 0.33 0.22 0.35 Media Trust H4b 0.09 0.72 0.83 0.72

Factual Knowledge H1 0.81 0.92 0.33 0.90 0.30 Factual Knowledge H2 0.12 0.72 0.83 0.72 Election Prediction H1a 0.57 0.92 0.39 0.90 Election Prediction H1b 0.88 0.92 0.39 0.90 0.25 Election Prediction H1c 0.90 0.92 0.97 0.90 Election Prediction H2a 0.71 0.92 0.83 0.90 0.20 Election Prediction H2b 0.39 0.88 0.83 0.90 Election Prediction H2c 0.88 0.92 0.83 0.90 0.15 Prediction uncertainty H3 0.48 0.92 0.76 0.90 Prediction uncertainty H4 0.76 0.92 0.76 0.90 Mail−Bombing H1a 0.32 0.85 0.97 0.90 0.10 Mail−Bombing H1b 0.51 0.92 0.97 0.90 Mail−Bombing H2a 0.62 0.92 0.97 0.90 0.05 Mail−Bombing H2b 0.66 0.92 0.97 0.90 Mail−Bombing H3a 0.83 0.92 0.97 0.90 Mail−Bombing H3b 0.83 0.92 0.72 0.90 0.00

71 Figure 18: Minimum detectable effects for adjusted ITT estimates. Cohen’s d reported.

MDE (ITT) Issue Opinions H1 0.07 0.20 Issue Opinions H2 0.07 Issue Opinions H3 0.10 0.19 Issue Opinions H4 0.10 News Browsing H1 0.10 News Browsing H2 0.10 0.18 Social Media H1 0.16 Social Media H2 0.17 0.17 Social Media H3 0.18 Social Media H4 0.15 Affective Polarization H1a 0.06 0.16 Affective Polarization H1b 0.06 Affective Polarization H1c 0.09 0.15 Affective Polarization H2a 0.06 Affective Polarization H2b 0.06 Affective Polarization H2c 0.09 0.14 Perceived Polarization H1a 0.15 Perceived Polarization H1b 0.15 0.13 Agenda Setting H1 0.08 Agenda Setting H2 0.08 0.12 Elite Approval H1a 0.04 Elite Approval H1b 0.06 Elite Approval H1c 0.08 0.11 Elite Approval H2a 0.04 Elite Approval H2b 0.06 0.10 Elite Approval H2c 0.08 Media Trust H1a 0.11 Media Trust H1b 0.12 0.09 Media Trust H2a 0.12 Media Trust H2b 0.11 0.08 Media Trust H3a 0.13 Media Trust H3b 0.13 Media Trust H4a 0.11 0.07 Media Trust H4b 0.11

Factual Knowledge H1 0.15 0.06 Factual Knowledge H2 0.15 Election Prediction H1a 0.12 Election Prediction H1b 0.14 0.05 Election Prediction H1c 0.12 Election Prediction H2a 0.12 0.04 Election Prediction H2b 0.14 Election Prediction H2c 0.13 0.03 Prediction uncertainty H3 0.15 Prediction uncertainty H4 0.16 Mail−Bombing H1a 0.13 0.02 Mail−Bombing H1b 0.13 Mail−Bombing H2a 0.12 0.01 Mail−Bombing H2b 0.12 Mail−Bombing H3a 0.10 Mail−Bombing H3b 0.10 0.00

72 8 Deviations with respect to pre-analysis plan

Although the results we present here are based on the pre-analysis plan we registered prior to obtaining the data (see details at https://osf.io/zj65h), there were a few areas in which we had to deviate from the plan. In all cases, these deviations were due to minor errors in questionnaire design or omissions in our pre-analysis plan. For the sake of transparency, we report each of these changes here:

• Not all survey items in the pro-immigration scale were included in the pre-encouragement waves. Instead, we use the issue attitudes scale and the one immigration attitudes question we included in Wave 2 (about DREAMers) as pre-treatment covariates.

• Our questionnaire did not include approval of Republicans or Democrats. In our pre- analysis plan we had intended to refer to Congressional control preference (for the 2018 midterm elections) instead of approval and that is what we use in the analysis.

• One of the issue attitudes items (attitudes towards war) was not included in the ﬁnal version of the questionnaire in Wave 4 for space reasons. We exclude it from the analysis.

• For the affective polarization hypothesis, we use the social distance item as opposed to the feeling thermometer item to measure attitudes with respect to Trump supporters, since the feeling thermometer was not included in the pre-encouragement waves.

• We do not test explicitly for the Media Trust RQ2 since we did not find a significant effect for the HuffPost group in Wave 4. We had envisioned examining this research question if both were statistically significant.

• In our pre-analysis plan we stated that we would “divide the estimated coefficient on the treatment effect by the share of compliers in the treatment group as measured by whether respondents in that group agreed to participate in the user test of a news service in the survey” in order to approximate the estimated population effect of our encouragement. We do not report this because we considered that reporting the raw coefficients would be easier to interpret. We note that this decision does not affect the statistical significance of our coefficients — only how we report them.

• When testing our hypotheses regarding Twitter behavior, we were not able to compute within-block estimates because some blocks did not have enough observations. In that case, blocks were deﬁned as only Chrome vs. other browsers.

• The pre-analysis plan states that visits to news sources will be measured as log(# URL visits). To account for zeros, we add 1 to all Pulse-based count variables before logging.

• All measures of news visits, conservative news visits, and liberal news visits are constructed to exclude any visits to either foxnews.com or huffingtonpost.com.

73 • In Figure 3, for illustrative purposes, we employ the traditional affective polarization measure: for Democrats (including leaners), deﬁned as the feeling thermometer score toward Democrats minus the feeling thermometer score toward Republicans; for Republicans (including leaners), the feeling thermometer score toward Republicans minus the feeling thermometer score toward Democrats. We use our pre-registered measures in all other reported results (with the exception noted above).

• For exploratory purposes, we include some analyses where we have relevant dependent variables in Waves 7 and 8 of the panel.

74 9 Survey questionnaire

This section shows the speciﬁc wordings we used for all the survey items used as part of this study. Questions are presented in the order they are used in Section 3. In square brackets we indicate the wave in which the questions were asked.

75 9.1 Issue opinions

FOREIGN ISSUES [DEPENDING ON ITEM] Here you can ﬁnd several statements with which some people agree while others do not. How about you? Please state your view on these issues.

(A) Going to war is unfortunate but sometimes the only solution to international problems. [W1, W2]

(B) The best way to ensure world peace is through American military strength. [W1]

(D) It is essential for the United States to work with other nations to solve problems such as overpop- ulation, hunger, and pollution. [W1]

(E) The U.S. should mind its own business internationally and let other countries get along the best they can on their own. [W1]

(F) We should not think so much in international terms but concentrate more on our own national problems. [W1, W2, W4]

(G) Free trade agreements like the North American Free Trade Agreement (NAFTA) have helped the U.S. economy. [W1, W2, W3, W4]

(H) The United States government should try to encourage international trade with other countries. [W2, W3, W4]

(I) Islam encourages violence more than other faiths. [W1, W2, W4]

(J) The United States will need to use military force to resolve the situation with North Korea. [W3, W4]

1. Strongly disagree

2. Somewhat disagree

3. Neither/nor

4. Somewhat agree

5. Strongly agree

– Don’t know

76 DOMESTIC ISSUES [DEPENDING ON ITEM] Here you can ﬁnd several statements with which some people agree while others do not. How about you? Please state your view on these issues.

(A) Gun control laws in the United States should be stricter. [W1, W2, W3, W4]

(B) A zero-tolerance policy for sexual harassment is essential to bringing about change in our society. [W1, W2, W3, W4]

(D) Global warming will pose a serious threat to me or my way of life in my lifetime. [W1, W2, W4]

(E) Black Lives Matter (BLM) is playing a positive role in bringing attention to the issue of police misconduct. [W1]

(F) President Trump should have the ability to use his constitutional pardon power to pardon himself. [W2, W3, W4]

(G) Government regulation of business is necessary to protect the public interest. [W2, W4]

(H) Poor people today have it easy because they can get government beneﬁts without doing anything in return. [W2, W4]

(I) Business corporations make too much proﬁt. [W2, W4]

1. Strongly disagree

2. Somewhat disagree

3. Neither/nor

4. Somewhat agree

5. Strongly agree

– Don’t know

77 POLICY TRADE-OFFS [DEPENDING ON ITEM] Here you can ﬁnd different arguments people make on current issues. Which side do you agree with more? Please locate your position on these issues. [Slider 1-11]

(A) The FBI should thoroughly investigate potential collusion between Russia and the Trump campaign – The FBI’s investigation only harms the U.S. [W1, W2, W4]

(B) “DREAMers,” or people who were children when they came to the U.S. with their parents as undocumented immigrants, should be allowed to stay and earn a path to citizenship – DREAMers should be forced to leave the country and apply for citizenship like everyone else [W1, W2, W4]

(C) Search and social media platforms should be regulated to ensure that they serve the public interest – Regulation of technology ﬁrms threatens free speech online [W1]

(D) The United States will be able to resolve the situation with North Korea diplomatically The United States will need to use military force to resolve the situation with North Korea [W1, W2, W4]

(E) The United States should allow travelers in a Central American migrant caravan to enter and seek refugee status – The United States should turn away migrants in the caravan and actively deter them from seeking entry [W4]

(F) The federal government should forbid insurance companies from charging higher rates to those with preexisting conditions – Insurance companies should be allowed to take preexisting conditions into account when determining coverage [W4]

THREAT PERCEPTIONS [W4] How concerned are you about the prospect of:

(A) A terrorist attack in the United States in the near future

(B) Russia interfering in the American midterm elections

(D) Iran developing nuclear weapons

(E) Undocumented immigrants taking jobs away from American citizens

(F) The loss of American jobs to China

1. Extremely concerned

2. Very concerned

3. Somewhat concerned

4. Not at all concerned

78 9.2 Affective polarization

AFFECTIVE POLARIZATION DEMOCRATS FEELING THERMOMETER [W2, W4] We would like to get your feelings toward Democrats using something we call the feeling thermometer. Ratings between 50 degrees and 100 degrees mean that you feel favorable and warm toward the group. Ratings between 0 degrees and 50 degrees mean that you don’t feel favorable toward the group and that you don’t care too much for that group. You would rate the group at the 50 degree mark if you don’t feel particularly warm or cold toward the group.

1.0

2. ...

3. 100

AFFECTIVE POLARIZATION REPUBLICANS FEELING THERMOMETER [W2, W4] We now would like to get your feelings toward Republicans using the same feeling thermometer.

1.0

2. ...

3. 100

AFFECTIVE POLARIZATION SOCIAL DISTANCE, TRUMP SUPPORTERS [W2, W4] Suppose a son or daughter of yours was getting married. How would you feel if he or she married a Trump supporter? Would you be pleased, would you be displeased, or would it make no difference?

1. Generally happy (1)

2. ...

3. It wouldn’t matter at all (4)

4. ...

5. Generally unhappy (7)

79 9.3 Perceived polarization

PERCEIVED POLARIZATION [W4] How much do you agree or disagree with the following statements?

(A) Democrats and Republicans hate each other.

(B) The difference between Democrats and Republicans are too great to be reconciled.

1. Strongly disagree

2. Somewhat disagree

3. Neither/nor

4. Somewhat agree

5. Strongly agree

6. Don’t know

80 9.4 Agenda-setting effects

MOST IMPORTANT PROBLEM, CLOSED [W2, W3, W4] There are many issues that concern people. How about you? Which of the following issues and problems do you consider to be particularly important in the United States? (check all that apply)

1. economy/unemployment

2. relationship with North Korea

3. relationship with Western countries (e.g., Canada, France, Germany)

4. international trade imbalances

5. immigration

6. terrorism

7. inequality

8. racism

9. morality and values

10. health care

11. crime

12. Islam

13. fake news

14. political polarization

15. Donald Trump and his administration

16. gun control

17. women’s rights

18. identity politics

19. alt-right movement

20. Black Lives Matter movement

21. free speech

22. none of the above

81 9.5 Approval of President/Congress

PRESIDENTIAL APPROVAL [W1, W2, W3, W4] Do you approve or disapprove of the way Donald Trump is handling his job as president?

1. Strongly approve

2. Somewhat approve

3. Neither approve nor disapprove

4. Somewhat disapprove

5. Strongly disapprove

CONGRESS CONTROL PREFERENCE [W1, W3, W4] Which party would you prefer to control Congress after the midterm elections?

1. Democrats

2. Republicans

3. Divided between House and Senate

4. None of the above

9.6 Voting behavior

TURNOUT INTENTION MIDTERM [W3, W4] Do you plan to vote in the 2018 midterm elections in November?

1. Yes

2. No

3. Not sure

4. I’m not registered to vote

5. I already voted (for example, by mailing a ballot)

82 9.7 Media trust FOX NEWS TRUST [W4] How much do you trust Fox News when it comes to reporting the news about government and politics fully, accurately, and fairly?

1. A great deal

2. A fair amount

3. Not very much

4. Not at all

HUFFPOST TRUST [W4] How much do you trust The Hufﬁngton Post when it comes to reporting the news about government and politics fully, accurately, and fairly?

1. A great deal

2. A fair amount

3. Not very much

4. Not at all

MEDIA SLANT [W4] In presenting the news dealing with political and social issues, do you think that news organizations deal fairly with all sides, or do they tend to favor one side?

1. Tend to favor the liberal side (1)

2. ...

3. Tend to favor neither side (3)

4. ...

5. Tend to favor the conservative side (5)

TRUST IN MEDIA [W4] How much trust and conﬁdence do you have in the press when it comes to reporting the news about government and politics fully, accurately, and fairly?

1. A great deal

2. A fair amount

3. Not very much

4. None at all

83 9.8 Knowledge

CHEATING PLEDGE [W1, W2, W3] It is important to us that you do NOT use outside sources like the Internet to search for the correct answer. Will you answer the following questions without help from outside sources?

1. Yes

2. No

FACTUAL KNOWLEDGE IMMIGRANTS [W4] Out of every 100 people living in our country, how many do you think were born outside the United States? [Slider 0-100] FACTUAL KNOWLEDGE UNEMPLOYMENT [W4] What is the current unemployment rate in the U.S., as reported by the Bureau of Labor Statistics? [Slider 0-100]

84 KNOWLEDGE ABOUT RECENT EVENTS [DEPENDING ON ITEM] Below are some events that may or may not have taken place in the past few weeks. Please select the events that you believe have indeed happened.

1. Donald Trump and Kim Jong Un met in Singapore [TRUE] [W2]

2. The U.S. government decided to permanently drop out of the G7 meetings [FALSE] [W2]

3. Thousands of immigrant families were separated at the border with Mexico after a change in border control policies [TRUE] [W2]

4. Kim Kardashian met with President Trump in the Oval Ofﬁce [TRUE] [W2]

5. The economy grew at a 0.5% rate in the third quarter of 2018 [FALSE] [W2]

6. Justice Ruth Bader Ginsburg announced she was stepping down from the Supreme Court [FALSE] [W2]

7. The trial against former Trump campaign chairman Paul Manafort began [TRUE] [W2]

8. The Spanish national soccer team won the World Cup [FALSE] [W2]

9. A caravan of Central American migrants crossed into Mexico with the eventual goal of reaching the United States. [TRUE] [W4]

10. A Russian national was charged with attempting to interfere in the 2018 U.S. midterm elections. [TRUE] [W4]

11. Former Microsoft chairman Bill Gates died of cancer. [FALSE] [W4]

12. Macy’s declared bankruptcy. [FALSE] [W4]

13. Donald Trump paid a million dollars to Elizabeth Warren after she proved her Native American ancestry with a DNA test. [FALSE] [W4]

14. Kanye West met in the Oval Ofﬁce with President Donald Trump. [TRUE] [W4]

9.9 Election prediction

PREDICTED HOUSE WINNER [W2, W4] Regardless of which party you support, and trying to be as objective as possible, who do you think will win control of the House of Representatives in November – the Democrats or the Republicans (who currently have the majority)?

1. The Democratic Party

2. The Republican Party

85 PREDICTED VOTE SHARE [W2, W4] Regardless of which party you support, and trying to be as objective as possible, what do you think the national vote share of the Democratic Party will be in the midterm elections this November? [Note we speciﬁed “two-party” vote share in W4.]

1.0

2. ...

3. 100

PREDICTED CONGRESSIONAL DISTRICT WINNER [W4] Regardless of which party you support, and trying to be as objective as possible, who do you think will win the race for the House seat in your Congressional district?

1. The Democratic candidate

2. The Republican candidate

3. Another candidate

PREDICTION UNCERTAINTY [W4] How certain are you of this outcome on a scale from 1 (extremely uncertain) to 10 (extremely certain)?

1. Extremely uncertain (1)

2. ...

3. Extremely certain (10)

86 9.10 Mail-bombing incident

TRUMP MESSAGE RECEPTION [W4] Recently, prominent Democrats and critics of President Trump, including Barack Obama, Hillary Clin- ton, George Soros, and Robert De Niro, were the apparent targets of a mail-bombing spree. Below are several explanations for the mail bombs with which some people agree while others do not. How about you? Please select the responses that come closest to your views.

(A) Liberals or Democrats sent the packages as a “false ﬂag” operation intended to paint conservatives as violent extremists before the Nov. 6 midterm elections.

(B) The mainstream media’s unfair coverage, hostility, and negative attacks were responsible for the attempted bombings.

1. Strongly disagree

2. Somewhat disagree

3. Neither agree nor disagree

4. Somewhat agree

5. Strongly agree

9.11 Experiment factual questions

POST EXPERIMENT FOX [W4] Last time, you may have been enlisted in research about a news website. Now we want to follow up and ask about your impressions of the coverage you saw. Please answer to the best of your ability, even if you don’t recall having visited a given website. Among the following options, which do you remember reading about the most on Fox News?

• Climate change

• Mollie Tibbetts

• Federal leak investigation

• Mitch McConnell

• President Trump’s taxes

87 POST EXPERIMENT HUFFPOST [W4] Among the following options, which do you remember reading about the most on HuffPost?

• Climate change

• Mollie Tibbetts

• Federal leak investigation

• Mitch McConnell

• President Trump’s taxes

9.12 Other variables used as controls POLITICAL INTEREST [W1] How interested in politics are you?

1. Very interested

2. Interested

3. Moderately interested

4. Slightly interested

5. Not interested at all

88 MEDIA SOURCES, POLITICAL INFORMATION [W1] How often do you get political information from the following sources:

(A)TV

(B) Newspapers or print magazines

(D) Internet

(E) Personal discussions

1. Several times a day

2. About once a day

3. 3 to 6 days a week

4. 1 to 2 days a week

5. Every few weeks

6. Less often

7. Never

8. Don’t know

89 References

Bail, C. A., Argyle, L. P., Brown, T. W., Bumpus, J. P., Chen, H., Hunzaker, M. F., Lee, J., Mann, M., Merhout, F., and Volfovsky, A. (2018). Exposure to opposing views on social media can increase political polarization. Proceedings of the National Academy of Sciences, 115(37):9216–9221.

Bakshy, E., Messing, S., and Adamic, L. A. (2015). Exposure to ideologically diverse news and opinion on facebook. Science, 348(6239):1130–1132.

Benjamini, Y., Krieger, A. M., and Yekutieli, D. (2006). Adaptive linear step-up procedures that control the false discovery rate. Biometrika, 93(3):491–507.

Dimock, M., Doherty, C., Kiley, J., and Oates, R. (2014). Political polarization in the american public: How increasing ideological uniformity and partisan antipathy affect politics, compro- mise and everyday life. Pew Research Center, 12.

Guess, A. M. (2021). (Almost) everything in moderation: New evidence on americans’ online media diets. American Journal of Political Science, 0.

Guess, A. M., Nyhan, B., and Reiﬂer, J. (2020). Exposure to untrustworthy websites in the 2016 us election. Nature Human Behaviour, 4(5):472–480.

Lin, W. (2013). Agnostic notes on regression adjustments to experimental data: Reexamining freedman’s critique. Annals of Applied Statistics, 7(1):295—318.

Marbach, M. and Hangartner, D. (2020). Proﬁling compliers and noncompliers for instrumental-variable analysis. Political Analysis, pages 1–10.

Van Schuur, W. H. (2003). Mokken scale analysis: Between the guttman scale and parametric item response theory. Political Analysis, 11(2):139–163.