Football and Public Opinion

Football and Public Opinion: A Partial Replication and Extension*

Ethan C. Busby [email protected]

James N. Druckman [email protected]

Department of Political Science Northwestern University Scott Hall 601 University Place Evanston, IL 60208

Abstract Do events irrelevant to politics, such as the weather and sporting events, affect political opinions? A growing experimental literature suggests that such events can matter. However, extant experimental evidence may over-state irrelevant event effects; this could occur if these studies happen to focus on particular scenarios where irrelevant event effects are likely to occur. One way to address this possibility is through replication, which is what we do. Specifically, we replicate an experimental study that showed the outcome of a college football game can influence presidential approval. Our results partially replicate the previous study and suggest the impact is constrained to a limited set of outcome variables. The findings accentuate the need for scholars to identify the conditions under which irrelevant effects occur. While the effects clearly can occur, there relevance to politics remains unclear.

*We thank Jake Druckman, Adam Howat, Elizabeth Meehan, Jacob Rothschild, and Richard Shafranek for research assistance. We also thank Daniel Biggers, Anthony Fowler, Seth Hill, Adam Howat and Neil Malhotra for excellent advice. The data, code, and additional materials required to replicate all analyses in this article are available at the Journal of Experimental Political Science Dataverse within the Harvard Dataverse Network, at: doi:10.7910/DVN/BKVLFI.

Over the last decade, scholars of political opinion formation and voting have debated whether events irrelevant to politics influence attitudes and behaviors. Two noteworthy studies, that use observational data, report that college sport victories and shark attacks can influence

1 incumbents’ vote shares (Healy, Malhotra, and Mo 2010; Achen and Bartels 2016). Others contest these two results, claiming that they are false positives (Fowler and Montagnes 2015;

Fowler and Hall n.d.; although see Healy, Malhotra, and Mo 2015). Numerous experimental studies demonstrate that irrelevant events, such as the weather, sporting events, and random lotteries, can influence political opinions and/or behaviors (e.g., Healy et al. 2010, study 2;

Huber, Hill, and Lenz 2012; Bagues and Esteve-Volart 2016; Bassi 2017; Busby, Druckman, and

Fredendall 2017).

Yet, these experiments have not been replicated on different samples or with different outcomes. We seek to replicate one of these studies (i.e., Busby et al. 2017). Such replication attempts can indicate if the aggregate literature over-states the presence of irrelevant event effects. This could occur from happenstance if particular conditions likely to generate irrelevant event effects happen to be met in extant studies.1 To be clear, this is not a critique of existing papers, which faithfully report careful studies that establish the existence of irrelevant event effects (i.e., researchers did not actively set up studies most likely to produce effects). Rather, replication with a different event, sample, and time is a way to move the literature forward to assess robustness and the conditions under which irrelevant event effects occur.

The Original Study

A “politically irrelevant” event is one that occurs outside the control of elected officials

(Healy and Malhotra 2013). The event can affect individuals’ moods. Affected individuals then may unknowingly use their moods as information when they evaluate a politician or office holder: a positive (negative) mood leads to a favorable (unfavorable) assessment (Huber et al.

2012, 731; Bassi 2017).

1 Another possibility is publication bias such that only significant results enter the published literature (Franco, Malhotra, and Simonovits 2014; Brown, Mehta, and Allison 2017).

2 Busby et al. (2017) offer evidence of such a process in their study of the 2015 College

Football Playoff National Championship game, where The Ohio State University beat the

University of Oregon. The authors drew a random sample of students from each school and then randomly assigned them to receive a survey, advertised as a study of the “social, economic, and political attitudes of college students,” either two days before- or after- the game.

They find that, relative to comparable students surveyed before the game, students from the winning school (OSU) who were surveyed after the game reported a more positive mood, more satisfaction with their school, and more approval of President Obama. Oregon students displayed the opposite dynamics. Thus, the football game, which was beyond political control, ostensibly affected moods and then presidential approval.2 The effect on approval, however, dissipated one week later. Busby et al. (2017) provide causal evidence of a short-term irrelevant event effect but are far from definitive. The authors conclude (2017, 349) that “caution should be taken in generalizing our results. In some sense, our sample size was two – two schools around one event…”

Our Replication

We utilized the same design and procedure as Busby et al. (2017), focusing this time on the 2016 College Football National Championship game. The game took place at 8:30PM

Eastern time on January 11, 2016, and pitted the University of Alabama against Clemson

University. Alabama won the game 45-40.3 We randomly selected Alabama and Clemson students to participate; we then randomly assigned them to receive a before- or after-game

2 Busby et al. (2017) also find that the after-game respondents for the winning team displayed a significant increase in their evaluations of the economy and were significantly more likely to post how they felt on social media. There were no effects on these variables for the losing team respondents.

3 The 2016 game differed from the 2015 game in that it had a smaller viewership (23 percent drop), involved two Southern schools from distinct conferences, and occurred during the start of the 2016 Presidential primary election.

3 invitation to participate in the survey. There thus, in essence, are four cross-sections of data: the before- and after-groups for Alabama and Clemson. We launched the before-game survey on

January 9th, closing it at 5:00PM Eastern time on January 11th; we launched the after-game survey on January 12th, closing it at 5:00PM Eastern Time on January 14th. This was the time 1

(T1) survey. We sent one reminder, and launched a time 2 (T2) follow-up eight days later. The

T2 follow-up asked those who participated at T1 to complete a survey with near identical items to the initial survey.

Like Busby et al. (2017), we measured: presidential approval, satisfaction with one’s university, positive and negative mood, evaluation of the economy, and willingness to post on social media.4 To explore the boundaries of irrelevant effects, we also asked how favorably respondents viewed Pope Francis and how satisfied they were with their lives. The Pope Francis item tests whether effects extend to notable figures beyond the President, while the life satisfaction item comes from classic mood-as-information studies (e.g., Schwarz and Clore 1983,

2003). Finally, we asked respondents how important their university was to their identity since college sport events can influence identity (Cialdini et al. 1976). We provide additional sample, procedural, and question wording details in the Supplementary Appendix.

Results

We assume that, for each school, the before- and after-game survey groups are equivalent, on average (since we randomly assigned participation to the before- or after-game survey). Therefore, any average differences between the groups for each school (we do not compare across school) would indicate an effect of the game. In the Supplementary Appendix, we present demographics and balance checks that suggest comparability of the before- and after-

4 These latter two measures appear in Busby et al.’s (2017) appendix.

4 groups.5 As mentioned, we invited those who responded at T1 to participate in an (identical) survey eight days later. We use these data to assess whether any effects found at T1 endure (e.g., do other factors/events eliminate the immediate irrelevant event effect?).6

[Insert Table 1 About Here]

We present the results in Table 1, which shows the before- and after-game mean scores

(and the differences between the means) on each outcome variable for each school. The results replicate Busby et al.’s (2017) findings on presidential approval, university satisfaction, and mood only for the losing school Clemson. We find no effects for Alabama respondents, in contrast to Busby et al.’s (2017) finding that the game affected respondents from both schools.

Moreover, even for Clemson, we find no significant differences on any other outcome measure, suggesting that the impact is confined to a limited set of attitudes.7

[Insert Table 2 About Here]

We also explored whether the effects sustained one week later. We focus exclusively on the two significant Clemson effects (we did not measure mood in the T2 survey), since those are the effects that might last. In Table 2, we present the T1 and T2 means (and the differences between them) for the Clemson before- and after-game respondents. Note the T1 means in Table

2 do not match the T1 means in Table 1 since in the former we look only at those who responded

5 Alabama and Clemson had respectively beaten Michigan State University and the University of Oklahoma in semi-final games. We collected analogous experimental data from the semi-final losing schools. We find no impact of the game on any political attitudes for these schools; this suggests that any changes observed among Clemson and Alabama respondents come from the game and not other national political events that occurred between the before- and after-game administrations. Discussion of the semi-final loser data are presented in the Supplementary Appendix.

6 Our T1 response rates for the before-game group and after-game group for Clemson and Alabama, respectively, were 11.4% (103/903), 11.3% (99/880), 10.4% (104/1000), 7.8% (78/997). The T2 response rates for before-game group and after-game group for Clemson and Alabama, respectively, were 61.2% (63/103), 60.6% (60/99), 65.4% (68/104), 51.3% (40/78). In the Supplementary Appendix, we discuss variations in response rates.

7 See the Supplementary Appendix for exploratory evidence on how mood may be a mediator, and evidence on the direct effect of positive mood on posting to social media.

5 at T2.8 We find, as did Busby et al. (2017), that the impact of the game was fleeting. Both after- game T2 scores significantly increased to resemble those of the before-game group, suggesting that people reverted back to attitudes unaffected by the game (also see Pierce, Rogers, and

Snyder 2016).9 In the Supplementary Appendix, we provide a host of robustness checks for our

T1 and our over-time results (including accounting for non-response at T2).

Conclusion

The first generation of experimental studies of irrelevant event effects have clearly established that such effects can occur. Our results echo this reality as we replicated one such effect. On the other hand, we failed to replicate the effect for the winning team and found that the effects appear to be transitory and confined to selected outcome variables. This should not be taken as definitive evidence that the extant literature over-states the extent of irrelevant events; yet, it serves as a (cautionary) prompt to the next generation of work.

The obvious question is “under what conditions to irrelevant event effects occur?” Our read of the research thus far suggests they occur more often when people are surprised by the event (e.g., Healy et al. 2010, 12806; Eldar et al. 2016, 15–16),10 are able to (unconsciously) attribute their mood to the object under evaluation (e.g., the president, their university, but not the economy or the Pope) (Schwarz and Clore 2003, 299), and are not motivated to systematically assess information beyond mood. It may be that existing evidence inadvertently

8 If we look at the T1 means for only those who responded at T2 (as in Table 2), the differences are still significant (for presidential approval, at the .1 level). We see an increase in the T1 presidential approval score, absolutely, between all who responded at T1 and only those who responded at T2 (i.e., compare Table 2 to Table 1). The stems from Democrats being significantly more likely to respond at T2. This could possibly reflect increased interest in responding to a survey in light of the State of the Union that occurred during the after-game data collection. Even if so, this would not confound our treatment (see the Supplementary Appendix).

9 Busby et al. (2017) find that the winning team satisfaction result maintained (but the losing team dissatisfaction at T1 seemed to disappear, as we find).

10 In our case, Alabama was favored so the victory was not a surprise, but Clemson fans may have expected victory, despite pre-game odds, since they were ranked first and undefeated.

6 happens to meet these and other unspecified conditions. Replications under different circumstances is one way to move the literature forward to isolate conditions and ultimately assess whether politically irrelevant events occur regularly and widely matter for politics.

7 References Achen, Christopher H., and Larry M. Bartels. 2016. Democracy for Realists: Why Elections Do Not Produce Responsive Government. Princeton, New Jersey: Princeton University Press. Bassi, Anna. 2017. “Weather, Risk, and Voting: An Experimental Analysis of the Effect of Weather on Vote Choice.” Journal of Experimental Political Science Forthcoming. Bagues, Manuel, and Berta Esteve-Volart. 2016. “Politicians’ Luck of the Draw: Evidence from the Spanish Christmas Lottery.” Journal of Political Economy 124: 1269-1294. Brown, Andrew W., Tapan S. Mehta, and David B. Allison. 2017. “Publication Bias in Science: What Is It, Why Is It Problematic, and How Can It Be Addressed?” In The Oxford Handbook of the Science of Science Communication, edited by Kathleen Hall Jamieson, Dan M. Kahan, and Dietram A. Scheufele, 93–101. New York: Oxford University Press. Busby, Ethan C., and James N. Druckman. 2017. “Replication Data for: “Football and Public Opinion: A Partial Replication and Extension.” Harvard Dataverse. doi:10.7910/DVN/BKVLFI. Busby, Ethan C., James N. Druckman, and Alexandria Fredendall. 2017. “The Political Relevance of Irrelevant Events.” The Journal of Politics 79 (1): 346–50. Cialdini, Robert B., Richard J. Borden, Avril Thorne, Marcus Randall Walker, Stephen Freeman, and Lloyd Reynolds Sloan. 1976. “Basking in Reflected Glory: Three (Football) Field Studies.” Journal of Personality and Social Psychology 34 (3): 366. Eldar, Eran, Robb B. Rutledge, Raymond J. Dolan, and Yael Niv. 2016. “Mood as Representation of Momentum.” Trends in Cognitive Sciences 20 (1): 15–24. Fowler, Anthony, and Andrew B. Hall. N.d. “Do Shark Attacks Influence Presidential Elections? Reassessing a Prominent Finding on Voter Competence.” The Journal of Politics. Forthcoming. Fowler, Anthony, and B. Pablo Montagnes. 2015. “College Football, Elections, and False- Positive Results in Observational Research.” Proceedings of the National Academy of Sciences 112 (45): 13800–804. Franco, Annie, Neil Malhotra, and Gabor Simonovits. 2014. “Publication Bias in the Social Sciences: Unlocking the File Drawer.” Science 345 (6203): 1502–5. Healy, Andrew J., and Neil Malhotra. 2013. “Retrospective Voting Reconsidered.” Annual Review of Political Science 16 (1): 285–306. Healy, Andrew J., Neil Malhotra, and Cecilia Hyunjung Mo. 2010. “Irrelevant Events Affect Voters’ Evaluations of Government Performance.” Proceedings of the National Academy of Sciences 107 (29): 12804–12809. Healy, Andrew J., Neil Malhotra, and Cecilia Hyunjung Mo. 2015. “Determining False-Positives Requires Considering the Totality of Evidence.” Proceedings of the National Academy of Sciences 112 (48): E6591–E6591. Huber, Gregory A., Seth J. Hill, and Gabriel S. Lenz. 2012. “Sources of Bias in Retrospective Decision Making: Experimental Evidence on Voters’ Limitations in Controlling Incumbents.” American Political Science Review 106 (04): 720–741. Pierce, Lamar, Todd Rogers, and Jason A. Snyder. 2016. “Losing Hurts: The Happiness Impact of Partisan Electoral Loss.” Journal of Experimental Political Science 3 (1): 44–59. Schwarz, Norbert. 2012. “Feelings-as-Information Theory.” In Handbook of Theories of Social Psychology, edited by Paul A. M. Van Lange, Arie W. Kruglanski, and E. Tory Higgins, 1:289–308. Thousand Oaks, CA: Sage.

8 Schwarz, Norbert, and Gerald L. Clore. 1983. “Mood, Misattribution, and Judgments of Well- Being: Informative and Directive Functions of Affective States.” Journal of Personality and Social Psychology 45 (3): 513. Schwarz, Norbert, and Gerald L. Clore. 2003. “Mood as Information: 20 Years Later.” Psychological Inquiry 14 (3/4): 296–303.

9 Table 1: Effects on Clemson (Losing Team) and Alabama (Winning Team) Respondents Clemson (Losing Team) Alabama (Winning Team) Respondents Respondents

Before- After- Difference Before- After- Difference Game Game (After- Game Game (After- Before) Before) Presidential 3.83 3.35 -0.47** 3.64 3.83 0.19 approval (std. dev. (2.13; (1.94; (1.96; (7-point scale) = 1.80; N 99) 104) 78) = 103) Satisfaction with 5.89 5.43 -0.46** 5.50 5.58 0.08 university (1.53; (1.63; (1.59; (1.59; (7-point scale) 102) 98) 103) 78) Positive Mood (5- 3.28 3.01 -0.26** 2.95 2.97 0.03 point scale) (.89; 101) (.82; 93) (.96; (.92; 101) 75) Negative Mood (5- 1.65 1.99 0.34*** 1.63 1.62 -0.01 point scale) (.60; 100) (.71; 92) (.60; (.56; 100) 77) Evaluation of 2.83 2.95 -0.11 2.67 2.85 0.17 Economy (5-point (1.04; (1.10; (.91; (1.06; scale) 103) 99) 104) 78) Pope Favorability 3.10 3.10 0.00 3.02 3.01 -0.01 (4-point scale) (.71; 103) (.76; 99) (.72; (.70; 104) 77) Life Satisfaction 7.25 7.38 0.13 7.04 7.39 0.35 (10-point scale) (1.88; (1.69; (1.89; (1.84; 101) 96) 102) 77) School Identity 3.62 3.60 -0.02 3.23 3.27 0.04 Importance (5-point (1.16; (1.04; (1.24; (1.10; scale) 102) 98) 103) 78) Likelihood of 1.99 1.77 -0.22 1.92 2.10 0.18 Posting Feelings on (1.29; (1.17; (1.18; (1.19; Social Media (5- 101) 93) 102) 77) point scale) ***p ≤ .01, **p ≤ .05, *p ≤ .10 for one-tailed tests. Slight differences between the values in the difference column and the subtraction of the before and after-game groups are due to rounding.

10 Table 2: Clemson Over-Time Effects Clemson (Losing Team) Respondents Before-Game After-Game T1 T2 Difference T1 T2 Difference (T2-T1) (T2-T1) Presidential 4.14 4.02 -0.13 3.67 4.22 0.55*** approval (1.85; (1.81; (2.15; (1.76; (7-point scale) 63) 63) 60) 60)

Satisfaction with 5.82 5.97 0.15 5.10 5.77 0.67*** university (1.62; (1.32; (1.73; (1.16; (7-point scale) 62) 62) 60) 60) ***p ≤ .01, **p ≤ .05, *p ≤ .10 for one-tailed tests.

11 Supplementary Appendix

As noted in the text, we collected data from the University of Alabama and Clemson University and from the semi-final losers: the University of Oklahoma and Michigan State University. Since we collected the data for all four schools simultaneously, we largely present details on all four schools in tandem in the sections that follow.

A1. Sampling and administrative details.

We collected our sample by using the public, online student directories for the University of Oklahoma (http://www.ou.edu/content/ousearch.html?q=directory&type= web&sa=GO), Michigan State University (https://search.msu.edu/people/index.php), the University of Alabama (http://directory.ua.edu/), and Clemson University (https://my.clemson .edu/#/directory/advanced-search). Research assistants were provided with lists of randomly generated letters (in sets of 3 letters) and numbers to use in the sampling. Each of the research assistants searched the random 3 letter string and started sampling with the random number assigned to that three letter string (i.e., if the number was 4, they started with the fourth person to come up in the search results). This process continued until we had sampled between 1,000 and 2,000 students at each university (1,997 for Alabama; 1,783 for Clemson; 1,697 for Michigan State; and 1,218 for Oklahoma). These numbers varied somewhat due to the difficulty of accessing different schools’ directories. The sample for each school was then randomly divided into two groups – a before- and after-game group.

The invitation to participate in initial time 1 (T1) wave of the survey was sent to the before-game groups on January 9th, and subjects had until 5:00PM Eastern Time on January 11th (the day of the game) to complete the survey. The T1 invitation was sent to the after-game groups on January 12th, and subjects had until 5:00PM Eastern Time on January 14th to complete the survey. For both groups, a reminder email was sent the day after the initial invitation.

The second wave time 2 (T2) invitation was sent to the before-game group on January 18th, and subjects had until January 20th to complete the survey. T2 invitations were sent to the after-game group on January 21st; subjects had until January 23rd to complete the survey. One reminder was sent regarding the T2 survey two days after the initial invitation.

A research assistant blind to the study hypotheses and procedures monitored the online survey software (hosted via SurveyMonkey) while data collection was ongoing.

More administrative details, including the text of the email invitations, the subject lines used, etc., are available upon request.

A2. Demographics of the sample by school.

Table A.1: Sample Characteristics Michigan Clemson Alabama Oklahoma State % Female 51.8 66.5 60.0 56.5

12 % White 74.3 84.6 72.1 72.0 Average Party 4.3 4.2 3.6 3.6 Identificationa $70,000- $70,000- $70,000- $70,000- Median Family Income $99,999 $99,999 $99,999 $99,999 Average Age 22.4 23.3 21.4 20.3 a 3 refers to an independent-leaning Democrat, 4 a pure independent, and 5 an independent-leaning Republican

A3. Balance checks.

Our study uses randomization to the before- and after-game conditions to make two comparable groups that differ only in their exposure to the game outcome. The analyses we present in the paper rely on the assumption that this randomization was successful. In order to evaluate this assumption, we ran a logit model for each school, where the outcome was the before/after-game assignment and the predictors were age, year in school, grade point average (GPA), if the school was the respondent’s top choice, if the respondent had family members who were alumni of the school, income, gender, if the respondent was white, if the respondent was Catholic, political ideology, party identification, political interest, and if the respondent watched the championship game.

Our analyses suggest very few, if any, differences between the before-game and after-game conditions within each school. Importantly, for the one school where we observe a clear effect of the game outcome (Clemson), we find no evidence of systematic differences between the before- and after-game groups on any of the variables listed above. This suggests that our findings regarding Clemson are not likely due to some imbalance between the two conditions. In the analyses for the three other schools where we do not find significant effects from the game, we only see statistically meaningful differences between the before- and after-game conditions on a few variables. Given the number of predictors in each model and that we ran four such models, finding four (out of 52) significant relationships is not inconsistent with successful randomization. (We would expect about three significant relationships due solely to chance.) In addition, the differences between the before- and after-game groups deal almost primarily with nonpolitical variables – for Oklahoma, individuals in the after-game group reported a higher income. For Alabama, individuals in the after-game group reported a higher income, were farther along in their studies at their university, and reported less interest in politics. The only explicitly political variable was interest in politics in the Alabama sample, and this relationship does not persist when we account for missing data using multiple imputation (see later section). The detailed results of these balance-check models are as follows.

Clemson (N=162)

13 14 Alabama (N=127)

Oklahoma (N=38)

15 Michigan State (N=145)

As we had some missing values on these predictors, we also performed these analyses using multiple imputation. Using the multiple imputation procedures available in Stata, we performed 200 chained imputations to estimate the missing values of these covariates and any missing data at time 2 (see section A8). The results of the imputed models lead to the same conclusions as the unimputed models. The detailed results of the imputed analyses are as follows.

Clemson

Alabama

16 Oklahoma

17 Michigan State

These imputed models tell a similar story as the unimputed analyses; only 5 variables are statistically different between the before- and after-game groups at the p<0.10 level, which is about what we would expect by chance. For Oklahoma, those in the after-game group have a lower average income; for Michigan State, those in the after-game group are more likely to have chosen Michigan State as their top choice. For Alabama, individuals in the after-game group are more likely to have a higher income. For Clemson, those in the after-game group are more interested in politics and less likely to have chosen Clemson as their top choice. The nonpolitical variable of choosing Clemson as one’s top choice is likely to bias the results against finding an effect, as these individuals should care less about the school and be less prone to irrelevant event effects. When we control for political interest, the effects of the game for the Clemson sample persist,11 and there do not seem to be effects from political interest on the main outcomes. This suggests that differences in interest does not account for the estimated effect of the game in the Clemson sample.

Taken as a whole, we do not find evidence that there are serious imbalances in our randomization, supporting our assumption of comparability between the before- and after-game groups.

A4. Oklahoma and Michigan State results.

The tables below present analyses for both Oklahoma and Michigan State that are parallel to the analyses presented in the main text.

Table A.2: Effects on Michigan State and Oklahoma Respondents Michigan State Oklahoma

11 In these analyses (which use multiple imputation), the effect of the game on presidential approval is significant at the p=0.05 level, the effect on school satisfaction is significant at the p=0.03 level, the effect on positive mood is significant at the p=0.01 level, and the effect on negative mood is significant at the p=0.001 level. All of these p- values are for one-tailed tests, given the directionality of our predictions. The results of these analyses are available upon request.

18 Respondents Respondents Before- After- Difference Before- After- Difference Game Game (After- Game Game (After- Before) Before) Presidential 4.85 4.63 -0.22 4.46 4.21 -0.25 approval (1.59; (1.58; (1.91; (1.79; (7-point scale) 86) 107) 37) 24)

Satisfaction with 5.55 6.02 0.47*** 5.41 5.79 0.39 university (1.42; (1.08; (1.48; (1.22; (7-point scale) 85) 107) 37) 24) Positive Mood (5- 2.86 2.98 0.11 2.78 3.02 0.24 point scale) (.87; 81) (.77; (.97; 36) (1.04; 102) 22) Negative Mood (5- 1.79 1.68 -0.12 1.64 1.70 0.06 point scale) (.62; 81) (.63; (.60; 37) (.44; 101) 22) Evaluation of 2.84 2.99 0.15 2.78 2.92 0.13 Economy (5-point (1.03; (.99; (1.11; (.93; scale) 86) 108) 37) 24) Pope Favorability 3.12 3.06 -0.06 2.89 3.00 0.11 (4-point scale) (.59; 85) (.56; (.81; 37) (.78; 108) 24) Life Satisfaction 6.70 6.78 0.08 6.59 6.96 0.36 (10-point scale) (2.09; (2.11; (2.15; (1.46; 84) 105) 37) 23) School Identity 3.35 3.85 0.50*** 3.27 3.29 0.02 Importance (5- (1.10; (1.04; (1.28; (1.40; point scale) 85) 107) 37) 24) Likelihood of 1.82 1.87 0.05 2.28 2.04 -0.24 Posting Feelings (1.16; (1.19; (1.39; (1.33; on Social Media 84) 102) 36) 23) (5-point scale) ***p ≤ .01, **p ≤ .05, *p ≤ .10 for one-tailed tests. Slight differences between the values in the difference column and the subtraction of the before and after-game groups are due to rounding.

19 Table A.3: Michigan State Over-Time Effects Michigan State Respondents Before-Game After-Game T1 T2 Difference T1 T2 Difference (T2-T1) (T2-T1) Presidential 4.98 4.91 0.08 4.75 4.71 -0.05 approval (1.41; (1.26; (1.60; (1.54; (7-point scale) 53) 53) 65) 65)

Satisfaction with 5.68 5.96 0.28** 6.17 6.14 -0.03 university (1.31; (0.88; (0.98; (0.95; (7-point scale) 53) 53) 65) 65)

***p ≤ .01, **p ≤ .05, *p ≤ .10 for one-tailed tests.

It is possible that fans from one of these schools may have been in a better mood if the team that had beaten their team won the championship game since it would signal that that team was superior. This struck us as unlikely, however, as it involves considerably complex assessments; moreover, neither of the teams in the game were from the conferences of the losers so they had no investment in terms of any shared identity. And indeed, we find that that neither school shows a change in emotion or a change in our political variables which is robustness evidence that other dynamics did not confound our treatment.

As demonstrated in tables A.2 and A.3, we observe two significant difference between the before- and after-game groups (but not on politically relevant variables). For Michigan State, the after-game group was significantly more satisfied with their university than the before-game group and more strongly identified with their school. We observe no differences on any of the other outcome variables for Michigan State or for Oklahoma on any variable.

One explanation of the Michigan State effects is that these changes reflect Michigan State students starting school on January 11th (the day of the football game). This means the before- game group either had not yet began classes or had just that day taken a class or two. The after- game group likely had settled in and felt decreased anxiety, increased school connection/identity, and thus satisfaction. This would be a somewhat distinct mediational process, although note on the single emotion of anxiety, we did find a significant decline in the after-game group. No other schools would have overlapped in returning in this precise way.

We also looked at the over-time dynamics for the two Michigan State scores that demonstrated significant change. As in the main text, we again focused exclusively on those who responded at time 2 (T2). We see, as evidenced in Table A.3, that that the before-game group scores significantly increased at T2 to resemble those of the after-game group scores (which did not significantly change at T2). This suggests the effects are not fleeting, which is, in fact, exactly what we would expect if our speculation that arriving at school and starting the semester increases school identity and satisfaction, in the case of Michigan State.

In sum, this does not undermine our conclusions about the effect of the game. However, it does

20 indicate that other events in individuals’ lives may affect these outcomes, and that researchers should take care to isolate or understand such variables in studies of irrelevant events.

A5. Alabama and Clemson response rates.

We excluded, from the total sent, any e-mails that bounced back. There were a total of 49 bounce-backs across all four schools, although a notably high number of them came from the after-game Clemson conditions, for reasons we do not know (e.g., we suspect chance). Our response rates resemble those from Busby et al. (2017) whose rates ranged from 10.4% to 13.0%. Our before-game versus after-game Alabama response rates are significantly different from one another. We do not have evidence that this lower response rate lead to significant differences between the Alabama conditions, as described in section A.2. The one political difference noted in that section is that the after-game Alabama group is significantly less interested in politics. If anything, we would expect this to make them more susceptible to the game effect since they may likely have less crystalized attitudes. Given our results, this turns out not to have come to fruition.

A6. Oklahoma and Michigan State response rates.

The Oklahoma and Michigan State before-game group and after-game group, respective time 1 (T1) response rates were: 4.4% (37/849), 2.8% (24/848), 14.1% (86/609), 17.7% (108/609). The respective time 2 (T2) response rates were 56.8% (21/37), 54.2% (13/24), 61.6% (53/86), 61.1% (66/108).

The T1 response rate for Oklahoma was much lower than that of the other schools. We are not sure why, but it may be because Oklahoma started school on January 19th, well after the survey. Clemson started before the survey (January 6th) and Michigan State started during the before- game survey (January 11th). Alabama started on January 13th which was also after the survey but given that timing, it may have been students had already returned or were at least checking emails in anticipation of the semester (or simply more engaged given the game and ongoing communications with fellow students.) This is all of course speculative, and it is important to note that our conclusions about Oklahoma may be limited due to the low response rate. Whatever nonresponse biases exist, however, it does appear that the before- and after-game groups are comparable to one another, giving us some confidence in our Oklahoma findings.

A7. Mediation results.

The proposed theoretical mechanism behind many irrelevant event effects is mood. As noted in note in the text, we included measures of positive and negative mood (using the Positive and Negative Affect Schedule measures). Following Busby et al. (2017), the positive moods measured were “elated,” “enthusiastic,” “proud,” and “interested.” The negative moods were “sad,” “afraid,” “angry,” “hatred,” “bitter,” “contempt,” “worried,” “anxious,” and “resentful.” The respective alphas to create positive and negative mood scales (across all four schools) are . 81 and .84, although the inter-item covariance is notably higher for the positive mood scales (.64 versus .33).

21 Like Busby et. al (2017), the design of our study precludes us from making strong claims about mediation (e.g., Bullock and Ha 2011; Imai, Keele, and Tingley 2010). However, our findings are consistent with our theory about mood as the mechanism behind the effect of the game – the experience of a loss for Clemson fans created a more negative and less positive mood. This change in mood then influenced perceptions of the status quo, including that of President Obama.

We observe meaningful changes in mood in reaction to the loss for Clemson fans and changes in presidential approval that are consistent with this theory. For the Clemson sample, when we regress presidential approval on experimental condition, the condition is significant. When we add our mood measures, the experimental condition variable falls to insignificance while the positive and the negative mood measures are significant (at the p=0.10 level using a one-tailed test) (see Baron and Kenney 1986). This is also true of college satisfaction. While we cannot establish rigorous causal estimates of these mediators, we take this evidence as consistent with what we would expect of a mediated relationship.

A8. Posting to social media regressions overall conditions.

In addition to the main analyses in the paper, we also analyzed whether individuals’ moods made them more or less likely to post about their feelings on social media. This outcome relates to contagion through social networks, as posting about one’s mood could serve to spread the effects of these events to other people.

We find that, across conditions, individuals who have a more positive mood indicate that they are more likely to post their feelings on social media (results summarized in Table A.4). This is true across schools and even after holding constant if the subjects took the survey before or after the game. We do not find evidence of a negative effect from negative mood, which is in keeping with the findings of Busby et al. (2017). We find no statistically significant effects from the game on willingness to post on social media, which may be because we only observe effects from the game for the losing team (Clemson).

22 Table A.4: Social Media Effects on Mood (OLS Regression with p-values in parentheses) All Clemson Clemson Clemson Clemson Alabama Alabama Alabama Alabama schools Positive 0.29 0.21 -- 0.19 0.23 0.30 -- 0.30 0.36 mood (0.00) (0.04) (0.06) (0.04) (0.00) (0.00) (0.00) Negative -0.09 -0.02 -- 0.03 0.10 -0.19 -- -0.19 -0.31 mood (0.22) (0.89) (0.83) (0.49) (0.22) (0.22) (0.07) After------0.22 -0.25 -0.28 -- 0.18 0.10 0.22 game (0.22) (0.15) (0.15) (0.31) (0.58) (0.23) Controls No No No No Yes No No No Yes R2 0.05 0.02 0.01 0.03 0.07 0.07 0.01 0.07 0.13 N 592 186 194 186 171 174 179 174 158 Notes: The dependent variable is how likely individuals state they are to post their feelings on social media. Coefficients are presented, with p-values in parentheses. P-values are calculated using heteroskedasticity-robust standard errors.

A9. Over-time robustness checks.

The over-time nature of some of our analyses allows for a component of nonresponse – some subjects who took the study at time 1 (T1) did not respond at time 2 (T2). Having a biased subsample of our original participants might influence the results of our overtime analyses; to evaluate this possibility, we conducted a series of robustness checks.

First, we recalculated our main treatment effects (at T1) on only the subset of participants who completed both waves of the study. Table A.5 shows the effects for this subset of the participants. These findings lead to the same conclusions as the main analyses, with some additional uncertainty due to the decreased sample size.

Table A.5: Effects on Clemson (Losing Team) and Alabama (Winning Team) Respondents Clemson (Losing Alabama (Winning Team) Team) Respondents Respondents Before- After- Difference Before- After- Difference Game Game (After- Game Game (After- Before) Before) Presidential 4.14 3.67 -0.48* 3.62 3.75 0.13 approval (std. dev. (2.14; (1.92; 68) (1.97; (7-point scale) = 1.85; N 60) 40) = 63) Satisfaction 5.82 5.1 -0.72** 5.44 5.46 0.02 with (1.61; 62) (1.73; (1.57; 68) (1.59; university 60) 39) (7-point scale) Positive Mood 3.25 2.94 -0.31** 2.84 2.88 0.03 (5-point scale) (.87; 63) (.84; (.90; 68) (.98; 59) 40) Negative 1.62 2.11 0.49*** 1.57 1.69 0.12 Mood (5-point (.65; 63) (.80; (.62; 68) (.57;

23 scale) 57) 40) ***p ≤ .01, **p ≤ .05, *p ≤ .10 for one-tailed tests. Slight differences between the values in the difference column and the subtraction of the before and after-game groups are due to rounding.

Second, we modelled nonresponse at T2 as a function of our demographic variables (age, year in school, GPA, income, gender, Catholic faith, party ID, interest, and watching the game), T1 dependent variables, and experimental condition. Table A.6 lists the statistically significant relationships between nonresponse for each school, associated two-tailed p-values, and the direction of those relationships.

Table A.6: Modeling Nonresponse at Time 2 Clemson Alabama Oklahom Michigan State a After-game -- Negative (p=0.04) -- -- condition Presidential approval Positive (p=0.09) ------College satisfaction Negative (p=0.06) ------Positive mood -- Negative (p=0.03) -- -- Negative mood ------Age ------Year in school ------Positive (p=0.03) GPA ------Positive (p=0.01) Income -- Negative (p=0.01) -- -- Gender Positive (p=0.02) -- -- Positive (p=0.07) Catholic ------Party ID ------Interest ------Watched game ------

Table A.6 suggests that there may be some differences between the T1 and T2 samples, especially with regards to the Clemson sample. Those who responded to T2 were more positive in their presidential approval than the T1 only sample. This may make it more likely to find an increase in presidential approval at T2, but does not explain the differences between the before- and after-game groups on this measure. Importantly, we do not see a different likelihood of responding at T2 for the before- and after-game groups. The finding that the T2 sample was more negative in their satisfaction with their university suggests that our estimates of a rebound effect on this variable may be conservative, as the T2 sample was more negative towards their university at T1. Third, and to verify the robustness of our over-time analyses to differences reported in Table A.6, we re-estimated the T2 effects accounting for missing data using the imputed dataset discussed in section A2. These analyses, which are available upon request, confirm the results presented in the main text. This leads us to conclude that our over-time findings (that the effects of the game dissipate after a week for the Clemson sample) are robust to biases due to missing data at T2. A10. Robustness checks.

24 We consider a series of other robustness checks to verify that our main findings persist even when we examine other subgroups in the data or consider confounding explanations.

The first deals with the difference in game watching by before- and after-game groups. We allowed respondents to enter any response on these items, and so the sum of games watched and games attended often exceed the total number of games played. This is in fact plausible since respondents may have attended game while also watching parts of it on-line via a mobile device (and our “watching” question did not preclude this).

The total games played for Alabama, Clemson, Oklahoma, and Michigan State, respectively, were 15, 15, 13, and 14. The games watched total for before- and after- Clemson and Alabama are: 8.44 (4.76; 100) versus. 9.50 (5.50; 94) (t192 = 1.44; p < .10 for a one-tailed test) and 8.86 (4.98; 99) versus 9.88 (5.72; 77) (t174 = 1.27; p < .10 for a one-tailed test). The near one-game difference presumably reflects the championship game. We fail to observe these differences for the Michigan State and Oklahoma samples; this is sensible as these two teams were not playing in the championship game.12 Our Alabama and Clemson results further confirm balance between experimental conditions as groups seemed to pay attention to the games at similar rates (i.e., the one game difference is because the after-game groups watched the championship game while the before-game groups could not have done so yet).

A second potential confounder in our analysis is President Obama’s State of the Union Address, which was given on January 12th, during the after-game time 1 period. It could be the case that this speech, rather than the outcome of the game, influenced presidential approval, mood, and school satisfaction. Fortunately, the survey included several measures about the state of the union, including if subjects planned on watching or did watch it,13 how much media they watched about it, and how much they talked with other people about the state of the union.

At first glance, it seems unlikely that the State of the Union Address influenced our findings, simply because few participants watched it, consumed much media about it, or talked much about it. Table A.7 presents these data.

12 More specifically, the number of games watched for the before- and after-game groups for Oklahoma differed only by 0.17 (non-significant). For MSU, the number of games watched differed by 0.77 (non-significant).

13 Only the after-game group was asked this question, as it was added to the survey after the data collection began for the before-game group. The wording was about planning to or actually watching it as some subjects took the survey during the day on January 12th, before the speech was given.

25 Table A.7: State of the Union Statistics Reported watching (T1) – all schools 32.1% Reported watching (T2) – all schools 18.7% Reported watching (T1) – Clemson 31.9% Reported watching (T2) – Clemson 18.2% Reported watching (T1) – Alabama 32.5% Reported watching (T2) – Alabama 17.9% Average media consumed (T2) – all Small amount or less schools Average media consumed (T2) – Clemson Small amount or less Average media consumed (T2) – Alabama Small amount or less Average discussion (T2) – all schools Small amount or less Average discussion (T2) – Clemson Small amount or less Average discussion (T2) – Alabama Small amount or less

If we restrict the results to include only individuals who did not report viewing the State of the Union Address, only consumed a small amount of media about the speech, or only discussed the speech a small amount, our results remain unchanged. The effects for the Alabama sample remain statistically insignificant. The effects of the game for Clemson respondents remain statistically significant – the most dramatic changes are from the p=0.05 to the p=0.07 level, which is sensible given the reduced sample size. In most instances, the effects remain statistically significant at or below the p=0.05 level. This leads us to conclude that our findings are likely not biased by the State of the Union Address. Detailed results of these analyses are available upon request.

Our sampling includes both individuals who did and did not watch the game. This suggests a third possible confounder, that some subjects may not have experienced the outcome of the game (a victory or loss) in the same way as others. While we suspect that the campus atmosphere at both schools influenced nearly all students regardless of if they watched the game, we conducted our analyses only on subjects who reported that they watched the game.14 We did this by using an item on the survey that asked respondents whether they planned (before-game group) or had (after-game group) watched the National Championship game; roughly 95% of Clemson respondents from each group and 90.5% of Alabama respondents reported watching.

When we restrict the sample to this group (those who watched the game), the main results remain largely unchanged. The effect of the game (for Clemson respondents) on college satisfaction, positive mood, and negative mood remain significant well below traditional thresholds of p=0.05. The statistical significant of effect of the game on presidential approval drops to p=0.08,15 but again, this is not unexpected given the reductions in sample size. Taken as a whole, these findings lead us to conclude that we see the same pattern of effects when we consider the whole sample and when we examine only those who watched the game. As we think

14 We do this by looking at those in the before-game group who said they planned on watching the game and those who, in the after-game group, reported watching the game. We also examine only subjects who reported watching the game in time 2. The results are similar across both approaches.

15 When we examine only those who, at time 1, reported watching the game, p=0.12

26 it likely that students at both schools could be affected by the game without watching it (through social media, the general atmosphere on campus, interpersonal relationships, etc.), we feel that, if anything, the effects on the whole sample provide more accurate estimates due to the increased sample size.

Fourth, our sample included a small number of graduate students, between 15 and 30 per school (despite our efforts to include only undergraduates, based on the public school directories’ listed statuses). While many of these students are likely to be affected by the game, it is possible that the effects for them may be different than for undergraduates. When we restrict the analyses to only undergraduate students, we continue to find the same effects as presented in the main text. In fact, the results for Clemson are more statistically significant on the undergraduate sample than on the sample that includes graduate students.

Fifth, one of the variables we discussed in the text deals with approval ratings for Pope Francis. We find no effects from the game on this measure; however, it is possible that Catholics and non-Catholics responded to this item differently. Catholics may think of the Pope more readily with regards to the status quo, and may be more prone to irrelevant event effects. To examine this possibility, we conducted our analyses of the Pope Francis approval item for Catholics and non-Catholics separately. While we are limited in terms of our sample size, we continue to find no evidence of an effect from the game on favorability for the Pope in the Clemson sample when we look at Catholics and non-Catholics separately. For the Alabama sample, we do find that Catholics in the after-game condition express higher levels of approval for the Pope than those in the before-game condition (p=0.06). We see no similar effect among non-Catholics at Alabama. However, given the small sample (for Catholics, n=33) and that this goes against the findings on the main outcomes, we recommend verifying this finding. A sixth issue for our inferences concerns spillover between different conditions (before- and after-game groups). In other words, if individuals in the before/after group spoke to others in the opposite group about the survey, our findings may have an unknowable amount of bias. As we do not ask about this on the survey, we cannot empirically evaluate this possibility. However, we consider this highly unlikely given number of students at the universities from which we sampled. Our samples make up less than 1 percent of Clemson’s student population, less than 0.5 percent of Alabama’s student population, 0.3 percent of Oklahoma’s students, and 0.4 percent of Michigan State’s student population. This makes spillover (as a statistical matter) incredibly unlikely. A final confounder deals with the weather during this period at the schools from which we sampled. If a significant change in the weather occurred around the time of the game, this might look like an effect from the game while actually occurring due to the weather. Table A.8 does not show evidence of a confounder due to weather – there were only minor, if any, differences in weather between the before-game group (in yellow) and the after-game group (in green). If anything, the fact that the weather was cooler in Clemson between T1 and T2 makes the finding of increased positivity towards the president and one’s school more important.

27 Table A.8: Weather at Clemson and Alabama Precip Clemso High Alabam High (inches n (F) Precip (inches) a (F) ) 9-Jan 54 0.53 9-Jan 59 0.21 10-Jan 53 0.24 10-Jan 47 0.00 11-Jan 46 0.00 11-Jan 46 0.00 12-Jan 54 0.00 12-Jan 57 0.00 13-Jan 52 0.00 13-Jan 57 0.00 14-Jan 62 0.00 14-Jan 62 0.00 15-Jan 52 1.32 15-Jan 55 0.18 16-Jan 61 0.00 16-Jan 56 0.00 17-Jan 55 0.01 17-Jan 50 0.00 18-Jan 41 0.00 18-Jan 37 0.00 19-Jan 39 0.00 19-Jan 39 0.00 20-Jan 39 0.02 20-Jan 48 0.18 21-Jan 48 0.12 21-Jan 57 1.77 22-Jan 39 1.76 22-Jan 48 0.00 23-Jan 37 0.05 23-Jan 42 0.00 24-Jan 51 0.00 24-Jan 50 0.00

From: Weather Underground

A11. Question wordings. Time 1 Variables

Presidential approval: How much do you disapprove or approve of the way Barack Obama is handling his job as president? Please select one response.

strongly disapprove somewhat neither disapprove somewhat approve strongly disapprove disapprove nor approve approve approve 1 2 3 4 5 6 7

State of the economy: What do you think about the state of the economy these days in the United States? Would you say that the state of the economy is very bad, somewhat bad, neither bad nor good, or very good? very somewhat neither bad somewhat very bad bad nor good good good 1 2 3 4 5

Evaluations of Pope Francis: Is your overall opinion of Pope Francis very unfavorable, mostly unfavorable, mostly favorable, or very favorable?

28 very mostly mostly very unfavorable unfavorable favorable favorable 1 2 3 4

College satisfaction: To what extent are you unsatisfied or satisfied with your decision to attend XXXX?

extremely very somewhat neither satisfied somewhat very extremely unsatisfied unsatisfied unsatisfied nor unsatisfied satisfied satisfied satisfied 1 2 3 4 5 6 7

College-based identity: How important is being a student at XXXX to your identity? not at all a little moderately very extremely important important important important important 1 2 3 4 5 Mood: The box below contains a number of words that describe different feelings and emotions. Read each item (in the first column) and then mark the appropriate answer in the row after that word. Indicate to what extent you feel this way right now, that is, at the present moment.

1 2 3 4 5 very slightly a little moderately quite a bit extremely or not at all Elated Enthusiastic Proud Interested Sad Afraid Angry Hatred Bitter Contempt Worried Anxious Resentful

Life satisfaction: All things considered, how satisfied are you with your life? 0 1 2 3 4 5 6 7 8 9 10 totally neither totally dissatisfied satisfied satisfied nor dissatisfied

29 Posting on social media: In two or three sentences, how do you currently feel (emotionally)? Please use the space provided below to write your answer. ______

If you use social media (Facebook, Twitter, etc.), how unlikely or likely are you to post the feelings described above? very somewhat neither unlikely somewhat very unlikely unlikely nor likely likely likely 1 2 3 4 5

Demographic/control variables:

What is your estimate of your family’s annual household income (before taxes)?

< $30,000 $30,000 - $69,999 $70,000-$99,999 $100,000-$200,000 >$200,000 1 2 3 4 5

Please indicate your sex. male female 0 1

Which of the following do you consider to be your primary racial or ethnic group (you may check more than one)?

White African American Asian American Hispanic Native American Other

Do you consider yourself to be Protestant, Catholic, Muslim, Jewish, other religion, or no religion? protestant catholic muslim jewish other religion no religion

Which point on this scale best describes your political views? very moderately somewhat moderate somewhat moderately very liberal liberal liberal conservative conservative conservative 1 2 3 4 5 6 7

Generally speaking, do you consider yourself a Democrat, Independent, or Republican? ____ strong weak Independent Independent Independent weak strong Democrat Democrat leans Democrat leans Republican Republican Republican 1 2 3 4 5 6 7

Some people don’t pay much attention to political campaigns. How about you? Would you say you have been not much interested, somewhat interested, or very much interested in the political campaigns this year?

30 not much somewhat very much interested interested interested 1 2 3

This past football season, about how many of your school’s football games did you attend? ____

This past football season, about how many of your school’s football games did you watch, at least part of, on television or the internet? ____

Before-game group only: Are you planning on watching the College Football Championship Game? ______No Yes 0 1

After-game group only: Did you watch the College Football Championship Game? ______No Yes 0 1

After-game group only: Do you plan on watching (or did you watch) President’s Obama State of the Union Address on January 12th, 2016. ______No Yes 0 1

Time 2 Variables Presidential approval: How much do you disapprove or approve of the way Barack Obama is handling his job as president? Please select one response.

strongly disapprove somewhat neither disapprove somewhat approve strongly disapprove disapprove nor approve approve approve 1 2 3 4 5 6 7

State of the economy: What do you think about the state of the economy these days in the United States? Would you say that the state of the economy is very bad, somewhat bad, neither bad nor good, or very good? very somewhat neither bad somewhat very bad bad nor good good good 1 2 3 4 5

Evaluations of Pope Francis: Is your overall opinion of Pope Francis very unfavorable, mostly unfavorable, mostly favorable,

31 or very favorable? very mostly mostly very unfavorable unfavorable favorable favorable 1 2 3 4

College satisfaction: To what extent are you unsatisfied or satisfied with your decision to attend XXXX?

extremely very somewhat neither satisfied somewhat very extremely unsatisfied unsatisfied unsatisfied nor unsatisfied satisfied satisfied satisfied 1 2 3 4 5 6 7

College-based identity: How important is being a student at XXXX to your identity? not at all a little moderately very extremely important important important important important 1 2 3 4 5

Life satisfaction: All things considered, how satisfied are you with your life? 0 1 2 3 4 5 6 7 8 9 10 totally neither totally dissatisfied satisfied satisfied nor dissatisfied Control variables:

Did you watch the College Football Championship Game? ______No Yes 0 1

Did you watch President Obama’s State of the Union Address on January 12th, 2016? ______No Yes 0 1

How much media coverage of President Obama’s State of the Union Address did you consume (i.e., via the internet, newspapers, etc)?

None Small Moderate Good A lot Amount Amount Amount 1 2 3 4 5

How much did you talk with others about President Obama’s State of the Union Address?

None Small Moderate Good A lot Amount Amount Amount

32 1 2 3 4 5

33 Appendix References

Bullock, John G., and Shang E. Ha. 2011. “Mediation Analysis Is Harder Than It Looks.” In Cambridge Handbook of Experimental Political Science, edited by James N. Druckman, Donald P Green, James H. Kuklinski, and Arthur Lupia, 508–21. New York: Cambridge University Press. Imai, Kosuke, Luke Keele, and Dustin Tingley. 2010. “A General Approach to Causal Mediation Analysis.” Psychological Methods 15 (4): 309.