PARTISAN-MOTIVATED SAMPLING
Note: This manuscript is a pre-print version of a manuscript accepted for publication in the Journal of Personality and Social Psychology. This paper is not the copy of record and may not exactly replicate the final, authoritative version of the article. Please do not copy or cite without authors' permission. The final article will be available, upon publication, via its DOI: 10.1037/pspi0000375.
Data and Materials for this manuscript are available at: https://osf.io/9627e/? view_only=4f5c8a589e014c92a1fca171bb0e4369.
Please email [email protected] if you have questions. 2 Partisan-Motivated Sampling
Partisan-Motivated Sampling: Re-Examining Politically Motivated Reasoning
Across the Information Processing Stream
Yrian Derreumaux1, Robin Bergh2, Brent L. Hughes1
1University of California, Riverside, 2Uppsala University
Correspondence should be addressed to:
Brent Hughes Department of Psychology University of California, Riverside 900 University Ave, Riverside CA, USA Email: [email protected] 3 Partisan-Motivated Sampling
Abstract
The U.S. is increasingly politically polarized, fueling intergroup conflict and intensifying partisan biases in cognition and behavior. To date, research on intergroup bias has separately examined biases in how people search for information and how they interpret information. Here, we integrate these two perspectives to elucidate how partisan biases manifest across the information processing stream, beginning with (1) a biased selection of information, leading to
(2) skewed samples of information that interact with (3) motivated interpretations to produce evaluative biases. Across 3 experiments and 4 internal meta-analyses, participants (N = 2,431) freely sampled information about ingroup and outgroup members or ingroup and outgroup political candidates until they felt confident to evaluate them. Across experiments, we reliably find that most participants begin sampling information from the ingroup, which was associated with individual differences in group-based motives, and that participants sampled overall more information from the ingroup. This sampling behavior, in turn, generates more variability in ingroup (relative to outgroup) experiences. We find that more variability in ingroup experiences predicted when participants decided to stop sampling, and was associated with more biased evaluations. We further demonstrate that participants employ different sampling strategies over time when 4 Partisan-Motivated Sampling the ingroup is de facto worse–obfuscating real group differences–and that participants selectively integrate their experiences into evaluations based on congeniality. The proposed framework extends classic findings in psychology by demonstrating how biases in sampling behavior interact with motivated interpretations to produce downstream evaluative biases and has implications for intergroup bias interventions.
Keywords: intergroup bias, political partisanship, experience sampling, motivated reasoning
The U.S. is more polarized now than during civil rights movement
(Abramowitz & Saunders, 2008; Levendusky, 2010; Pew Research Center, 2017).
Elevated polarization increases partisan biases in information processing
(Druckman et al., 2013). Consider the electorate’s experience of President Donald
Trump’s impeachment hearings: People were exposed to different amounts of pro- vs-anti impeachment arguments, based largely on what news sources they consumed (e.g., Fox News vs. CNN). Moreover, even when people had access to the same information (e.g., that Trump asked Ukrainian President Zelensky to investigate the Bidens), conclusions tended to diverge along party lines (i.e. 91% 5 Partisan-Motivated Sampling
Democrats thought Trump had acted illegally vs. 32% of Republicans; Pew
Research Center, 2020).
These examples highlight two important aspects of information processing that can generate biased evaluations and beliefs. First, people tend to reach conclusions that re-affirm their pre-existing partisan beliefs, even when exposed to the same data (an interpretive bias). This phenomenon fits into a long tradition of research on motivated reasoning, which describes how pre-existing beliefs (e.g., that the ingroup is better) bias the interpretation of information to form conclusions that are most desired rather than those that are most accurate (Greenberg &
Pyszczynski, 1985; Kunda, 1987; Wyer & Frey, 1983; Kunda, 1990). Second, although Republicans and Democrats had access to diverse and comprehensive information, they gathered only a subset of such information, which likewise led to congenial conclusions (a sampling bias). Such sampling biases stem from people accessing only a subset of information that is non-representative and skewed relative to all available information (e.g., Fiedler, 2000; Lindskog, Winman, &
Juslin, 2013). To date, research has typically examined sampling and interpretive biases separately (e.g., Denrell, 2005; Gampa et al., 2019), rather than jointly modeling these influences on evaluations of groups (or other judgments and decisions). 6 Partisan-Motivated Sampling
Here, we argue that group-based bias is driven in part by a motivation preceding those ubiquitous in studies of motivated reasoning: a wish to gather information first and most frequently from one’s own groups. This, in turn, leads to non-representative samples that, when unaccounted for, interact with partisan- based interpretive motives to bolster biased evaluations favoring the ingroup. In other words, we examine connections between three sources of evaluative bias across the information-processing stream: (1) a biased selection of information, which leads to (2) skewed samples of information, which interacts with (3) motivated interpretations to produce evaluative biases. Together, we provide a comprehensive framework for understanding the emergence of partisan biases across information processing by integrating sampling and evaluative sources of bias. We review the literature on each source of bias, stepping backwards through the causal chain to capture the evolution of the literature (interpretive biases have the longest history; e.g., Hastorf & Cantril, 1954), before outlining the framework for the current experiments.
Partisan Evaluative Biases
Social identities provide people with an important source of value and status, which in turn increases their tendency to engage in motivated reasoning when exposed to identity-relevant information (e.g., Kahan, 2013; Kahan et al., 2011, 7 Partisan-Motivated Sampling
2013). Political identity in particular has become one of the strongest social identity attachments in the U.S., along with race, ethnicity, and religion (Iyengar et al., 2019; Iyengar & Krupenkin, 2018; Mason, 2018). As such, a wealth of research has examined interpretive biases that arise in politically motivated reasoning
(Cohen, 2003; for meta-analysis see: Ditto et al., 2019; for a review of methods see: Kahan, 2016; Tappin et al., 2020). In a typical study within this tradition, participants are asked to evaluate information that is matched as closely as possible on all dimensions, except for whether the information (or the source of the information) favors one’s political identity or not. A consistent finding is that congenial information (i.e. information that favors pre-existing beliefs) or information from congenial sources (e.g., coming from an ingroup source) is interpreted more favorably and perceived as more valid than uncongenial information (Flynn et al., 2017; Gampa et al., 2019; Hughes et al., 2017; Lord et al., 1979; Tappin et al., 2017). For example, Democrats and Republicans evaluate identical information more favorably when it is supported by members of their own party (i.e. “party over policy”; Cohen, 2003).
A growing body of research suggests that people not only favor congenial information, but are also motivated skeptics of uncongenial information
(disconfirmation bias; Ditto et al., 1998; Ditto & Lopez, 1992; Kraft et al., 2015; 8 Partisan-Motivated Sampling
Taber et al., 2009; Taber & Lodge, 2012). From this perspective, people adopt differential judgement criteria when evaluating uncongenial relative to congenial information, holding arguments they dislike to higher standards that require more and stronger evidence. Taken together, this research explains how politically motivated reasoning, which affects both Democrats and Republicans (Ditto et al.,
2019; cf. Baron & Jost, 2019), can generate discrepancies in peoples’ evaluations of the same (i.e. matched) information based on group membership.
Importantly, the majority of these frameworks focus on the interpretation phase of information processing, where the congeniality of the information, most often qualitative in nature (e.g., a title of an article that is pro/anti-gun rights) or the partisan source of the information (e.g., coming from a conservative or liberal source) are explicitly labeled. However, they do not consider the sampling phase of information processing, wherein people gather and explore information that precedes interpretation. In addition, it is often difficult to establish causal links between partisan-motivations and other motivations and explanations, due in part to the qualitative nature of the information used in studies of politically motivated reasoning (see e.g., Tappin et al., 2020). Moreover, in the real world, people often have to make inferences about underlying distributions from “noisy” data that require prolonged search to arrive at an accurate judgement. As reviewed next, that 9 Partisan-Motivated Sampling is because people often rely on limited samples that introduce other sources of bias.
Sampling Biases
In contrast to motivated reasoning perspectives, some cognitive models propose that biases can emerge from processes that are more innocuous than group-favoring motivations. For instance, sampling models describe how people draw inferences about categories or groups from their limited experiences with individual category members (Denrell, 2005; Denrell & Le Mens, 2017; Fiedler,
2000; Kutzner & Vogel, 2011; Le Mens et al., 2018; Le Mens & Denrell, 2011;
Lindskog et al., 2013; Lindskog & Winman, 2014; Meiser, 2005). A fundamental lesson in statistics is that samples, particularly small ones, are inherently noisy and often skewed. Using this analogy, some scholars have described people as naïve statisticians who only experience a subset of all available information when making a range of judgements and decisions (e.g., Herzog & Hertwig, 2009; Juslin et al., 2007; Lindskog et al., 2013). These models provide alternative accounts to a number of judgement biases previously attributed to motivated reasoning (e.g., confirmation bias, illusory correlation; Fiedler, 2000; Meiser, 2005). This perspective demonstrates that people are capable of producing veridical representations of data they encounter, but they are naïve to the processes that 10 Partisan-Motivated Sampling generate them (Fiedler, 2000; Juslin et al., 2007; Lindskog et al., 2013; Nisbett &
Wilson, 1977; Tversky & Kahneman, 1971). Further, people tend to fail to correct for skewed samples, which can account for biased judgements devoid of motivational influences.
Although group-based bias can arise in a data-driven way due to the natural distribution of information in an environment, some models further suggest that people manufacture biased distributions based on when they terminate the information search (e.g., Denrell, 2005; Coenen & Gureckis, 2021). These models highlight the potential downstream effects of positive or negative intergroup interactions, absent ingroup favoritism. For example, people are more likely to stop interacting with individuals with whom they have a negative experience, irrespective of their group membership. Yet, people are more likely to have additional, new experiences with ingroup members. As such, initial negative impression of the ingroup (due to “a few bad apples”) are likely to be updated by new experiences, but equivalent outgroup impressions are not due to more limited outgroup experiences (Denrell, 2005). The motivation in this case is interpersonal
(e.g., “avoid unpleasant individuals”) rather than group-based (e.g., “avoid outgroups”). Additionally, because people tend to have larger ingroup samples, ingroup experiences are often more variable than outgroup experiences, which can 11 Partisan-Motivated Sampling further bias interpretations (e.g., the in-group heterogeneity effect; Konovalova &
Le Mens, 2020). Collectively, these sampling perspectives emphasize how group- based bias may arise as a product of skewed experiences, rather than motivated, group-serving preferences.
Motivated Information Seeking
Group-based bias may arise both in a data-driven way due to the distribution of information in the environment, and when information search is terminated
(Denrell, 2005; Fiedler, 2000; Juslin et al., 2007). However, people are not always passive in their search for information. Instead, people may actively search for congenial information in their environment in order to confirm pre-existing beliefs.
For example, one model suggests that these effects can be attributed to a positive test strategy (Klayman & Ha, 1987), whereby people are more likely to sample cases that are expected to have certain properties rather than cases that are not
(e.g., Wason, 1960). Positive test strategies make pre-existing beliefs harder to disconfirm by skewing the sampling process to produce more potentially confirming than disconfirming evidence, regardless of their underlying distribution
(Kunda et al., 1993). Importantly, this process is not mutually exclusive with sampling models of group-based bias (e.g., Fiedler, 2000). 12 Partisan-Motivated Sampling
Beyond positive test strategies, there are a number of models that describe people’s preferences for congenial information, notably, selective exposure (also referred to as congeniality bias or confirmation bias; for meta-analysis, see: Hart et al., 2009). From this perspective, people may select more congenial relative to uncongenial information in order to defend their attitudes, beliefs and behaviors from challenges (e.g., Albarracín et al., 2005; Festinger, 1962). Within this framework, the variable of interest is typically measured as the count of participants’ choices from a list of congenial and uncongenial alternatives (e.g.,
Fischer et al., 2008; Fischer & Greitemeyer, 2010), or by the amount of time participants devote to viewing congenial versus uncongenial information (e.g.,
Ashmore & Del Boca, 1981; Ditto & Lopez, 1992; Olson & Zanna, 1979).
A challenge of integrating sampling models with motivated information seeking, at least in the context of partisan biases, is that they tend to trade in different types of information. First, sampling models emphasize quantities of information and their distributional properties to explain evaluative biases (e.g.
Fiedler, 2000; Lindskog et al., 2013). In many cases, bias is also quantified as a deviation from a true numerical value (e.g. Denrell, 2005; Fiedler, 2000). In contrast, it is often hard to gauge bias in such a statistical sense in studies on partisan-motivated information seeking, as the experienced information tends to be 13 Partisan-Motivated Sampling qualitative and lacking a formal connection to the outcome (Ditto et al., 2003;
Ditto & Lopez, 1992; Frimer et al., 2017; Kahan, 2016; Keys & Schwartz, 2007).
Second, the information used in studies on partisan-motivated information seeking tends to be explicitly labeled as congenial and uncongenial, such that people know the outcome of their choices in advance. Conversely, a premise of sampling models is that people do not know a priori the congeniality of information (e.g., whether an interaction will be pleasant or unpleasant) and only learn about its congeniality via their sampling behavior.
Recent work has begun to bridge the gap between motivated information- seeking and sampling perspectives on group-based biases (Bergh & Lindskog,
2019). In these studies, participants were assigned to novel, minimal groups (Tajfel
& Billig, 1973) and proceeded to gather information about ingroup and outgroup members (e.g., IQ scores). Unbeknownst to participants, half were assigned to a condition where the ingroup had statistically higher IQ scores and the other where the ingroup had statistically lower IQ scores than the outgroup. This research found that participants sampled from their novel ingroup first and more often than the outgroup. Moreover, participants accurately estimated real-group averages, but only when the ingroup de facto had higher IQ scores. When the ingroup had lower
IQ scores, participants estimated both groups as being equally intelligent. 14 Partisan-Motivated Sampling
Collectively, these findings provide initial evidence for how sampling and interpretive biases can interact, and in the minimal group context, manifest in over- estimations of ingroup averages. It remains unknown whether these findings generalize to how people gather information pertaining to real polarized groups, such as Democrats and Republicans.
Current Experiments
The current experiments incorporate both sampling and motivated interpretation perspectives on partisan bias by examining how Democrats and
Republicans gather and interpret information about each other (Experiment 1a) and their political candidates (Experiment 1b and 2). We propose that people are more interested in their own group, which generates skewed samples of information, which then fall prey to partisan-favoring interpretations. Experiments 1b and 2 were pre-registered (see https://aspredicted.org/blind.php?x=r4p4px and https://aspredicted.org/blind.php?x=sz7aq8).
Importantly, we do not claim that these processes are unique to the political partisan context. Indeed, studies on politically motivated reasoning have been criticized on the basis of not isolating partisan-specific explanations (Tappin et al.,
2020). We consider partisan motivated reasoning to reflect more generic ingroup- favoring information processing. Political identities are good candidates for 15 Partisan-Motivated Sampling examining broader intergroup dynamics for several reasons. First, there has been a recent surge in the strength and polarization of political identities, making them a particularly salient category in the United States (Iyengar et al., 2012, 2019;
Iyengar & Krupenkin, 2018; Mason, 2018; West & Iyengar, 2020). Second, biases between Democrats and Republicans are perhaps some of the clearest examples of how “us” versus “them” thinking manifest itself (Cikara et al., 2017). Third, many real-world groups may also be colored by their social status, which often represents a confounding factor for ingroup-outgroup explanations (Bergh et al., 2016).
Studies of political partisan biases do not suffer from this confound, as Democrats and Republicans are ambiguous in terms of their relative status.
To date, research has primarily examined partisan biases using qualitative information (e.g., statements on emotionally-laden issues such as abortion or gun rights; MacKuen et al., 2010; Weeks, 2015). In the current experiments, we use numerical information (i.e. political knowledge scores [Experiment 1a], and fact- check ratings [Experiments 1b and 2]) about groups that enables an explicit connection between the information people sample to their subsequent evaluations.
By stripping away the polarizing policies and emotion-laden issues present in qualitative information, we also reduce normative pressures that introduce 16 Partisan-Motivated Sampling potential confounds (Tappin et al., 2020), and provide a strong test of the robustness of partisan biases in minimalistic contexts (Tamir & Hughes, 2018).
The two outcomes of interest across experiments are people’s sampling behavior and their subsequent evaluations. To study the role of choice and sampling biases, we allowed participants to freely gather as much information they needed to confidently evaluate ingroup and outgroup attributes. In this way, our design emulates real-world sampling environments where people are free to gather information about different groups, but are also unable to gather all of the information due to time and/or energy constraints. Regarding sampling behavior on aggregate, we predict that participants will (1) begin sampling first from their own group, and (2) sample overall more from their own group. This is in line with prior findings suggesting that people are by default more interested in their own group
(Bergh & Lindskog, 2019). We also predict that the tendency to sample first from the ingroup will be driven by participants with high collective self-esteem and also those who identify most strongly with their political affiliation, in line with prior work connecting these individual differences in group-based motives to partisan bias (Crocker & Major, 1989; Mason, 2018).
To examine how biases in sampling behavior interact with biases in interpretations, we manipulated the first piece of sampled information to be 17 Partisan-Motivated Sampling positive or negative. If most participants begin sampling from the ingroup, this will introduce systematic biases in their overall ingroup experiences (but not outgroup experiences). This is because an overly positive or negative initial experience will deviate from the true group mean. If biased evaluations depend solely on motivated sampling, rather than motivated interpretations, then participants should be equally likely to over-or-under estimate ingroup averages based on their overly positive or overly negative initial experiences. That is, both positive and negative initial experiences should lead to over- and under-estimations of group means, respectively, if participants evaluate groups based solely on the sampled information. Devoid of any motivated interpretations, we would also expect these over- and under-estimations to be of equal magnitude. Conversely, if biased evaluations depend solely on motivated interpretations, rather than motivated sampling, then we would expect participants to be equally likely to begin sampling from the ingroup and outgroup. Furthermore, if evaluative biases were solely reflecting motivated interpretations, then we might expect a constant degree of ingroup boosting, irrespective of whether the first sample was positive or negative
(i.e. people should overestimate the ingroup both when it appears initially good or bad). In contrast to the isolated effects of sampling and interpretive biases, we predict: (1) that the majority of participant will begin by sampling from their own 18 Partisan-Motivated Sampling group, representing a sampling bias, and (2) that participants will then selectively attend to congenial sampled information by overestimating the ingroup following a positive initial sample, but not underestimating the ingroup following a negative initial sample (i.e. an interaction with sampled information), representing an interpretive bias. In other words, we predict that biased evaluations depend on the interaction of sampling biases (i.e. generating more variability in one group;
Konovalova & Le Mens, 2020), and interpretive biases that shift attention towards congenial information (Bergh & Lindskog, 2019).
To examine the influence of the overall congeniality of the sampling environments on sampling and evaluations, we manipulated the means and distributions of ingroup and outgroup information, such that (1) the ingroup will either be de facto better, (2) the ingroup will be de facto worse, or (3) both the ingroup and outgroup will be the same across all experiments (hereafter real group differences). In line with work demonstrating that people approach positive and negative information differently (Baumeister et al., 2001; Ditto & Lopez, 1992; Ito et al., 1998; Sharot & Garrett, 2016), we predict that participants’ sampling trajectories will diverge over time based on the real group differences and first sample valence. 19 Partisan-Motivated Sampling
Regarding evaluations, manipulating the real group differences provides a second opportunity to examine how sampling and interpretive sources of bias interact. For instance, if participants are not biased in their interpretations of the sampled information, and instead are data-driven, then their evaluations about ingroup and outgroup members should be similarly accurate, regardless of the congeniality of the environment. However, we predict that evaluations will interact with the overall congeniality of the environment, such that participants will be quite accurate in detecting the direction and magnitude of the real group differences, but only when the ingroup is de facto better. In contrast, when the ingroup is de facto worse, we predict that participants will be skeptical of uncongenial information and estimate both groups as the same by downplaying the outgroup average (Bergh & Lindskog, 2019). Lastly, when both groups are statistically the same, we predict that participants will perceive the ingroup as better, consistent with research showing ingroup-favoring biases of identical information (Hughes et al., 2017; Tappin et al., 2017; Westerwick et al., 2017).
Taken together, the two manipulations provide testable hypotheses regarding the expected evaluations if participants are only biased in their sampling behavior, their interpretations, or if biased evaluations depend on the interaction of biased sampling and interpretations. 20 Partisan-Motivated Sampling
We begin by reporting Experiments 1a, 1b and 2 that examine the influence of partisan motivations on sampling and evaluation in different environments, followed by 4 internal meta-analyses that ensure sufficient power to test for different motivations associated with biased sampling (Meta-Analysis 1) as well as different sampling mechanism that give rise to biased or accurate evaluations
(Meta-Analyses 2-4). In Experiment 1a, participants gathered information about
Democrats and Republicans’ political knowledge to evaluate which group was more knowledgeable. In Experiment 1b, we replicate and extend Experiment 1a by employing a sampling task with potentially higher ecologically validity. Here, participants gathered non-partisan fact-check ratings from a debate between a
Democrat and Republican candidate locked in a tight race. Experiment 2 extends these findings by replacing the first sample valence manipulation with a global opinion from pundit experts to examine how broader impressions influence sampling and evaluations. Meta-Analysis 1 tests whether individual differences in group-based motivations predict biases in sampling behavior. Meta-Analysis 2 tests whether sampling first from the outgroup can reduce evaluative biases. Meta-
Analysis 3 examines how sampling trajectories change over time as a function of the environment (i.e. valence of initial experience and real group differences), and if specific sampling strategies are related to subsequent biases in evaluations. 21 Partisan-Motivated Sampling
Meta-Analysis 4 tests whether biases in sampling behavior generate more variable ingroup experiences and examines how more variable ingroup experiences increase evaluative biases based on when participants decide to stop sampling.
Experiments 1a and 1b: Sampling Information About Group Members and
Representatives
We first conducted two experiments on how people form political impressions based on sampling information about different group members
(Experiment 1a) and group representatives (Experiment 1b). Experiment 1a examined how people gather information about the political knowledge of ingroup and outgroup members. Experiment 1b provides a conceptual replication and extension of Experiment 1a where we changed the context and nature of the sampled information. Here, participants gathered fact-check ratings from a
Democrat and Republican candidate during a debate. Taken together, we sought to illustrate that group-motivated sampling influences both how people aggregate impressions from multiple individuals and aggregate different pieces of information about particular ingroup and outgroup representatives. This is an important distinction, given the general emphasis on differences between interpersonal and intergroup cognition and bias (Baumeister & Finkel, 2010; Tajfel 22 Partisan-Motivated Sampling
& Billig, 1973; Willer et al., 1989), and that sampling from social circles versus individuals may engender different judgement biases (Galesic et al., 2012). Aside from the described differences in the cover stories and the meaning of the ratings, the two experiments were identical. As such, Experiment 1b provided a close replication of the initial findings, following a pre-registered protocol.
Methods
Participants
Participants (N = 599 and 974 in Experiment 1a and 1b, respectively) were recruited from Amazon’s Mechanical Turk (MTurk). In both experiments, participants received $2.00 payment for completing the roughly 20-minute study.
All participants self-identified their political identity on a 7-point scale from 1
(very liberal) to 7 (very conservative), with 4 being neither. The task was only made visible to participants who self-identified as either a Democrat or
Republican. However, a few participants still responded with a 4 (i.e. they do not identify as either liberal or conservative) and they were excluded from all analyses.
In terms of the breakdown of political affiliation, 58% and 56% of participants self-identified as Democrats, in Experiment 1a and 1b respectively. Participants were also excluded if they failed 2 or more attention checks (e.g., “If you are paying attention click 1”) that were randomly placed throughout personality 23 Partisan-Motivated Sampling questionnaires at the end of the experiments (see Meta-analysis 1; for full list of individual difference covariates, see Supplemental Materials). After applying these exclusion criteria, 540 and 905 participants remained in Experiment 1a and 1b, with mean ages of 41.05/40.83, SD = 12.31/12.64, in the respective experiments. In both experiments, 54% were female.
Simulations using the Simr package (Green & Macleod, 2016) in R revealed an observed power of .65 to detect the interaction of Sampled Group × First
Sample Valence in Experiment 1a. These simulations indicated that increasing our sample size by roughly 50% would provide sufficient power (.80) for an equivalent effect size for Experiment 1b.
Procedure
The experiments each consisted of four parts: (1) a cover story, (2) a sampling task, (3) evaluations, and (4) personality and demographic questions to test for moderating effects. In Experiment 1a, participants received a cover story that described an ongoing debate in the US regarding whether Democrats or
Republicans are more knowledgeable about US history and politics. Participants were told that we had created a test similar to the US citizenship test to better understand which group is actually more knowledgeable, and they were led to believe that we had administered it to 1,000 MTurk workers. The cover story 24 Partisan-Motivated Sampling further explained that the purpose here was to examine how Democrats or
Republicans form impressions about political knowledge, based on reviewing the results from the supposed previous experiment. They were told that they would have a chance to sample political knowledge scores ostensibly from these previous
MTurk workers, and then evaluate the knowledge scores of each group.
In Experiment 1b, the cover story described the task as gathering fact-check ratings from a debate on different policies from a Democrat and Republican candidate locked in a tight race. Participants were further told that it is often difficult to evaluate in real-time whether a political candidate is honest during a debate, and as such, we compiled nonpartisan fact-check ratings during a debate between the candidates (see Supplemental Materials for full cover story). Overall, the scenarios were designed to decontextualize the information environment (e.g., not using actual political candidates or specific policies) in order to quantify motivated behavior devoid of salient pre-existing beliefs (e.g., about 2nd amendment rights).
Across both Experiments, on each trial in the sampling task (part 2), participants selected to sample a knowledge score from an ingroup or outgroup member, which was represented by a numerical score from 0-100. Participants were subsequently given an option to continue sampling, or stop sampling when 25 Partisan-Motivated Sampling they felt they could confidently evaluate each group’s knowledge. By manipulating the congeniality of the environment, we tested the extent to which evaluative biases reflect sampling behavior and/or interpretive biases. That is, if people sample similarly between groups and across congenial and uncongenial environments, but evaluations vary in their degree of bias, then evaluative biases should reflect motivated interpretations more than biased samples. In Experiment
1a, participants were thus randomly assigned into Real Group Differences where their ingroup was: (1) statistically more knowledgeable than the outgroup by .5 SD
(ingroup mean: M = 68.5; SD = 12; outgroup mean: M = 62.5; SD = 12), (2) less knowledgeable than the outgroup by .5 SD (ingroup mean: M = 62.5; SD = 12; outgroup mean: M = 68.5; SD = 12), or (3) the same as the outgroup (both groups:
M = 65; SD = 12). In Experiment 1b, we similarly manipulated the sampled data such that the ingroup candidate was either more, less, or equally honest as the outgroup candidate, on average. The numerical differences were identical to those in Experiment 1a (e.g., the average honesty scores of the ingroup candidate across topics [correct %, overall, in fact-checks] were manipulated to be 68.5, 65, or
62.5). The Real Group Difference distributions were chosen on the basis of previous work that has demonstrated distributions with a Cohens D of .5 are noticeably different (see Bergh & Lindskog, 2019). 26 Partisan-Motivated Sampling
Prior work also shows that early sampling experiences create systematic biases in information processing (Bergh & Lindskog, 2019; Harris et al., 2020).
Therefore, participants across both experiments were also randomly assigned to receive either a positive (i.e. 79 out of 100) or negative (i.e. 51 out of 100) initial score. In all, participants were randomly assigned to a 2 (First Sample Valence:
Positive or Negative) by 3 (Real Group Difference: ingroup more knowledgeable than, the same as, or less knowledgeable than the outgroup) design. The experiments were programmed in Qualtrics with imported values for the sampling task from a database on webtask.io.
Participants were asked to gather as much information about political knowledge scores in order to confidently estimate the overall knowledge of
Democrats and Republican (Experiment 1a), or honesty of the party representatives
(Experiment 1b; for more details of the sampling task, see Supplemental
Materials). Participants were informed that they would see two buttons with the labels “Democrat” and “Republican” and that selecting one of these buttons would present them with a piece of information about a group member or representative from that corresponding category. Scores for the two categories were 120 integers pulled from a truncated normal distribution, based on which condition that participant was randomly assigned to, ranging from 0-100. When participants 27 Partisan-Motivated Sampling clicked on one category, Qualtrics retrieved random numbers (with replacement) from the corresponding group dataset on webtask.io, except for the very first score, which was subject to the First Sample Valence manipulation. The position of the
“Democrat” and “Republican” buttons was counterbalanced across participants.
In Experiment 1a, the information presented after each sampling choice included the political affiliation of another presumptive MTurk worker, an anonymous worker ID, and their score from the political knowledge test ranging from 0 (completely unknowledgeable) to 100 (completely knowledgeable). The presentation order of the knowledge scores was individually randomized for each participant. In addition, the unique IDs—indicating that each score was ostensibly from a different person—was a randomly generated string of 6 letters and numbers.
These unique IDs were then randomly assigned on a trial-to-trial basis. After viewing an example trial, participants could freely gather information from the two categories until they felt they had collected enough information to make a judgement about which group was more knowledgeable about US politics.
In Experiment 1b, participants were similarly asked to gather information from the Democrat and Republican categories until they felt they had collected enough information to evaluate the honesty of each candidate. When they clicked 28 Partisan-Motivated Sampling on a candidate, they saw a fact-checking score for a particular de-identified topic in the debate (see Supplemental Materials for more details).
In the evaluation phase (part 3), participants were asked to evaluate each groups’ average knowledge score (Experiment 1a), or the average fact-checking scores of candidates (Experiment 1b) based on the sampled information on a 100- point slider scale from 0-100. The order of the ingroup and outgroup questions was randomized between participants. After the experimental procedure, participants were debriefed about the true intent of the experiment. De-identified raw data and analysis scripts for all seven experiments are publicly available for download: https://osf.io/9627e/?view_only=4f5c8a589e014c92a1fca171bb0e4369.
Results
Sampling Behaviors
In line with our prediction, more participants (71% and 70% in Experiment
1a and 1b) chose to sample the first piece of information from their own group (see
Figure 1A), binomial test H0 = .5: p < .00001, 95% CIs [66%, 74% in Experiment
1a, and 66%, 72% in Experiment 1b]. We also predicted that participants would gather more information from their own group, with the possibility that this tendency may change as a function of their group’s performance (Real Group
Difference), the valence of the first sample of information (First Sample Valence) 29 Partisan-Motivated Sampling and participant’s Political Affiliation. To test these hypotheses, we fit a Poisson generalized linear mixed effects model with First Sample Valence (positive [high test score] vs. negative [low test score]), Real Group Difference (ingroup better than, the same as, or worse than the outgroup), Sampled Group (ingroup vs. outgroup) and Political Affiliation (Democrat vs. Republican) as fixed factors and
Participants as a random factor. For this model, all predictors were effects coded.
Across all experiments we found no indication of overdispersion (all dispersion parameters > .75 and < 1.4; Korner-Nievergelt et al., 2015). All Poisson models were estimated using the glmer function of the lme4 package (Bates et al., 2015) in
R, and p-values for the mixed models were estimated with likelihood ratio tests.
In Experiment 1a, this model revealed a marginally significant effect of
Sampled Group, χ2(1) = 3.34, b = .044, SE = .02, z = 1.85, p = .06. In Experiment
1b, this effect was highly significant; χ2(1) = 19.15, b = .09, SE = .02, z = 4.44, p
< .00001. These effects illustrate that participants sampled more from the ingroup,
M = 6.50 and 5.20, SD = 4.00/4.37, in Experiment 1a/1b, than the outgroup, M =
6.20/4.74, SD = 4.60/4.01 (see Figure 1B). In the larger Experiment 1b, we also observed a significant main effect of Political Affiliation, χ2(1) = 6.88, b = .06, SE
= .023, z = 2.63, p = .008, illustrating that Democrats, M = 10.40, SD = 7.19, sampled significantly more than Republicans, M = 9.38, SD = 8.34. Finally, in 30 Partisan-Motivated Sampling
Experiment 1b there was a significant First Sample Valence × Real Group
Difference interaction, χ2(2) = 9.45, p = .008. This interaction indicated that when participants’ ingroup candidate was de facto worse, participants sampled significantly more after receiving an initial positive first sample compared to participants who received an initial negative first sample, b = -.08, SE = .03, z = -
2.43, p = .01. 31 Partisan-Motivated Sampling
a. b. s s e l e l E p p x e m l p m a p e a S r
S m i f
m a f o
S o e r
t r n e s e t b r
i b 1 m F a m u u N N
Ingroup Outgroup Ingroup Outgroup c. dd.. s s e e l E l p p x e m p l m a e p a S r S
i m
f m f a o o
e S
r
n r t e t s e b
r b 1 i b m F m u u N N
Ingroup Outgroup Ingroup Outgroup e. ff.. s s e e l l p E p e x m l m p a a p e S S
r m f i f a m o o S
r e r t e n e s r b t b i
2 F m m u u N N
Ingroup Outgroup Ingroup Outgroup
Figure 1. Proportion of initial ingroup and outgroup samples (Panels A, C, E), and average sampling behavior from the ingroup and outgroup (Panels B, D, F) as a 32 Partisan-Motivated Sampling function of experiment. Vertical bars denote standard error of the mean. SE for initial sample is too small to appear on figure. Evaluation Results
After the sampling phase, participants evaluated the average political knowledge scores of Democrats and Republicans (Experiment 1a) and average fact-check scores of the Democrat and Republican political candidate (Experiment
1b). We investigated the extent to which participants were biased in their evaluations of political knowledge as a function of First Sample Valence, Real
Group Differences and Political Affiliation by conducting a linear mixed model with Evaluated Group (Ingroup vs. Outgroup), First Sample Valence (positive vs. negative]), Real Group Differences (ingroup better than, the same as, or worse than the outgroup), and Political Affiliation (Democrat vs. Republican) as fixed factors and Participant as a random factor. This and all subsequent evaluation models were estimated using the lmer function of the lme4 package (Bates et al., 2015) in R.
Once again, we estimated p-values using likelihood ratio tests. The degrees of freedom for all contrasts were estimated using Satterthwaite approximations
(Kuznetsova et al., 2017).
In both experiments, this model revealed two consistent main effects and two consistent interactions. First, we observed a significant main effect of
Evaluated Group, χ2(1) = 37.87, b = 1.61, SE = .26, t = 6.13, p < .00001, in 33 Partisan-Motivated Sampling
Experiment 1a, and χ2(1) = 45.96, b = 1.61, SE = .23, t = 6.86, p < .00001, in
Experiment 1b. In line with our hypothesis, participants evaluated the ingroup as more knowledgeable, M = 65.89, SD = 8.49, and the ingroup candidate as more honest, M = 64.35, SD = 9.47 than the outgroup counterpart, M = 62.67/61.10, SD
= 8.91/11.34 (Experiment 1a/1b). Across the experiments, we also found a consistent main effect of First Sample Valence on evaluations, χ2(1) = 39.36, b = -
1.64, SE = .26, t = -6.33, p < .0001, in Experiment 1a, and χ2(1) = 11.05, b = -.85,
SE = .26, t = -3.30, p = .0008, in Experiment 1b. Specifically, participants provided higher ratings on both political knowledge and honesty after a positive first sample, compared to a negative sample. More importantly, this effect was consistently qualified by an Evaluated Group × First Sample Valence interaction, χ2(2) = 8.89, b = -.77, SE = .26, t = -2.98, p = .003, in Experiment 1a, and χ2(2) = 4.48, b = -.50,
SE = .23, t = -2.17, p = .03, in Experiment 1b. In the first experiment, simple contrasts revealed that participants rated their ingroup as more knowledgeable after viewing a positive first sample compared to a negative first sample, t(538) = 6.40, d = .56, p < .0001, and that this effect was attenuated following a negative first sample, t(538) = 2.30, d = .20, p = .02. Similar simple contrasts were observed in
Experiment 1b: Participants evaluated the ingroup candidate as significantly more honest than the outgroup following a positive first sample, t(903) = 6.06, d = .44, p 34 Partisan-Motivated Sampling
< .0001, and this effect was attenuated following a negative first sample, t(903) =
3.87, d = .23, p = .0001. These findings demonstrate that participants selectively integrated congenial experiences while downplaying uncongenial experiences in their evaluations (see Figure 2B and 2D).
Finally, the model also revealed a consistent Evaluated Group × Real Group
Difference interaction, χ2(2) = 29.97, b = -1.50, SE = .37, t = -4.12, p < .0001, in
Experiment 1a, and χ2(2) = 56.06, b = -1.90, SE = .33, t = -5.76, p < .0001, in
Experiment 1b. Simple contrasts in Experiment 1a revealed that when the ingroup was de facto more knowledgeable, participants evaluated the ingroup as significantly more knowledgeable than the outgroup t(536) = 7.98, d = .80, p
< .0001. When the ingroup was de facto less knowledgeable, however, participants estimating the ingroup and outgroup similarly, t(536) = .013, d = .005, p = .99.
When both groups were equally knowledgeable, participants once again estimated the ingroup as significantly more knowledgeable than the outgroup, t(536) = 2.65, d = .28, p = .008 (see Figure 2A).
Similarly, in Experiment 1b, simple contrasts revealed that when the ingroup candidate was de facto more honest, participants evaluated the ingroup candidate as significantly more honest than the outgroup candidate, t(901) = 10.11, d = .80, p
< .0001. Conversely, when the outgroup candidate was de facto more honest, 35 Partisan-Motivated Sampling participants evaluated the ingroup and outgroup similarly, t(901) = -.65, d = .05, p
= .51. Lastly, when both candidates were equally honest, participants evaluated the ingroup as more honest than the outgroup, t(901) = 3,04, d = .26, p = .002 (see
Figure 2C). The contrasts from both of the experiments suggest that people are sensitive to underlying properties when it favors their group, but are biased in favor of their group when it is unfavorable. Together, these results demonstrate that participants were adept at inferring the direction and magnitude of real group difference but only when the environment favored the ingroup. 36 Partisan-Motivated Sampling
Evaluated Group Ingroup Outgroup a. b. e e g g E d d x e e 70 70 p l l e w w r o i o m n n 60
60 e K K
n
l l t a a
1 c c i i a t t
50 i 50 i l l o o P P Worse Same Better Negative Positive Real Group Differences First Sample Valence c. d. y y E t t s x s 70 70 e e p n n e o r o i m H H
60 60 e e e t t n a a t
d d 1 i i b d d 50 50 n n a a C C Worse Same Better Negative Positive Real Group Differences First Sample Valence e. f. y y t t s E s e x e
70 n 70 p n o e o r H H i
m e e t
t 60
60 e a a n d d i t i
d 2 d n n
50 a 50 a C C Worse Same Better Negative Positive Real Group Differences First Sample Valence
Figure 2. Evaluations of ingroups and outgroups as a function of Evaluated Group and Real Group Differences (Panels A, C, E) or Valence of the initial impression (Panels B, D, F) in Experiment 1a (Panel A and B), Experiment 1b (Panels C and 37 Partisan-Motivated Sampling
D) and Experiment 2 (Panels E and F). Worse, Same and Better is stated in reference to the ingroup. Vertical bars denote standard error of the mean. Experiment 1a and 1b Discussion
These experiments provide converging evidence that biased evaluations depend on the interaction of biased sampling and biased interpretations. Across both experiments, we find that participants sampled first and most often from their own group, representing a sampling bias. In addition, participants asymmetrically integrated initial experiences based on congeniality, such that congenial experiences were integrated into evaluations whereas uncongenial experiences were not, representing an interpretive bias. Likewise, participants were accurate in estimating the direction and magnitude of the real group differences when their group was de facto better, but failed to evaluate the outgroup as better when the ingroup was de facto worse. Thus, biased samples generated more variable ingroup experiences which provided participants with more opportunities to selectively integrate congenial information. These findings are also consistent with research in minimal groups, suggesting that people seem to sample in a similar way when it comes to estimating a new group attribute for known groups as they do for novel groups (Bergh & Lindskog, 2019).
An important distinction in Experiment 1b is that participants sampled information from one ingroup and one outgroup representative (as opposed to 38 Partisan-Motivated Sampling sampling information about groups of people in Experiment 1a). Sampling biases were reliable across these different sampling contexts (i.e. learning about groups versus learning about individual group representatives). The only distinguishable difference between these sampling environments is that participants sampled more information overall when gathering information from groups of people as compared to individual representatives. This may be due to greater variability across groups of people rather than within a single representative of a group.
Nonetheless, the overall consistency in sampling behavior aligns with findings in social identity research suggesting that people treat leaders as group prototypes
(Hogg, 2001). If people treat representatives as group prototypes, then they might exploit information about prototypes as a shortcut to uncovering group attributes.
People may therefore sample less information about prototypes compared to contexts where they must make similar inferences about a group from multiple individual group members.
Experiment 2: The Influence of Global Impressions on Sampling Political
Candidate Honesty
When navigating information environments, people often rely on the opinions of experts from their social group to make evaluations (Cohen, 2003). To 39 Partisan-Motivated Sampling date, little attention has been paid to how global impressions influence downstream information sampling. In Experiments 1a and 1b, we tested the impact of initial impressions on sampling behavior and evaluations by manipulating the first numerical piece of information to be either high (79 out of 100) or low (51 out of
100). In Experiment 2, we instead test the influence of a more global initial impression by replacing the first sample valence manipulation with an initial pundit impression manipulation that is either favorable or unfavorable to the ingroup.
Method
Participants
Participants (N = 1,076) were recruited from Amazon’s Mechanical Turk and paid for their participation using the same recruitment strategy as Experiments
1a and 1b. The sample had a mean age of 38.3, SD = 12.1 (57% males). N = 986 remained for analyses after applying the same exclusion criteria as Experiments 1a and 1b.
Procedure
The procedure was identical to Experiments 1a and 1b except for two key differences. First, the pundit impression manipulation took place prior to the sampling portion of the task (as opposed to manipulating the valence of the first 40 Partisan-Motivated Sampling sample). Specifically, prior to sampling, participants were provided either favorable or unfavorable impressions from an ingroup or outgroup pundit about the relative difference in honesty between the Democrat and Republican candidate. A second key difference was the nature of the information provided by the pundits.
The pundit impression manipulation provided a relative difference between the two candidates (“I believe the Democrat/Republican candidate is simply more honest”), as opposed to a numerical score. Participants were led to believe that the pundit impression manipulation was a pundit’s expert opinion about the information the participant was about to sample (i.e. fact-check scores), but that the fact-check scores themselves were drawn from a nonpartisan 3rd party source.
Thus, participants were randomly assigned to a 2 (Initial impression valence: positive ingroup [negative outgroup] vs. negative ingroup [positive outgroup]) by 2
(Source: ingroup vs. outgroup) by 3 (Real Group Difference: ingroup more knowledgeable than, the same as, or less knowledgeable than the outgroup) design.
Like Experiments 1a and 1b, the experiment consisted of four parts: (1) a cover story, (2) a sampling task, (3) evaluations, and (4) personality and demographics.
Results
Sampling Behavior 41 Partisan-Motivated Sampling
Preliminary results indicated that the source manipulation (i.e. coming from an ingroup or outgroup candidate) did not significantly alter sampling behavior
(see Supplemental Materials for model comparison with source as a separate factor). Therefore, the current analyses drop the source of the pundit impression manipulations into a single Valence of Pundit Impression factor (“I believe the
Democrat/Republican candidate is simply more honest”).
We first investigated from which group participants began to gather information. Replicating Experiments 1a and 1b, we found that 65% of participants picked the first sample from the ingroup (see Figure 1E), H0 = .5: p < .00001, 95%
CI [61%, 67%]. However, because the global impression manipulation preceded sampling in this experiment, we also ran this analysis separately based on whether the pundit impression was positive (indicating that the ingroup candidate is more honest than the outgroup) or negative (indicating that the outgroup candidate was more honest than the ingroup). We found that after a positive pundit impression,
77% of participants picked the first sample from the ingroup, H0 = .5: p < .00001,
95% CI [72%, 80%]. In contrast, after a negative pundit impression, only 51% of participants picked the first sample from the ingroup, H0 = .5: p < .52, 95% CI
[47%, 56%]. These results demonstrate that pundit impressions differentially affect 42 Partisan-Motivated Sampling which candidate participants sample from first based on whether the impression is congenial or uncongenial (see Figure 3).
We again expected participants to gather more information from their own group overall. To investigate, we fit a Poisson generalized linear mixed effects model identical to Experiments 1a and 1b. It revealed two significant main effects and one significant interaction. First, replicating Experiments 1a and 1b, we observed a significant main effect of Sampled Group, χ2(1) = 24.29, b = .09, SE
= .02, z = 5.00, p < .00001. Participants sampled significantly more from the ingroup, M = 5.20, SD = 4.40, than the outgroup, M = 4.70, SD = 4.00 (see Figure
1F). Next, we replicate Experiment 1b and observed a significant main effect of
Political Affiliation, χ2(1) = 11.24, b = .07, SE = .02, z = 3.36, p = .0007, illustrating that Democrats, M = 10.30, SD = 7.40, sampled significantly more than
Republicans, M = 9.5, SD = 8.40, respectively. Finally, we observed a significant
Valence of Pundit Impression × Real Group Difference interaction, χ2(2) = 7.98, p
= .018. This interaction indicated that participants sampled significantly more following a positive pundit impression compared to participants who viewed a negative pundit impression, when both candidates were in fact equally honest, b
= .09, SE = .03, z = -2.79, p = .005. We did not observe any differences in sampling behavior when participant’s ingroup candidate was more honest or less 43 Partisan-Motivated Sampling honest than the outgroup candidate as a function of the Valence of Pundit
Impression (all p-values > .34). Taken together, participants sampled more information following a positive pundit impression and were overall more interested in their ingroup as they sampled more information from them in total.
Figure 3. Proportion of initial ingroup and outgroup samples as a function of a positive (Panel A) and negative (Panel B) pundit impression. SE are too small to appear on figure. 44 Partisan-Motivated Sampling
Evaluation Results
After the sampling phase, participants were asked to evaluate the average honesty of the Democrats and Republican candidates. Once again, we report results with source dropped. To investigate the extent to which participants were biased in their evaluations of candidate honesty as a function of the manipulations and political affiliation, we fit a linear mixed model identical to Experiments 1a and 1b.
This model revealed one significant main effect, two significant two-way interactions and one significant three-way interaction.
First, we found a significant main effect of Evaluated Group, χ2(1) = 44.05, b = 1.54, SE = .23, t = 6.5, p < .00001. Replicating Experiment 1a and 1b, participants evaluated the ingroup, M = 65.72, SD = 10.98, more favorably than the outgroup, M = 62.63, SD = 11.46. In addition, the model also revealed a significant
Evaluated Group × Real Group Difference interaction, χ2(2) = 47.44, p < .0001.
Simple contrasts revealed that when the ingroup candidate was de facto more honest, participants evaluated the ingroup as significantly more honest than the outgroup, t(981) = 9.03, d = .70, p < .0001. When the ingroup candidate was de facto less honest, participants evaluated the candidates similarly, t(981) = -.68, d =
-.05, p = .49. When both candidates were equally honest, participants again evaluated the ingroup as more honest than the outgroup, t(981) = 3.18, d = .25, p 45 Partisan-Motivated Sampling
= .001 (see Figure 2E). These results again demonstrate that participants accurately inferred the underlying group differences, but only when the ingroup was de facto better.
We also observed a significant Real Group Differences × Valence of Pundit
Impression interaction, χ2(2) = 6.57, b = -.88 SE = .37, t = -2.33, p = .037. Simple contrasts revealed that when both candidates were equally honest, participants made higher evaluations after receiving a positive pundit impression relative to a negative pundit impression, t(978) = 2.36, d = -.21, p = .01 (all other p-values
>.21). Importantly, this interaction was further qualified by an Evaluated Group ×
Real Group Difference × Valence of Pundit Impression interaction, χ2(2) = 6.29, b
= -1.5 SE = .36, t = -4.12, p = .043. Simple contrast revealed that the effect of the
Valence of the Pundit Impression when both candidates were equally honest was asymmetrically assimilated into evaluations based on whether the participant was evaluating the ingroup or outgroup candidate. Specifically, whereas a positive pundit impression increased ingroup evaluations relative to outgroups, t(979) =
3.55, d = .38, p = .0004, a negative pundit impression did not significantly alter ingroup evaluations relative to outgroups, t(979) = .95, d = .10, p = .33. In sum, participants integrated the pundit impression when it was congenial, but downplayed it when it was uncongenial. 46 Partisan-Motivated Sampling
Discussion
Experiment 2 replicates findings from Experiments 1a and 1b with an important distinction. Here, participants received a favorable or unfavorable global pundit impression prior to information sampling. Evaluative biases did not differ between initial impressions that were based on early sampled data points
(Experiments 1a and 1b) and global pundit impressions (Experiment 2). These two forms of initial impressions, despite being qualitatively different, led to quantitatively similar biases in sampling behavior and evaluations. This suggests that people categorize and weigh qualitatively different forms of initial experiences similarly, and highlights a unique ability for people to exploit both subtle versus transparent as well as specific versus global information and arrive at congenial conclusions.
In addition to demonstrating that early global impressions contribute to biased information processing, the pundit impression manipulation also helps to disentangle different motivations that generate biased sampling behaviors and biased evaluations. These biases may arise from a motivation to affirmatively test a hypothesis (Klayman & Ha, 1987) and more general group-based expectations that are not specifically focused on favoring the ingroup (Fiedler et al., 1999).
However, it is reasonable to expect that participants’ search for information is also 47 Partisan-Motivated Sampling shaped by more specific group-based motivations tied to ingroup favoritism (e.g., greater intrinsic interest in the ingroup). Experiment 2 created a context in which group-motivated and non-group-motivated biases can be aligned or misaligned.
When the pundit indicated that the ingroup candidate was more honest, a positive test strategy of the pundit assertion aligns with motives to first and primarily learn about the ingroup. When these motives aligned, we observed that the majority of participants sampled first from the ingroup. In contrast, when the pundit indicated that the outgroup candidate was more honest, a positive search strategy alone would point toward sampling from the outgroup first, which conflicts with a motivation to primarily learn about the ingroup. When the motives misaligned, we observed an overall ambivalence about where to sample first, with only 51% of participants sampling first from the ingroup. This suggests that a greater intrinsic interest for ingroups is moderated, but not completely overridden, by positive test strategies (e.g., assertion that the outgroup is better). The lack of a dominant choice in the negative pundit condition also suggests that individual differences in ingroup favoring motives may explain where a person starts sampling.
Meta-Analysis 1: Individual Differences in Group-Serving Motivations
Moderate Sampling 48 Partisan-Motivated Sampling
We find robust evidence that people begin sampling from their own group and that initial experiences are asymmetrically integrated into evaluations based on its congeniality. While Experiment 2 helped to elucidate the specific motivations driving participant’s propensity to sample first and foremost from their own group, it remains unclear to what extent there are systematic individual differences in these choices. In order to ensure sufficient power and generalizability, we aggregated the data and conducted an internal meta-analysis (Goh et al., 2016).
One natural candidate to test for group-specific motivation is liberal versus conservative identification. In fact, compared to concrete attitudes on social and political issues, political identification can be a better predictor of partisan prejudice (Mason, 2018). Another natural candidate with theoretical connections to group-based motives is collective self-esteem (CSE). Prior work has demonstrated that individuals high on CSE tend to evaluate ingroup members more positively and outgroup members more harshly (Crocker & Luhtanen, 1990). The public facet of CSE in particular has been shown to moderate the extent to which individuals attempt to protect or enhance their collective identities (Crocker & Major, 1989), attributes often associated with group-serving motivations. Therefore, we test for converging evidence that group-based motivations drive sampling behavior using these individual differences measures. We predict that there will be a stronger 49 Partisan-Motivated Sampling tendency to sample first from the ingroup for participants with strong political identification and with high CSE. Notably, our predictions are specific to first samples, as sampling behavior on aggregate is subject to different motivations elicited by real group differences (e.g., participants may sample more from the outgroup over time when they are worse out of spite). We test these hypotheses by regressing participants' propensity to begin sampling from the ingroup or the outgroup (0=ingroup; 1=outgroup) on political identification and CSE.
Importantly, we tested these predictors in different models because we were interested in showing convergent (common) effects, rather their unique contributions in a regression with both (See Supplemental Materials for a multiple regression model demonstrating their unique effects).
Method
Participants
Participants (N = 1,445) were aggregated across Experiments 1a and 1b.
Experiment 2 was not included in this analysis because here we test the hypothesis that group-based motives predict sampling first from the ingroup without any broader knowledge about the underlying distribution. This was not the case in
Experiment 2 because participants were provided pundit impressions about the 50 Partisan-Motivated Sampling relative difference in honesty between the two candidates prior to any sampling taking place.
Measures
Political Identification. Identification with an ideological label was operationalized using a standard 7-point measure of ideological self-description: 1
(liberal, strong), 2 (liberal, not very strong), 3 (moderate/neither, lean liberal), 4
(moderate/neither), 5 (moderate/neither, lean conservative), 6 (conservative, not very strong), and 7 (conservative, strong). This measure was collapsed across conservatives and liberals to achieve a “party-blind” measure of identification strength and subsequently coded as a factor with 3 levels: “Very Affiliated with
Party”, “Somewhat Affiliated with Party” [reference group], and “Closer to my
Party”. We chose to dummy code in order to make meaningful comparisons between those who identified moderately and those high vs. low in strength of identification.
Collective Self-Esteem. Collective self-esteem (CSE) – capturing the portion of an individual’s self-concept that is based on their memberships within a social identity – was measured via Luhtanen and Crocker’s 16-item scale (Luhtanen &
Crocker, 1992). CSE is broken into 4 facets, each associated with different aspects of social identity. These include membership self-esteem (e.g., “I am a worthy 51 Partisan-Motivated Sampling member of my group”), private collective self-esteem (e.g., I feel good about the group I belong to”), public collective self-esteem (e.g., “overall my group is considered good by others”), and importance to identity (“the group I belong to is an important reflection of who I am”).
Results
The current logistic regression models were estimated in R studio with the glm function. To test whether participants who identified strongly with their political identity were more likely to begin sampling from their own group as compared to those who identified more moderately, we regressed political ideology onto first sample choice, controlling for experiment as a covariate. This model demonstrated that compared to participants with moderate political affiliation, participants who were “Very Affiliated with Party” were significantly more likely to sample from the ingroup first, = .45, SE = .14, z = 3.27, OR = 1.56, p = .001
(Figure 4A). Participants who identified as “Closer to Democrat/Republican” were not significantly more likely to sample from the ingroup or outgroup, p >.2. We found no effect of experiment on first choice (p > .71).
Next, we regressed CSE onto first sample choice aggregated across all facets, once again controlling for experiment as a covariate. This model demonstrated that participants with higher CSE were significantly more likely to 52 Partisan-Motivated Sampling sample from the ingroup first, = .11, SE = .06, z = 1.98, OR = 1.18, p = .047.
Next, we re-estimated the model separately for each facet of CSE to examine if these effects were driven by a particular facet of the scale. This model demonstrated that the effect of CSE on first choice was driven by the public facet, such that participants higher on CSE Public were significantly more likely to sample from the ingroup first, = .18, SE = .05, z = 3.13, OR = 1.19, p = .001 (see
Figure 4B), all other facet p-values > .28. Once again, we found no effect of experiment (p > .98).
a. b. e e l l p p m m a a S S
t t s s r r i i F F
f f o o
y y t t i i l l i i b b a a b b o o r r P P
- + Political Affiliation Strength CSE Public
Figure 4. Probability of sampling from the ingroup first as a function of “party- blind” strength in political affiliation (Panel A). Probability of sampling from the ingroup first as a function of collective self-esteem of the public facet (Panel B). Vertical bars denote standard error of the mean. Discussion 53 Partisan-Motivated Sampling
The current Meta-analysis confirmed our hypotheses that individuals who identify more strongly with their political identity and with higher CSE are significantly more likely to begin sampling from their own group. These findings extend work connecting strength of political identification and CSE to partisan prejudice and interpretive biases by demonstrating that these individual differences also contribute to group-motivated information sampling.
These findings provide insight into the motivations driving first sample choices. Prior work has demonstrated that individuals who identify strongly with their political affiliation are also more likely to rate their ingroup highly across a variety of traits, suggesting that they may have an a priori hypothesis that their ingroup is better (Mason, 2018). As such, one explanation is that those who identify more strongly with their political affiliation sample first from the ingroup to test an internally generated hypothesis that their ingroup is better.
Meta-Analysis 2: Sampling from the Outgroup First Reduce Biases, but Only
When they Outperform the Ingroup
Our results demonstrate that people have a strong tendency to sample first and foremost from their own groups and that this tendency is stronger for individuals with stronger political convictions and those who have higher CSE. If 54 Partisan-Motivated Sampling people are more likely to sample first from the ingroup, and first experiences are very positive or negative (as in these experiments), then people will have more varied ingroup experiences than they have outgroup experiences. To the extent that a person asymmetrically draws on varied experiences about the ingroup based on congeniality, then ingroup evaluations should be more biased than outgroup evaluations (Bergh & Lindskog, 2019). Although the majority of participants began by sampling from their ingroup, not all did. What happens when people instead start sampling from the outgroup? People who start sampling from the outgroup in these experiments should have more variable outgroup experiences, but we do not yet know how those outgroup experiences are interpreted.
One possibility is that people who start sampling from the outgroup are more interested in testing whether the outgroup is bad, compared to testing whether the ingroup is good (analogous to the difference between outgroup derogation and ingroup favoritism; Brewer, 1999). As such, people may attach more weight to initially negative, compared to positive, outgroup experiences in their outgroup evaluations. A second possibility is that people who start sampling from the outgroup are less attached to their own group, more curious about the other group, and less inclined to evaluative biases. If so, they might interpret the data more symmetrically (i.e. being equally likely to over- and underestimate the outgroup as 55 Partisan-Motivated Sampling a function of whether their first experience was very positive or negative). To examine these different possibilities, we conducted a meta-analysis to test these questions and ensure sufficient power and generalizability (Goh et al., 2016).
It is important to note that this question and analysis concerns results from
Experiments 1a and 1b where participants did not have any prior knowledge about ingroups and outgroups before sampling. This was not the case in Experiment 2, because participants already had received a positive or negative impression before their first sample, thereby obfuscating the interactive effects of the valence of initial experiences and first choice. Therefore, the current analyses aggregate participants who freely sampled from the outgroup first across Experiments 1a and
1b.
Method
Participants
Participants (N = 431) who sampled from the outgroup first were aggregated across Experiments 1a and 1b. This particular subset of the original sample had a mean age of 42.3, SD = 12.77 (55% females).
Results
To examine evaluative biases for participants who started sampling from the outgroup first, we fit a linear mixed model with Evaluated Group (ingroup vs. 56 Partisan-Motivated Sampling outgroup), Initial Experience (positive vs. negative), Real Group Difference
(ingroup better than, the same as, or worse than the outgroup), and Political
Affiliation (Democrat vs. Republican) as fixed factors and Participant as a random factor (for full model output see Supplemental Materials). The degrees of freedom for all contrasts were estimated using Satterthwaite approximations (Kuznetsova et al., 2017).
The first effect of interest was an Evaluated Group × First Sample Valence interaction, χ2(1) = 5.77, p = .017. Simple contrasts demonstrate that participants evaluated the ingroup and outgroup similar following a positive initial outgroup sample, t(437) = .409, d = .04, p = .68, demonstrating a failure to integrate initial positive outgroup experiences. Following a negative initial outgroup sample, however, participants evaluated the ingroup significantly more favorably than the outgroup, t(437) = 4.00, d = .36, p = .0001, demonstrating that participants integrated initial negative outgroup experiences (see Figure 5B). This suggests that, much like participants who sampled first from the ingroup, participants who sampled first from the outgroup were asymmetric in how they integrated initial experiences into evaluations.
The second effect of interest was a significant Evaluated Group × Real
Group Difference interaction, χ2(2) = 44.83, b = -5.49, SE = .83, t = -6.55, p 57 Partisan-Motivated Sampling
< .0001. Simple contrasts demonstrate that when the ingroup was de facto better, participants evaluated the ingroup as significantly higher than the outgroup, t(437)
= 5.60, d = .66, p < .0001. However, when participant’s ingroup was de facto worse, participants evaluated the outgroup as significantly higher than the ingroup, t(437) = -3.60, d = -.42, p = .0004. When both groups were the same, participants interpreted the environment as favorable and evaluated the ingroup as better than the outgroup, t(437) = 3.28, d = .39, p = .001 (see Figure 5A). These results demonstrate that under certain conditions (i.e. when the outgroup is de facto better and participants start by sampling from them), sampling first from the outgroup can lead to more even-handed evaluations.
Figure 5. Evaluations of participants who sampled from the outgroup first as a function of Real Group Differences (Panel A) and First Sample Valence (Panel B). Vertical bars denote standard error of the mean. Discussion 58 Partisan-Motivated Sampling
Sampling from the outgroup first does not guarantee more accurate evaluations. In most situations, participants who sampled from the outgroup first were just as likely to end up with biased evaluations as those who sampled from the ingroup first, but via different mechanisms. For example, we previously found that participants evaluated the ingroup significantly higher than the outgroup following a positive initial experience, suggesting the integration of positive ingroup information. However, those participants did not evaluate the ingroup and outgroup differently following a negative initial experience, suggesting that they downplayed negative ingroup information. In contrast, when initial experiences were about the outgroup, participants evaluated the ingroup significantly higher than the outgroup following a negative outgroup initial experience, suggesting that they instead incorporated negative outgroup information when evaluating the groups. But participants did not evaluate the ingroup and outgroup differently following a positive outgroup initial experience, suggesting that they downplayed positive outgroup information.
We did find evidence that sampling from the outgroup first can lead to more accurate evaluations under certain conditions. Specifically, when the outgroup was de facto better, participants who sampled first from the outgroup were quite accurate in inferring the direction and magnitude of the differences and evaluated 59 Partisan-Motivated Sampling the outgroup significantly higher than the ingroup. This is intriguing given the level of affective polarization between Democrats and Republicans, and suggests that even in a highly polarizing climate, certain experiences can lead to even- handed evaluations. These findings suggest that people who sample from the outgroup first are not merely less inclined to evaluative biases, as they continued to asymmetrically integrate initial experiences in a way that favored the ingroup, and yet, they acknowledged that the outgroup was better when they were indeed better.
Meta-Analysis 3: Sampling Trajectories Change over time Based on
Congeniality of the Environment
On aggregate, we find that people generate skewed samples by sampling first and more often from their ingroup, and asymmetrically integrate these experiences to produce evaluative biases. What remains unknown is whether and how sampling behaviors might dynamically adapt to the congeniality of the environment. In order to interrogate how and when sampling strategies emerge over time, we fit a logistic mixed growth model. This approach is sensitive to changes in the propensity to sample from the ingroup or outgroup over time. Given the power needed to detect complex cross-level interactions, we aggregated the 60 Partisan-Motivated Sampling data across all experiments and treated this analysis as an internal meta-analysis
(Goh et al., 2016), with experiment as a covariate.
We further sought to connect these sampling trajectories to evaluative biases. It is important to not only characterize sampling trajectories over time, but also how these trajectories predict evaluations. To that end, we regressed sampling trajectories on a continuous measure of evaluative bias. The evaluative bias measure represents the difference between participants’ ingroup and outgroup evaluations. It is relevant to note that the evaluative bias variable serves as a predictor of sampling trajectories in the estimated model, but the sampling trajectories themselves precede evaluations and as such the causal relation would be in the opposite direction (i.e. sampling trajectories predict downstream evaluative biases). The reason for modeling a “reverse effect” is that growth models are focused on change as an outcome, and other variables serve as predictors but not outcomes of that change (i.e. there are no well-established models, to our knowledge, for estimating outcomes of individually varying slopes).
Importantly, the chosen analytic strategy still provides an estimate of the association between sampling trajectories and subsequent evaluations.
Methods
Participants 61 Partisan-Motivated Sampling
Participants (N = 2,431) were aggregated across all experiments. The sample had a mean age of 39.82, SD = 12.41 (51% females).
Model Design
In order to address sampling behavior as a function of time (i.e. sample), and the effects of Initial Experiences, Real Group Difference, Political Affiliation and
Experiment on trial-to-trial sampling behavior, we fit a logistic growth mixed model using the glmer function of the lme4 package (Bates et al., 2015) in R.
Importantly, the current model focuses on the first 14 trials (or samples). Our rationale for selecting the first 14 trials was pragmatic, pertaining to convergence of models due to missingness of data past 14 trials (i.e. as sampling trials increases, more participants are missing values from the dataset as they stop sampling and choose to evaluate). Only 25% of participants sampled more than 14 trials, and in order to ensure sufficient power and generalizability of these analyses, we chose the 3rd quartile of total samples as a cut off. In order to ensure these effects did not diverge based on the chosen criterion, and subsequently how we dealt with convergence issues when more trials were included, we also report models with different sample cutoffs in our publicly available scripts (https://osf.io/9627e/? view_only=4f5c8a589e014c92a1fca171bb0e4369). Regarding model estimation, the binary dependent variable of interest (i.e. participant’s choice on a given trial) 62 Partisan-Motivated Sampling was coded as 0 if participants sampled from the outgroup and 1 if participants sampled from the ingroup. The model was estimated using a bottom-up (data- driven) approach (Hox, 2010) with all factors effects coded and the continuous measure of evaluative bias grand mean centered. We began by fitting an intercept only model with participants as a random effect. We then added trial as a variable
(linear and centered) and systematically built the model up (e.g., adding fixed and random effects), using deviance, BIC and AIC statistics to evaluate model fit (for model comparisons see Supplemental Materials). Regarding random effects, the final model had random intercepts for participants and random slopes for trial within subjects.
Results
We first tested whether participants were sensitive to information over time and how their sampling trajectories changed based on the real group differences and the valence of initial experiences. This model revealed a significant Trial ×
Real Group Difference × Valence of Initial Impression interaction, χ2(1) = 14.00, p
= .007. This interaction demonstrates that the valence of initial experiences led to discrepant sampling trajectories based on the congeniality of the environment
(Figure 6). Specifically, when participant’s ingroup was de facto worse, a negative initial experience led to a greater propensity to sample from the ingroup over time, 63 Partisan-Motivated Sampling b = .028, SE = .009, z = 3.11, p = .001. Next, we tested whether participant’s evaluative biases were associated with specific sampling trajectories that emerge from the interaction of real group differences and initial experiences. This model shows that when the ingroup was de facto worse, participants with more biased evaluations were significantly more likely to sample more from the ingroup over time following a negative initial experience, b = .028, SE = .01, z = 2.5, p = .01 (all other p-values > .1).
Taken together, these findings suggest that when the ingroup is de facto worse, participants rely on different sampling strategies based on whether their initial experience was positive or negative and that these strategies are associated with more biased evaluations. 64 Partisan-Motivated Sampling
Figure 6. Predicted estimates representing the probability of sampling from the ingroup over time as a function of Real Group Differences and Valence of Initial Experience. Error bars denote standard error of the mean. Red lines denote a negative first experience and blue lines denote a positive first experience.
Discussion
In the real world, people often must discover underlying population characteristics and can only do so after learning and accumulating information.
Because participants did not have access to any a priori knowledge about the underlying population differences, initial experiences could only truly be registered 65 Partisan-Motivated Sampling as “positive” or “negative” after participants had explored their environments and accumulated information with which to compare those initial experiences.
The current results demonstrate that when the environment was uncongenial
(where the ingroup was de facto worse), sampling trajectories changed based on whether the first experience was positive or negative. Participants who had a positive initial experience were increasingly likely to sample from the outgroup over time, whereas participants who had a negative first experience were increasingly likely to sample from the ingroup over time. One interpretation of these results is that participants have an a priori hypothesis that the ingroup is better, and a negative first experience produces a surprise, thereby prompting them to sample more from the ingroup in order to re-examine their prior beliefs. In this case, sampling more in an attempt better inform estimates would be rational.
However, we find that the tendency to sample more from the ingroup following a negative first sample leads to overly favorable evaluations rather than evaluations that are calibrated to the sampled information. This suggests that this sampling behavior reflects a strategy aimed at increasing favorability, rather than appropriately adjusting prior beliefs.
One possibility is that these sampling strategies increase uncertainty surrounding ingroup estimates, which provides more opportunities to selectively 66 Partisan-Motivated Sampling attend to congenial information. For instance, given the design of the experiments, sampling more from the outgroup following a positive first ingroup experience is likely to produce information that is less favorable to the outgroup, compared to that first positive ingroup experience (i.e. the positive first sample score is higher still than the outgroup mean). Moreover, by sampling more from the outgroup over time, participants can maintain plausible deniability about outgroup superiority by keeping ingroup attributes uncertain and disproportionally relying on a single positive ingroup experience. In contrast, sampling more from the ingroup following a negative first experience is likely to produce information that is more favorable to the ingroup, compared to that first negative experience (i.e. the ingroup mean is higher still than the negative first sample score). In this case, by sampling more from the ingroup over time, participants can downplay the initial experience and keep outgroup attributes uncertain. Overall, the results indicate that people may keep one group’s attributes uncertain and sample in a way that either emphasizes or washes away initial experiences based on its favorability to avoid an unwanted conclusion (e.g., that the outgroup is actually doing better).
Meta-Analysis 4: Variability in Ingroup Experiences Predicts When
Participants Stop Sampling and Moderates Evaluative Biases 67 Partisan-Motivated Sampling
How does the variability of ingroup experiences impact intergroup evaluations? Experiments 1-2 suggest that participants generated more variable ingroup experiences because they predominantly sampled first and more often from the ingroup, and initial scores were manipulated to be more variable.
Participants then asymmetrically integrated those experiences into their evaluations based on congeniality. In addition, Meta-Analysis 3 suggests that participants applied sampling strategies that maintained uncertainty in ingroup attributes, which was associated with more biased evaluations. This strategy was only present for participants who’s ingroup was de facto worse, suggesting that participants exploit variability more when the environment limits their ability to sample congenial information. The goal of Meta-Analysis 4 was to explicitly test (a) whether participants generate more variable ingroup experiences, (b) if more variable ingroup experiences influence when participants decide to stop sampling, and (c) whether more variable ingroup experiences produce more biased evaluations by providing participants with more opportunities to selectively attend to congenial information.
To test whether participants had more variable ingroup or outgroup experiences (a), we compared the average ingroup and outgroup experiences that participants had across experiments using an independent sample t-Test. To test 68 Partisan-Motivated Sampling whether more or less variability in ingroup or outgroup experiences predicted when participants stopped sampling and evaluated (b), we estimated two mixed effects
Cox proportion hazard models (i.e. frailty models) using the coxme package
(Theråneau, 2020) in R studio. The first model regressed the cumulative probability of stopping onto the variability of ingroup experiences, while allowing this to interact with the real group difference manipulation (ingroup being better/same/worse). This model will determine whether uncertainty surrounding ingroup experiences differentially influences when participants stop sampling and evaluate based on the favorability of the environment. The second model was identical to the first, but this time regressed the cumulative probability of stopping onto the variability of outgroup experiences. This model will determine whether the uncertainty surrounding outgroup samples differentially influences when participants stop sampling and evaluate based on the favorability of the environment. To further test whether more or less variability in ingroup or outgroup experiences predicted downstream evaluative biases (c), we also estimated two multiple regression models regressing evaluative biases onto ingroup and outgroup variability, respectively.
Methods
Participants 69 Partisan-Motivated Sampling
Participants (N = 2,431) were aggregated across all experiments.
Participants were excluded from analyses if they did not have at least one sample from the ingroup and the outgroup. After these exclusion criteria, N = 2427 remained. The sample had a mean age of 39.82, SD = 12.41 (51% females).
Model Design
The frailty model included a continuous predictor of ingroup variability, a categorical predictor for the real group difference manipulation (dummy coded) and experiment as a covariate, as well as a random intercept for subjects that assumed a normal distribution. The ingroup variability predictor represented the standard deviation (i.e. uncertainty) of each participant’s ingroup sample at the point at which they decided to stop sampling. All continuous variables were scaled
(i.e. the mean was subtracted from each element and then divided by the standard deviation).
Results
We first sought to confirm that participants did in fact have more variable ingroup relative to outgroup experiences on average. Consistent with the notion that participants’ sampling behavior generated more variable ingroup experiences, we observed a small but significant difference in the variability of ingroup 70 Partisan-Motivated Sampling experiences (M = 9.69; SD = 3.91) compared to outgroup experiences (M = 9.43;
SD = 4) samples, t(1967) = 2.16, d = .06, p = .03.
Next, we tested whether variability in ingroup or outgroup experiences predicted when a participant decided to stop sampling and evaluate, and whether this was moderated by the favorability of the environment. To do so, we fit two separate mixed effects survival analysis using a Cox proportional hazard model: the first examining the influence of ingroup variability on the decision to stop sampling, and the second examining outgroup variability on the decision to stop sampling. This frailty model tests whether people adopt different stopping points based on the variability of experiences as a function of the Real Group Differences.
The model testing the influence of ingroup variability on the decision to stop sampling revealed a significant ingroup variance × Real Group Difference interaction. Specifically, we observed a significant change in the hazard ratio for each unit increase in ingroup variance when the ingroup was de facto worse compared to when the ingroup was de facto better, HR = .64, SE = .14, z = -3.04, p
= .002 (see Figure 7). This demonstrates that participants were significantly more likely to continue sampling (i.e. survive) when there was more uncertainty surrounding ingroup estimates when the ingroup was de facto worse compared to de facto better. We did not observe a significant difference in the hazard ratio for 71 Partisan-Motivated Sampling participants when both groups were the same compared to when the ingroup was de facto better, although the effect was in the same direction, HR = .96, SE = .16, z
= -.18, p = .85. Critically, this effect was specific to ingroup variability, as outgroup variability did not significantly predict when participants decided to stop sampling as a function of the Real Group Differences (all p-values > .13). These results suggest that participants rely on ingroup rather than outgroup variability when deciding whether to continue or stop sampling, and that participants with more variable ingroup experiences were significantly more likely to continue sampling.
Next, we sought to test whether the degree of variability in ingroup and outgroup samples was associated with evaluative biases. Specifically, if participants rely on more variable ingroup experiences to arrive at congenial conclusions, then they will need to be more strategic in their sampling behavior when the environment is ambiguous or unfavorable, as these environments provide less opportunities to gather congenial information. Thus, more variable ingroup experiences should be more strongly related to biased evaluations when the ingroup is de facto worse or the same as the outgroup compared to when the ingroup is de facto better. Indeed, this model revealed that, compared to when the ingroup was de facto better, ingroup variability was more strongly associated with 72 Partisan-Motivated Sampling biased evaluations when both groups were the same ( = .11, 95% CI [.02, .2], SE
2 = .05, t = 2.32, p = .02, R adjusted = .13), and when the ingroup was de facto worse (
2 = .085, 95% CI [.006, .16], SE = .04, t = 2.12, p = .03, R adjusted = .13). Notably, outgroup variability was not a significant predictor of evaluative bias when the ingroup was de facto worse or when both groups were the same compared to when the ingroup was de facto better (all p-values > .38). These results suggest that participants rely more heavily on uncertainty in ingroup estimates to arrive at congenial conclusions in unfavorable or ambiguous environments.
Figure 7. Probability of continuing sampling as a function of Condition and ingroup sample variability. Number at risk refers to the number of participants who have survived up until trial interval T (e.g., T10 = trial 10, T20 = trial 20, etc.). Variability was median split for visualizations purposes strictly, and was generated using the ggsurvplot function in R. Error bars denote standard error of the mean. 73 Partisan-Motivated Sampling
Discussion
An assumption of the current framework was that people exploit greater uncertainty surrounding ingroup experiences by asymmetrically integrating congenial information into evaluations while downplaying uncongenial information. Meta-Analysis 4 directly tested this assumption, and demonstrates that biases in sampling behavior generate more variable ingroup than outgroup experiences. Given that participants are limited in their ability to interpret congenial information in unfavorable environments, they may rely more on ingroup variability to guide sampling strategies in these contexts. Indeed, we find that participants with more variable ingroup experiences were also significantly more likely to continue sampling compared to participants with less variability.
One interpretation is that participants continue sampling simply because there is more uncertainty, and therefore require more evidence before they can confidently evaluate both groups. From this perspective, sampling to reduce uncertainty would be a rational response to uncertainty, and more sampling should lead to more accurate evaluations. However, we instead find that participants with more variable ingroup experiences produced more biased, as opposed to more accurate, evaluations. Moreover, if participants are sampling to reduce uncertainty, then the same should be true for variability surrounding outgroup estimates. 74 Partisan-Motivated Sampling
However, we find no evidence that outgroup variability predicted when participants stopped sampling, and outgroup variability had no influence on evaluations. Taken together, the current results support the notion that participants generate biased samples based on sampling behavior (Konovalova & Le Mens,
2020), and that this moderates evaluative biases. These results also extend research demonstrating how different stopping strategies can generate biased samples and increase overestimations of likelihood due to outliers (Coenen & Gureckis, 2021) by demonstrating that in group-based contexts, people specifically overestimate extreme positive outliers to help them arrive at congenial conclusions.
General Discussion
The current findings unite disparate literatures on information sampling and motivated reasoning to examine partisan bias as it unfolds along the information processing stream. Across different information environments (e.g., sampling political knowledge scores vs. political fact-check ratings) and categories (e.g., gathering information from groups vs. individual group representatives), we demonstrate that motivations permeate different stages of information processing, beginning with (1) a biased selection of information, leading to (2) skewed samples of information, which interact with (3) motivated interpretations to produce 75 Partisan-Motivated Sampling evaluative biases. Specifically, we find that the participants tended to sample from their ingroup first and most often, and that this proclivity is driven by ingroup favoritism (e.g., more interest in the ingroup following a positive ingroup expectation [Experiment 2]) and stronger political convictions (Meta-Analysis 1).
These biases in sampling behavior produced more variable ingroup experiences and predicted more biased evaluations (Meta-Analysis 4). Moreover, we find that participants dynamically shifted their sampling strategies based on first experiences when the ingroup was de facto worse, obscuring real group differences and allowing participants to maintain plausible deniability (e.g., increased uncertainty) about the inferiority of the ingroup (Meta-Analysis 3). This uncertainty also manifests in different stopping points and subsequent over- estimations of ingroup averages when the ingroup was de facto worse, suggesting that sampling experiences and uncertainty moderate biased evaluations (Meta-
Analysis 4). Together, these findings provide key insight into how group membership constrains people’s experiences, giving rise to biased samples that pave the road for people to reach partisan-biased conclusions.
Sampling: A key antecedent to group-based bias
In order to capture different stages of information processing, the current framework mimics the real world in that participants must explore and learn from 76 Partisan-Motivated Sampling noisy information (i.e. distributions around populations means) in order to form conclusions. The amount of information that a person deems sufficient to support an evaluation will be influenced by individual motives and beliefs. As such, different sample sizes may produce estimates that systematically vary around the population mean and provide more opportunities for motivated interpretations.
Thus, in order to understand how motives influence sampling, it is important to not only examine aggregate sampling behavior (Experiments 1-2, Meta-Analyses 1-2), but also how sampling strategies may change over time depending on features of the environment (Meta-Analysis 3), and what features of the sampled information predict when participants decide to stop sampling (Meta-Analysis 4). By manipulating environmental features (i.e. Real Group Differences & Valence of
Initial Experiences), we test specific mechanisms that describe how aggregated sampling, changes in sampling over time, and the decision to stop sampling give rise to biased evaluations.
Examining sampling behavior on aggregate highlights two important behaviors that each contribute to biased evaluations. First, across experiments we find that the majority of participants started sampling from their own group.
Because the first sample was either overly positive or negative, most participants had more varied experiences about their ingroup relative to the outgroup. However, 77 Partisan-Motivated Sampling more variability in ingroup samples alone does not guarantee biased evaluations. A completely objective agent should symmetrically over-estimate the ingroup following positive initial experiences and under-estimate the ingroup following negative initial experiences. A motivated agent may instead exploit variability and integrate experiences into evaluations asymmetrically by selectively over- estimating the ingroup following congenial information, but not under-estimating the ingroup following uncongenial information. Consistent with the motivated perspective, and that biased evaluations represent the interaction of sampling and interpretive biases, our results demonstrate that when individuals sample from the ingroup first, they over-estimate positive initial experiences and under-estimate negative initial experiences about their ingroup (Experiments 1a and 1b). When individuals instead sample from the outgroup first, the asymmetry is reversed and individuals under-estimate positive and over-estimate negative initial experiences about the outgroup (Meta-Analysis 2).
Second, participants sampled overall more information from their own group, generating larger and more uniform distributions of ingroup relative to outgroup samples. Increased ingroup sample size produces a greater pool of experiences from which people can selectively weight and interpret (Konovalova
& Le Mens, 2020), which corresponds with our results which suggest that 78 Partisan-Motivated Sampling participants with more varied ingroup experiences produced more biased evaluations (Meta-Analysis 4). However, participants sampled less from the outgroup overall, minimizing the opportunity to update outgroup evaluations
(Denrell, 2005), which likewise corresponds with outgroup variability having no association with biased evaluations (Meta-Analysis 4). Overall, individuals seem to create and cherry pick from a more varied array of ingroup experiences to maintain favorable ingroup impressions, whereas unfavorable outgroup impressions remain resistant to change due (in part) to fewer and less varied outgroup experiences.
Examining dynamic changes in sampling behavior over time provides further insights not accessible by examining static, aggregate sampling behavior.
For instance, when the ingroup was de facto worse, sampling trajectories deviated over time based on initial experience, such that a negative experience led to a greater likelihood to sample from the ingroup over time, whereas a positive experience led to a greater likelihood to sample from the outgroup over time.
These strategies may be the most effective at disguising ingroup and outgroup attributes. For example, when the outgroup is de facto better and most new information about the ingroup is uncongenial, one way to maintain positive ingroup evaluations is to rely on positive initial ingroup experiences and retain uncertainty about the ingroup by sampling more from the outgroup over time. 79 Partisan-Motivated Sampling
Critically, we demonstrate that participants who stuck to this strategy were more likely to report more biased evaluations. Notably, biased evaluations were not associated with sampling trajectories when both groups were the same or the ingroup was de facto better, suggesting that participants were especially flexible with their sampling strategies over time when the ingroup was de facto worse. In other words, participants were most flexible in how they gathered information when an agnostic search would fail to support a desired conclusion.
The current framework highlights how biased sampling behavior can generate more varied ingroup experiences, which participants can then asymmetrically integrate into their evaluations. However, another important feature of sampling behavior that can bias samples is when information search is terminated (analogous to “optimal stopping” as a form of naïve statistical p- hacking, see Simmons et al., 2011; Armitage et al., 1969). Arbitrary stopping rules can generate biased samples by capitalizing on variability surroundings the population mean. Thus, by deciding to continue sampling when the data does not fit an a priori hypothesis (e.g., the outgroup is better), but deciding to stop sampling when data does happen to fit an a priori hypothesis (e.g., the ingroup is better), participants can capitalize on variability. This stopping strategy is most likely to occur in an unfavorable environment, as participants have to rely more 80 Partisan-Motivated Sampling heavily on statistical anomalies, rather than overall trends, to reach their desired conclusion. Indeed, our results show that participants employed different stopping rules based on the uncertainty of ingroup, but not outgroup samples, when the ingroup was de facto worse. Moreover, more uncertainty in ingroup, but not outgroup samples, predicted more biased evaluations when both groups were the same and when the ingroup was de facto worse, relative to when the ingroup was de facto better. Taken together, these results demonstrate that motivated sampling strategies, which produce more variability in ingroup experiences, can explain ingroup-serving biases in evaluations.
From minimal groups to political groups: Ingroup favoritism versus outgroup derogation
The present findings dovetail with recent work on sampling in minimal group contexts (Bergh & Lindskog, 2019). Consistent across minimal and political group contexts, participants sampled first and foremost from their own group.
These sampling biases led to a similar overall pattern and magnitude of evaluative biases, which suggests that greater variability of ingroup experiences facilitated downstream evaluative biases. However, whereas research in minimal groups found that biased evaluations represented generally inflated ingroup evaluations that were driven by positive initial ingroup experiences (Bergh & Lindskog, 2019), 81 Partisan-Motivated Sampling here we find that ingroup evaluations were generally accurate (i.e. similar to the underlying distribution means) but outgroup evaluations were consistently underestimated (i.e. more negative than the underlying distribution means). One possibility for this shift in evaluative bias is that highly polarized group contexts, such as political groups, may create a shift from ingroup favoritism to additionally motivate outgroup derogation. These findings align with prior work suggesting outgroup derogation is likely to emerge in intergroup contexts characterized by a perception of threat or zero-sum contest over success and failure (Brewer, 1999;
Cikara et al., 2017).
Partisan similarities and differences in information processing
The current findings also add nuance to ongoing debates about differences in politically motivated reasoning across the political divide (e.g., Ditto et al., 2019;
Jost et al., 2018). Overall, we do not find any significant differences in how
Democrats and Republicans evaluate ingroup and outgroup candidates or group members. Both groups seem similarly susceptible to asymmetries in their integration of initial experiences based on its congeniality. However, we find some notable differences in sampling behavior, which raises the possibility that
Democrats and Republicans may arrive at their biased evaluations in different ways. For instance, Democrats reliably sampled more information than 82 Partisan-Motivated Sampling
Republicans. A larger sample of information should theoretically produce evaluations that are closer to the mean (i.e. as samples increase, they converge on underlying population parameters), but we did not detect any such differences between Democrat and Republican evaluations. Therefore, one could argue that
Democrats were less biased in their sampling behavior, sampling more information from both categories, but more biased in their interpretations, as more data did not reduce evaluative biases. This suggests that either (a) sampling differences exert a smaller effect compared to biased interpretations on evaluations, or (b) that there are different strategies to arrive at the same conclusion. One fruitful avenue for future work will be to more closely examine differences in sampling strategies and belief updating to disentangle the paths by which Democrats and Republicans may arrive at biased conclusions.
Informing Prejudice Interventions
In the political arena, our results suggest that interventions targeting the early stages of information processing – namely, more even-handed and representative experiences between individuals and across groups – may be one way to reduce downstream polarization and conflict. In line with prior work demonstrating that interparty contact can attenuate outgroup hostility (Huddy &
Yair, 2019; Wojcieszak & Warner, 2020), our results suggest that more varied 83 Partisan-Motivated Sampling initial and overall outgroup experiences may also attenuate outgroup hostility by limiting people’s ability to selectively attend to congenial ingroup information while increasing the pool of outgroup experiences. Notably, this strategy does not guarantee accurate outgroup evaluations as our results suggest that people may instead selectively attend to negative outgroup information and arrive at similarly biased evaluations (Meta-Analysis 2). Nonetheless, under certain conditions (e.g., when the outgroup is de facto better) more representative outgroup experiences represent one path to accurate evaluations, which has the potential to reduce partisan prejudice.
Conclusions
We live in one of the most politically polarized times in U.S. history. As a consequence, dialogue between individuals across the political divide is increasingly fraught and antipathy between Democrats and Republicans is at an all- time high, posing an imminent threat to effective governance (McCoy et al., 2018).
At the same time, people have access to limitless amounts of information but must form conclusions and update beliefs based on limited subsets of all available information. The current work provides a comprehensive framework to examine how evaluative biases emerge starting with the very first experiences people seek out, demonstrating that partisan-motivations play an integral role in shaping the 84 Partisan-Motivated Sampling experiences that people have, and in turn how they interpret those experiences to support desired conclusions.
85 Partisan-Motivated Sampling
References
Abramowitz, A. I., & Saunders, K. L. (2008). Is polarization a myth? Journal of
Politics, 70(2), 542–555. https://doi.org/10.1017/S0022381608080493
Armitage, P., McPherson, C.K., Rowe, B.C. (1998). Repeated significance tests
on accumulating data of repairable systems. Journal of the Roway Statistical
Society. Series A (General), Vol. 132, No. 2 (1969), pp. 235-244.
https://doi.org/10.1080/03610929808832152
Albarracín, D., Johnson, B. T., & Zanna, M. P. (2005). Handbook About Attitudes.
Ashmore, R. D., & Del Boca, F. K. (1981). Conceptual approaches to stereotypes
and stereotyping. Ashmore, R. D. Del Boca, F. K., Ashmore, R. D., Del Boca,
F. K. (1981).
Baron, J., & Jost, J. T. (2019). False Equivalence: Are Liberals and Conservatives
in the United States Equally Biased? Perspectives on Psychological Science,
14(2), 292–303. https://doi.org/10.1177/1745691618788876
Bates, D., Mächler, M., Bolker, B. M., & Walker, S. C. (2015). Fitting linear
mixed-effects models using lme4. Journal of Statistical Software, 67(1).
https://doi.org/10.18637/jss.v067.i01
Baumeister, R.F., Finkel, E. J. (2010). Advanced Social Psychology. Oxford
University Press, 1, 1–795. 86 Partisan-Motivated Sampling
Baumeister, R. F., Bratslavsky, E., & Vohs, K. D. (2001). Bad Is Stronger Than
Good. Review of General Psychology. 5(4), 323–370.
https://doi.org/10.1037//1089-2680.5.4.323
Bergh, R., Akrami, N., Sidanius, J., & Sibley, C. G. (2016). Is group membership
necessary for understanding generalized prejudice? A re-evaluation of why
prejudices are interrelated. Journal of Personality and Social Psychology,
111(3), 367–395. https://doi.org/10.1037/pspi0000064
Bergh, R., & Lindskog, M. (2019). The group-motivated sampler. Journal of
Experimental Psychology: General, 148(5), 845–862. https://doi.org/10.1037/
xge0000601
Brewer, M. B. (1979). In-group bias in the minimal intergroup situation. Vol.86
No.2, Psychological Bulletin. 86(2), 307–324.
http://prism.talis.com/greenwich-ac/items/443509
Brewer, M. B. (1999). The psychology of prejudice: Ingroup love or outgroup
hate? Journal of Social Issues, 55(3), 429–444. https://doi.org/10.1111/0022-
4537.00126
Cikara, M., Van Bavel, J. J., Ingbretsen, Z. A., & Lau, T. (2017). Decoding “Us”
and “Them”: Neural representations of generalized group concepts. Journal of
Experimental Psychology: General, 146(5), 621–631. https://doi.org/10.1037/ 87 Partisan-Motivated Sampling
xge0000287
Coenen, A., & Gureckis, T. (2021). The distorting effects of deciding to stop sampling
information. PsyArXiv. doi:10.31234/osf.io/tbrea
Cohen, G. L. (2003). Party Over Policy: The Dominating Impact of Group
Influence on Political Beliefs. Journal of Personality and Social Psychology,
85(5), 808–822. https://doi.org/10.1037/0022-3514.85.5.808
Crocker, J., & Luhtanen, R. (1990). Collective Self-Esteem and Ingroup Bias.
Journal of Personality and Social Psychology, 58(1), 60–67.
https://doi.org/10.1037/0022-3514.58.1.60
Crocker, J., & Major, B. (1989). Social Stigma and Self-Esteem: The Self-
Protective Properties of Stigma. Psychological Review, 96(4), 608–630.
https://doi.org/10.1037/0033-295X.96.4.608
Denrell, J. (2005). Why most people disapprove of Me: Experience sampling in
impression formation. Psychological Review, 112(4), 951–978. https://doi.org/
10.1037/0033-295X.112.4.951
Denrell, J., & Le Mens, G. (2017). Information Sampling, Belief Synchronization,
and Collective Illusions. Management Science, 63(2), 528–547. https://doi.org/
10.1287/mnsc.2015.2354 88 Partisan-Motivated Sampling
Ditto, P. H., Liu, B. S., Clark, C. J., Wojcik, S. P., Chen, E. E., Grady, R. H.,
Celniker, J. B., & Zinger, J. F. (2019). At Least Bias Is Bipartisan: A Meta-
Analytic Comparison of Partisan Bias in Liberals and Conservatives.
Perspectives on Psychological Science, 14(2), 273–291.
https://doi.org/10.1177/1745691617746796
Ditto, P. H., & Lopez, D. F. (1992). Motivated Skepticism: Use of Differential
Decision Criteria for Preferred and Nonpreferred Conclusions. Journal of
Personality and Social Psychology, 63(4), 568–584.
https://doi.org/10.1037/0022-3514.63.4.568
Ditto, P. H., Munro, G. D., Apanovitch, A. M., Scepansky, J. A., & Lockhart, L. K.
(2003). Spontaneous skepticism: The interplay of motivation and expectation
in responses to favorable and unfavorable medical diagnoses. Personality and
Social Psychology Bulletin, 29(9), 1120–1132.
https://doi.org/10.1177/0146167203254536
Ditto, P. H., Munro, G. D., Lockhart, L. K., Scepansky, J. A., & Apanovitch, A. M.
(1998). Motivated Sensitivity to Preference-Inconsistent Information. Journal
of Personality and Social Psychology, 75(1), 53–69.
https://doi.org/10.1037/0022-3514.75.1.53
Druckman, J. N., Peterson, E., & Slothuus, R. (2013). How elite partisan 89 Partisan-Motivated Sampling
polarization affects public opinion formation. American Political Science
Review, 107(1), 57–79. https://doi.org/10.1017/S0003055412000500
Festinger, L. (1962). Cognitive Dissonance. Scientific American, 207(4), 93-106.
Fiedler, K. (2000). Beware of Samples! A Cognitive-Ecological Sampling
approach to judgment biases. Psychological Review, 107(4), 659–676.
https://doi.org/10.1037//0033-295X.107A659
Fiedler, K., Walther, E., & Nickel, S. (1999). The auto-verification of social
hypotheses: Stereotyping and the power of sample size. Journal of Personality
and Social Psychology, 77(1), 5–18. https://doi.org/10.1037/0022-3514.77.1.5
Fischer, P., & Greitemeyer, T. (2010). A new look at selective-exposure effects:
An integrative model. Current Directions in Psychological Science, 19(6),
384–389. https://doi.org/10.1177/0963721410391246
Fischer, P., Schulz-Hardt, S., & Frey, D. (2008). Selective Exposure and
Information Quantity: How Different Information Quantities Moderate
Decision Makers’ Preference for Consistent and Inconsistent Information.
Journal of Personality and Social Psychology, 94(2), 231–244. https://doi.org/
10.1037/0022-3514.94.2.94.2.231
Flynn, D. J., Nyhan, B., & Reifler, J. (2017). The Nature and Origins of
Misperceptions: Understanding False and Unsupported Beliefs About Politics. 90 Partisan-Motivated Sampling
Political Psychology, 38, 127–150. https://doi.org/10.1111/pops.12394
Frimer, J. A., Tell, C. E., & Motyl, M. (2017). Sacralizing Liberals and Fair-
Minded Conservatives: Ideological Symmetry in the Moral Motives in the
Culture War. Analyses of Social Issues and Public Policy, 17(1), 33–59.
https://doi.org/10.1111/asap.12127
Galesic, M., Olsson, H., & Rieskamp, J. (2012). Social Sampling Explains
Apparent Biases in Judgments of Social Environments. Psychological Science,
23(12), 1515–1523. https://doi.org/10.1177/0956797612445313
Gampa, A., Wojcik, S. P., Motyl, M., Nosek, B. A., & Ditto, P. H. (2019).
(Ideo)Logical Reasoning: Ideology Impairs Sound Reasoning. Social
Psychological and Personality Science.
https://doi.org/10.1177/1948550619829059
Goh, J. X., Hall, J. A., & Rosenthal, R. (2016). Mini Meta-Analysis of Your Own
Studies: Some Arguments on Why and a Primer on How. Social and
Personality Psychology Compass, 10(10), 535–549.
https://doi.org/10.1111/spc3.12267
Green, P., & Macleod, C. J. (2016). SIMR: An R package for power analysis of
generalized linear mixed models by simulation. Methods in Ecology and
Evolution, 7(4), 493–498. https://doi.org/10.1111/2041-210X.12504 91 Partisan-Motivated Sampling
Greenberg, J., & Pyszczynski, T. (1985). Compensatory Self-Inflation. A Response
to the Threat to Self-Regard of Public Failure. Journal of Personality and
Social Psychology, 49(1), 273–280. https://doi.org/10.1037/0022-
3514.49.1.273
Harris, C., Fiedler, K., Marien, H., & Custers, R. (2020). Biased Preferences
Through Exploitation: How Initial Biases Are Consolidated in Reward-Rich
Environments. Journal of Experimental Psychology: General.
https://doi.org/10.1037/xge0000754
Hart, W., Albarracín, D., Eagly, A. H., Brechan, I., Lindberg, M. J., & Merrill, L.
(2009). Feeling Validated Versus Being Correct: A Meta-Analysis of Selective
Exposure to Information. Psychological Bulletin, 135(4), 555–588.
https://doi.org/10.1037/a0015701
Hastorf, A. H., & Cantril, H. (1954). They saw a game; a case study. Journal of
Abnormal and Social Psychology, 49(1), 129–134.
https://doi.org/10.1037/h0057880
Herzog, S. M., & Hertwig, R. (2009). The wisdom of many in one mind:
Improving individual judgments with dialectical bootstrapping. Psychological
Science, 20(2), 231–237. https://doi.org/10.1111/j.1467-9280.2009.02271.x
Hogg, M. A. (2001). A social identity theory of leadership. Personality and Social 92 Partisan-Motivated Sampling
Psychology Review, 5(3), 184–200.
https://doi.org/10.1207/S15327957PSPR0503_1
Hox, J. J. (2010) Quantitative methodology series. Multilevel analysis: Techniques
and applications (2nd ed.). Routledge/Taylor & Francis Group.
Huddy, L., & Yair, O. (2019). Reducing Affective Partisan Polarization: Warm
Group Relations or Policy Compromise? Working Paper, 1689–1699.
Hughes, B. L., Ambady, N., & Zaki, J. (2017). Trusting outgroup, but not ingroup
members, requires control: Neural and behavioral evidence. Social Cognitive
and Affective Neuroscience, 12(3), 372–381.
https://doi.org/10.1093/scan/nsw139
Ito, T. A., Larsen, J. T., Smith, N. K., & Cacioppo, J. T. (1998). Negative
information weighs more heavily on the brain: the negativity bias in evaluative
categorizations. Journal of Personality and Social Psychology, 75(4), 887–
900. https://doi.org/10.1037/0022-3514.75.4.887
Iyengar, S., & Krupenkin, M. (2018). The Strengthening of Partisan Affect.
Political Psychology, 39, 201–218. https://doi.org/10.1111/pops.12487
Iyengar, S., Lelkes, Y., Levendusky, M., Malhotra, N., & Westwood, S. J. (2019).
The Origins and Consequences of Affective Polarization in the United States.
Annual Review of Political Science, 22(1), 129–146. 93 Partisan-Motivated Sampling
https://doi.org/10.1146/annurev-polisci-051117-073034
Iyengar, S., Sood, G., & Lelkes, Y. (2012). Affect, not ideology: A social identity
perspective on polarization. Public Opinion Quarterly, 76(3), 405–431. https://
doi.org/10.1093/poq/nfs038
Jost, J. T., Glaser, J., Sulloway, F. J., & Kruglanski, A. W. (2018). Political
conservatism as motivated social cognition. The Motivated Mind: The Selected
Works of Arie Kruglanski, 129(3), 129–204.
https://doi.org/10.4324/9781315175867
Juslin, P., Winman, A., & Hansson, P. (2007). The Naïve Intuitive Statistician: A
Naïve Sampling Model of Intuitive Confidence Intervals. Psychological
Review, 114(3), 678–703. https://doi.org/10.1037/0033-295X.114.3.678
Kahan, D. M. (2013). Ideology, motivated reasoning, and cognitive reflection.
Judgment and Decision Making, 8(4), 407–424.
Kahan, D. M. (2016). The Politically Motivated Reasoning Paradigm, Part 1: What
Politically Motivated Reasoning Is and How to Measure It. Emerging Trends
in the Social and Behavioral Sciences, 1–16.
https://doi.org/10.1002/9781118900772.etrds0417
Kahan, D. M., Jenkins-Smith, H., & Braman, D. (2011). Cultural cognition of
scientific consensus. Journal of Risk Research, 14(2), 147–174. https://doi.org/ 94 Partisan-Motivated Sampling
10.1080/13669877.2010.511246
Kahan, D. M., Peters, E., Dawson, E., & Slovic, P. (2013). Motivated Numeracy
and Enlightened Self-Government. Cambridge University Press.
https://doi.org/10.2139/ssrn.2319992
Keys, D. J., & Schwartz, B. (2007). “Leaky’’ Rationality. Perspectives on
Psychological Science, 2(2), 162–180.
Klayman, J., & Ha, Y. (1987). Confirmation, disconfirmation, and information in
hypothesis testing. Psychological Review, 94(2), 211–228.
https://doi.org/10.1037//0033-295x.94.2.211
Konovalova, E., & Le Mens, G. (2020). An information sampling explanation for
the in-group heterogeneity effect. Psychological Review, 127.
https://doi.org/10.1037/rev0000160
Korner-Nievergelt, F., Roth, T., von Felten, S., Guélat, J., Almasi, B., & Korner-
Nievergelt, P. (2015). Bayesian Data Analysis in Ecology Using Linear
Models with R, BUGS, and Stan. Academic Press.
https://doi.org/10.1016/C2013-0-23227-X
Kraft, P. W., Lodge, M., & Taber, C. S. (2015). Why People “Don’t Trust the
Evidence”: Motivated Reasoning and Scientific Beliefs. Annals of the
American Academy of Political and Social Science, 658(1), 121–133. 95 Partisan-Motivated Sampling
https://doi.org/10.1177/0002716214554758
Kunda, Z. (1987). Motivated Inference: Self-Serving Generation and Evaluation of
Causal Theories. Journal of Personality and Social Psychology, 53(4), 636–
647. https://doi.org/10.1037/0022-3514.53.4.636
Kunda, Z., Fong, G. T., Sanitioso, R., & Reber, E. (1993). Directional questions
direct self-conceptions. In Journal of Experimental Social Psychology (Vol.
29, Issue 1, pp. 63–86). https://doi.org/10.1006/jesp.1993.1004
Kutzner, F., & Vogel, T. (2011). Contingency inferences driven by base rates:
Valid by sampling. Judgment and Decision Making. 6(3), 211–221.
http://journal.sjdm.org/11/9727/jdm9727.html
Kuznetsova, A., Brockhoff, P. B., & Christensen, R.H.B. (2017), LmerTest
Package: Tests in Linear Mixed Effects Models. Journal of Statistical
Software, 82(13). https://doi.org/10/18637/jssv082.i13
Le Mens, G., & Denrell, J. (2011). Rational Learning and Information Sampling:
On the “ Naivety” Assumption in Sampling Explanations of Judgment Biases.
Psychological Review, 118(2), 379–392. https://doi.org/10.1037/a0023010
Le Mens, G., Denrell, J., Kovács, B., & Karaman, H. (2018). Information
Sampling, Judgment, and the Environment: Application to the Effect of
Popularity on Evaluations. Topics in Cognitive Science, 1–16. 96 Partisan-Motivated Sampling
https://doi.org/10.1111/tops.12387
Levendusky., M. (2010). The Partisan Sort: How Liberals Became Democrats and
Conservatives Became Republicans. Public Opinion Quarterly, 74(4), 814–
816. https://doi.org/10.1093/poq/nfq050
Lindskog, M., & Winman, A. (2014). Are all data created equal? - Exploring some
boundary conditions for a lazy intuitive statistician. PLoS ONE, 9(5).
https://doi.org/10.1371/journal.pone.0097686
Lindskog, M., Winman, A., & Juslin, P. (2013a). Are there rapid feedback effects
on approximate number system acuity? Frontiers in Human Neuroscience,
7(MAY), 1–8. https://doi.org/10.3389/fnhum.2013.00270
Lindskog, M., Winman, A., & Juslin, P. (2013b). Naïve point estimation. Journal
of Experimental Psychology: Learning Memory and Cognition, 39(3), 782–
800. https://doi.org/10.1037/a0029670
Lindskog, M., Winman, A., Juslin, P., & Poom, L. (2013). Measuring acuity of the
approximate number system reliably and validly: The evaluation of an
adaptive test procedure. Frontiers in Psychology, 4(AUG), 1–14.
https://doi.org/10.3389/fpsyg.2013.00510
Lord, C. G., Ross, L., & Lepper, M. R. (1979). Biased assimilation and attitude
polarization: The effects of prior theories on subsequently considered 97 Partisan-Motivated Sampling
evidence. Journal of Personality and Social Psychology, 37(11), 2098–2109.
https://doi.org/10.1037/0022-3514.37.11.2098
Luhtanen, R., & Crocker, J. (1992). A Collective Self-Esteem Scale: Self-
Evaluation of One’s Social Identity. Personality and Social Psychology
Bulletin (Vol. 18, Issue 3, pp. 302–318).
https://doi.org/10.1177/0146167292183006
MacKuen, M., Wolak, J., Keele, L., & Marcus, G. E. (2010). Civic engagements:
Resolute partisanship or reflective deliberation. American Journal of Political
Science, 54(2), 440–458. https://doi.org/10.1111/j.1540-5907.2010.00440.x
Mason, L. (2018). Uncivil Agreement: How Politics Became Our Identity. In
Public Integrity (Vol. 21, Issue 2, pp. 214–219).
https://doi.org/10.1080/10999922.2018.1511673
McCoy, J., Rahman, T., & Somer, M. (2018). Polarization and the Global Crisis of
Democracy: Common Patterns, Dynamics, and Pernicious Consequences for
Democratic Polities. American Behavioral Scientist, 62(1), 16–42.
https://doi.org/10.1177/0002764218759576
Meiser, T. (2005). Contingency learning and biased group impressions.
Information Sampling and Adaptive Cognition, 183–209.
https://doi.org/10.1017/CBO9780511614576.009 98 Partisan-Motivated Sampling
Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal
reports on mental processes. Psychological Review, 84(3), 231–259.
https://doi.org/10.1037/0033-295X.84.3.231
Olson, J. M., & Zanna, M. P. (1979). A new look at selective exposure. Journal of
Experimental Social Psychology, 15(1), 1–15. https://doi.org/10.1016/0022-
1031(79)90014-3
Pew Research Center. (2017). The Partisan Divide on Political Values Grows
Even Wider.
Pew Research Center. (2020). By a Narrow Margin, Americans Say Senate Trial
Should Result in Trump’s Removal.
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive
psychology: Undisclosed flexibility in data collection and analysis allows
presenting anything as significant. Psychological Science, 22(11), 1359–1366.
https://doi.org/10.1177/0956797611417632
Sharot, T., & Garrett, N. (2016). Forming Beliefs: Why Valence Matters. Trends
in Cognitive Sciences, 20(1), 25–33. https://doi.org/10.1016/j.tics.2015.11.002
Taber, C. S., Cann, D., & Kucsova, S. (2009). The motivated processing of
political arguments. Political Behavior, 31(2), 137–155.
https://doi.org/10.1007/s11109-008-9075-8 99 Partisan-Motivated Sampling
Taber, C. S., & Lodge, M. (2012). Motivated Skepticism in the. Evaluation,
September 2014, 1–50. https://doi.org/10.1080/08913811.2012.711019
Tajfel, H., & Billig, M. (1973). Social categorization and similarity in intergroup
behaviour. European Journal of Social Psychology, 3(1), 27–52.
Tamir, D. I., & Hughes, B. L. (2018). Social Rewards: From Basic Social Building
Blocks to Complex Social Behavior. Perspectives on Psychological Science,
13(6), 700–717. https://doi.org/10.1177/1745691618776263
Tappin, B. M., Pennycook, G., & Rand, D. G. (2020). Thinking clearly about
causal inferences of politically motivated reasoning: why paradigmatic study
designs often undermine causal inference. Current Opinion in Behavioral
Sciences, 34, 81–87. https://doi.org/10.1016/j.cobeha.2020.01.003
Tappin, B. M., van der Leer, L., & McKay, R. T. (2017). The heart trumps the
head: Desirability bias in political belief revision. Journal of Experimental
Psychology: General, 146(8), 1143–1149. https://doi.org/10.1037/xge0000298
Therneau, T. M. (2020). coxme: Mixed Effects Cox Models. R package version
2.2-16. Https://CRAN.R-Project.Org/Package=coxme, 1–14. http://cran.r-
project.org/package=coxme%5Cnftp://ftp5.us.postgresql.org/pub/CRAN/web/
packages/coxme/vignettes/coxme.pdf
Tversky, A., & Kahneman, D. (1971). Belief in the law of small numbers. 100 Partisan-Motivated Sampling
Psychological Bulletin, 76(2), 105–110. https://doi.org/10.1037/h0031322
Weeks, B. E. (2015). Emotions, Partisanship, and Misperceptions: How Anger and
Anxiety Moderate the Effect of Partisan Bias on Susceptibility to Political
Misinformation. Journal of Communication, 65(4), 699–719.
https://doi.org/10.1111/jcom.12164
West, E. A., & Iyengar, S. (2020). Partisanship as a Social Identity: Implications
for Polarization. Political Behavior (Issue 0123456789). Springer US.
https://doi.org/10.1007/s11109-020-09637-y
Westerwick, A., Johnson, B. K., & Knobloch-Westerwick, S. (2017). Confirmation
biases in selective exposure to political online information: Source bias vs.
content bias. Communication Monographs, 84(3), 343–364.
https://doi.org/10.1080/03637751.2016.1272761
Willer, D., Turner, J. C., Hogg, M. A., Oakes, P. J., Reicher, S. D., & Wetherell,
M. S. (1989). Rediscovering the Social Group: A Self-Categorization Theory.
Contemporary Sociology (Vol. 18, Issue 4, p. 645).
https://doi.org/10.2307/2073157
Wojcieszak, M., & Warner, B. R. (2020). Can Interparty Contact Reduce Affective
Polarization? A Systematic Test of Different Forms of Intergroup Contact.
Political Communication, 00(00), 1–23. 101 Partisan-Motivated Sampling
https://doi.org/10.1080/10584609.2020.1760406
Wyer, R. S., & Frey, D. (1983). The effects of feedback about self and others on
the recall and judgments of feedback-relevant information. Journal of
Experimental Social Psychology, 19(6), 540–559.
https://doi.org/10.1016/0022-1031(83)90015-X