<<

Replication of Asch (1946)

Forming Impressions of Personality:

A Replication and Review of Asch's (1946) Evidence for a Primacy-of-Warmth Effect in Impression Formation

Sanne Nauts1, Inge Huijsmans1, Oliver Langner2, Roos Vonk1 & Daniël H. J. Wigboldus1

1Behavioural Science Institute, Radboud University Nijmegen, The Netherlands

2Friedrich Schiller University Jena, Germany

Sanne Nauts Montessorilaan 3, 6525 HR Nijmegen, The Netherlands Phone: + 31 24 3615682 Fax: +31 24 3612677 E-mail: [email protected]

1

Replication of Asch (1946)

Social psychological laboratories have undergone considerable change since the publication of

Asch's "Forming Impressions of Personality" (1946), leading to the inevitable demise of punch cards and slide carousels in favor of more technologically advanced experimental equipment. Still, the basic methodology underlying present-day person perception research is strongly grounded in (and often remarkably similar to) the methodology first introduced by Asch, now 67 years ago (Gilbert, 1998).

With over 2750 references, Asch's seminal research has been heralded for its paradigm-shifting methodology, in which impression formation was studied in a controlled laboratory setting, yielding high internal validity and experimental precision (Fiske, Cuddy, & Glick, 2007; Gilbert, 1998). But

Asch's legacy is by no means limited to the methodological realm: The ten experiments reported in

"Forming Impressions of Personality" have laid much of the groundwork for influential theories about person perception (e.g., attribution theory; Jones & Davis, 1965; the continuum model of impression formation; Fiske, Neuberg, Beattie, & Milberg, 1987).

Written long before the dawn of bite-size science (Bertamini & Munafo, 2012) and the advice to "role-play grandma" to create a clear storyline (Bem, 1987, p. 27), "Forming Impressions of

Personality" is as interesting as it is multi-faceted. Asch (1946) offers a multitude of meandering messages, frequently wedged between eloquent interludes, engaging observations, and elegant hand- crafted figures. Although there may not be one single, unitary message to be taken from his work, the one message that seems to have strongly resonated with present-day researchers concerns the primacy- of-warmth-effect. The primacy-of-warmth effect (e.g., Fiske, Cuddy, & Glick, 2007; Wojciszke,

2005)1 entails that warmth has a primary role in impression formation, in the sense that warmth-related judgments have a stronger influence on impressions (Wojciszke, Bazinska, & Jaworski, 1998), and that warmth- related inferences are made more quickly than competence-related inferences (Ybarra,

Chan, & Park, 2001).

The goal of the present research is to critically examine the evidence that Asch's (1946) research provides for such a primacy-of-warmth effect. Moreover, we will conduct a direct replication of those studies of Asch's paper that are relevant to this particular effect. Replication attempts of

2

Replication of Asch (1946)

Asch's work abound (e.g., Ahlering & Parker, 1989; Anderson & Barrios, 1961; Babad, Kaplowitz &

Darley, 1990; Grace & Greenshields, 1960; Hendrick & Constantini, 1970; Kelley, 1950; Luchins,

1948; Luchins & Luchins, 1986; McCarthy & Skowronski, 2011; McKelvie, 1990; Mensh & Wishner,

1947; Pettijohn, Pettijohn & McDermott, 2009; Semin, 1989; Singh, Onglacto, Sriram & Tay, 1997;

Veness & Brierley, 1963; Wishner, 1960), but most are conceptual rather than direct replications, many are incomplete, few relate to the primacy-of-warmth-effect, and finally, their results do not necessarily concur with Asch's original findings. Thus, although "forming impressions of personality" has been regarded as the point of inception for the primacy-of-warmth-effect (e.g., Abele &

Bruckmüller, 2011; Abele & Wojciszke, 2007; Cuddy, Fiske & Glick, 2008; Fiske, Cuddy & Glick,

2007; Judd, James-Hawkins, Yzerbyt, & Kashima, 2005; Kervyn, Yzerbyt, & Judd, 2010; Richetin,

Durante, Mari, Perugini & Volpato, 2012; Vonk, 1994), it is unclear to what extent Asch’s original studies represent replicable evidence for such an effect. As one of the founding articles of the person perception field, and perhaps even of more generally, we believe it is important to establish its truth value and provide a replication attempt that provides researchers with sufficient data to be able to interpret Asch's experiments, something we believe is not possible based on the data reported in Asch's research due to incomplete reporting of the data, underpowered studies, and a lack of systematic analysis of the data. Moreover, by conducting additional analyses based on more recent insights in the person perception literature, we hope to shed more light on the processes that underlie

Asch's classic effects. Before going on to discuss the present replication attempt, we will first give a short overview of Asch's main findings2.

Overview of Asch (1946)

In "Forming Impressions of Personality" ten studies were reported (total N = 8343) in which Asch

(1946) presented participants with lists of traits containing small manipulations. For example, in

Study I, participants were exposed either to a list of traits containing the word "warm" or a list containing the word "cold", while all other traits were identical for both groups. After this, they had to write down what their impression of the target person was (open-ended measure), pick from a list of

3

Replication of Asch (1946)

trait pairs which trait was most applicable to the target (trait-pair choice measure), and rank the traits in the stimulus list in order of importance for determining their decisions (ranking measure). From this study (frequently referred to as the warm-cold experiment) Asch inferred that participants rated

"warm" and "cold" as relatively central in determining their impressions. Moreover, participants more often picked positive traits as fitting the target in the trait-pair choice measure in the warm condition compared to the cold condition, except for several traits, such as strength, for which there was no difference (though Asch could not explain why). Based on this experiment, Asch concluded that participants' impressions are entirely transformed if "warm" is replaced by "cold". The nine subsequent studies in "Forming Impressions of Personality" featured variations to this basic paradigm: other traits were introduced, the order of traits was manipulated, participants were asked to give synonyms for elements of the trait lists, or were asked to judge how the traits in a list relate to each other. A short summary of all ten studies is provided in Table 1.

INSERT TABLE I HERE

Based on these ten experiments, Asch (1946) concluded that perceivers form coherent, unitary impressions of others. To form such a unitary impression, perceivers attribute different meanings and weights to traits, assigning a central role to some traits (these traits determine the meaning and function of other traits) and a peripheral role to others (their meaning and function is determined by the central traits). Which traits become central and which become peripheral is fully determined by the context of the traits, so that the same trait can be central in one impression but peripheral in another.

As conclusion, Asch presents a Gestalt view on impression formation, emphasizing that traits interact dynamically to shape each other's interpretation. For example, warmth may play a central role when accompanied by traits like intelligent, skillful, industrious, determined, practical and cautious (as in

Asch's Study I), but this central role may disappear in the context of traits like obedient, weak, shallow, unambitious, and vain (as in Asch's Study IV). This shows that impressions of personality are extremely complex, and that the interpretation and meaning of a trait can completely change depending on other traits being present (the change-of-meaning-explanation). Or, as Asch says it: "the

4

Replication of Asch (1946)

gaiety of an intelligent man is no more or less than the gaiety of a stupid man: it is different in quality"

(p.287). Thus, Asch suggests that the influence of central traits is not merely due to a general affective shift (or Halo-effect): instead, he argues that the cognitive content of specific traits changes (e.g., the gaiety of an intelligent man is not merely more or less positive than the gaiety of a stupid man: it entails different qualities and behaviors). Though there is evidence that both a change-in-meaning- effect (e.g., Hamilton & Zanna, 1974; Zanna & Hamilton, 1977) and an affective shift (e.g., Kaplan,

1971; 1974) can influence impression formation, it is unclear which processes play a role in Asch's studies. To find out more about this process, we will perform textual analyses of the open-ended responses to investigate changes in affective content as well as meaning.

Interpretations of Asch's Work

Much like punch cards and slide carousels, the Gestalt-view of impression formation has slowly but surely gone out of fashion (partly because there were more simple, parsimonious explanations for

Asch's data, e.g., Anderson, 1981; Rosenberg, Nelson & Vivekananthan 1968; Wishner, 1960).

Introductory textbook authors now generally put more emphasis on Asch's research (1946) as providing evidence for a primacy-of-warmth-effect than on the Gestalt-model outlined in Asch's work.

For example, many textbooks refer solely to Asch's Study I while ignoring Study IV (that puts the primacy-of-warmth-effect in a larger perspective4; e.g., Baron & Byrne, 2004; DeLamater & Meyers,

2010; Franzoi, 2009; Hogg & Vaughan, 2011; Kassin, Fein, & Markus, 2011; Pennington, 2000;

Taylor, Peplau, & Sears, 2006; Stainton-Rogers, 2011; Worchel, Cooper, Goethals, & Olson, 2000; for a notable exception, see Hewstone, Stroebe, & Jonas, 2007). Although Asch acknowledges the importance of warmth and coldness in impression formation, in his view any trait can be central as well as peripheral. Thus, no trait is central by design, and even traits that have special importance, such as warmth and coldness, can become peripheral in a different context because the meaning and weight of a trait is never fixed or stable. As apparent in Table I, based on the results of Study I, Asch concluded that warmth could have a central role in impression formation. Importantly, however, he

5

Replication of Asch (1946)

concluded based on the results of Study IV that warmth could become peripheral in other contexts.

Because Asch discounts a strong version of a primacy-of-warmth-effect in his theorizing, we believe the only way to establish whether Asch has justifiably been regarded as the progenitor of the primacy- of-warmth-effect is by going back to his initial observations. However, as we will describe in the next section, Asch's initial observations have been sparsely replicated, and it is difficult to determine whether they provide evidence for a primacy-of-warmth-effect.

Evidence for a Primacy-of-Warmth-Effect in Asch's Original Data (1946)

Asch's initial (1946) observations are based on three types of measures: open-ended questions in which participants wrote down their general impression of the target, trait-pair choice measures in which they chose which trait (out of a pair) is most applicable to the target (see Attachment I), and a ranking measure in which they ranked the stimulus traits from most to least important in forming their impression. In this section, we will review the extent to which these three measures provide evidence for a primacy-of-warmth effect, starting with the open-ended questions.

Asch (1946) did not analyze the open-ended questions systematically, so it is difficult to determine the extent to which they really provide evidence for a primacy-of-warmth effect.

Researchers replicating Asch's research did not include the open-ended questions (e.g., Mensh &

Wishner, 1947; Semin, 1989), did not provide a systematic analysis of this measure (Veness &

Brierley , 1963), or failed to replicate Asch's effects (Luchins, 1948; Gollin, 1954). Thus, so far the results of the open-ended question data are inconclusive regarding the evidence for primacy-of- warmth, and they have not been analyzed in ways to clarify whether the change-of-meaning-effect

Asch proposes, or whether other processes best explain the given responses.

The trait-pair choice results in Asch (1946) suggest that changing a trait in a list from positive

(e.g., "warm") to negative (e.g., "cold") makes the overall impression that participants have more negative (the negative trait of the trait-pairs is chosen more frequently). These effects have been supported in several direct replications of Study I (Mensh & Wishner, 1947; Veness & Brierley, 1963;

6

Replication of Asch (1946)

Semin ,1989), Study III (Semin, 1989) and Study IV (Mensh &Wishner, 1947). However, the extent to which these data provide support for a primacy-of-warmth-effect is questionable, as replacing any positive trait for any negative trait could potentially change the overall valence of the list. Thus, the data for the trait-pair choice measure seem robust, but they may not provide the most stringent test of the primacy-of-warmth hypothesis. Moreover, Asch could not explain why changing the central trait in the list influenced only certain peripheral traits, although this was essential for his argumentation favoring a change-of-meaning-explanation and discounting simple halo effects, which (in Asch's view) should have influenced all traits. Later theorizing (e.g., Fiske, Cuddy, & Glick, 2007; Rosenberg,

Nelson, & Vivekananthan, 1968) may provide an explanation for Asch's findings: given that there are two domains in (warmth and competence), changing a warmth-related trait should change the interpretation of other warmth-related traits but not the interpretation of competence- related traits. We would like to perform additional analyses to test whether, indeed, the influencing effects of central traits are limited to traits of the same domain.

The ranking measures, in Asch's (1946) view, suggest that warmth is ranked as one of the most important traits in Study I, but not in Study IV. For Studies I and IV, these results have been replicated by Mensh and Wishner (1947). The original results of Asch's Study I are depicted in Table

II.

INSERT TABLE II HERE.

The ranking measures of Study I do not provide unequivocal evidence for a primacy-of- warmth effect; in fact, in Study I, the number of participants indicating that warmth was the most important trait guiding their impression is exactly as large as the number of participants indicating it was the least important, a limitation Asch himself acknowledges, but that seems to have been overlooked in many later references to his work. Moreover, the data are reported incompletely by

Asch so it is unclear which trait perceivers did rank first given that most participants did not rank warmth as the most important trait in guiding their impression. In Study I, warmth is ranked in first

7

Replication of Asch (1946)

position by 6 of the 42 subjects, which is the exact number that would be expected based on chance

(given that there are seven options). This could mean one of two things: either every trait was ranked in first position by an equal number of participants (six, to be exact), which seems at odds with Asch's position that warmth was a central trait in this list. Alternatively, there may be one trait that was put in first position more often than warmth (so by more than six participants), which seems to be even more strongly at odds with Asch's position that warmth was the central trait in this list. Thus, for the ranking-measure, Asch's data are difficult to interpret because they are reported incompletely and to the extent that they are interpretable, they do not provide evidence for a primacy-of-warmth-effect.5.

In sum, the open-ended responses seem important for Asch's (1946) theorizing, but they were not systematically investigated by himself, and his contemporaries failed to replicate his effects on this measure (Gollin, 1954; Luchins, 1948). The trait-pair choice measure does provide robust results, but it does not provide a stringent test of the primacy-of-warmth effect. The ranking results do not provide very clear evidence for a primacy-of-warmth-effect, are only reported partially, have been sparsely replicated, and are underpowered. For all three measures, it is unclear whether the impressions perceivers formed based on the different stimulus lists differed significantly from each other, and what the effect size of these differences is. Because of this, we think a well-powered replication of Asch's research is in order. In the present research, we will conduct a replication of Asch's Studies I, III and

IV (these are the studies that are most relevant for the primacy-of-warmth effect), aiming to provide data that, unlike Asch's original work, are sufficiently powered, employ clear coding of the open- ended questions, provide complete accounts of the data, and provide statistical analysis of the results.

In so doing, we aim to shed light on the replicable evidence that Asch's results provide for the primacy of warmth in impression formation.

Moreover, we would like to add additional analyses of the trait-pair choice measure and several additional analyses of the open-ended responses. Asch's ranking measure may be suboptimal, as it presupposes that perceivers have full access to the factors that determine their impressions (which is unlikely; Nisbett & Wilson, 1977). We chose to focus our efforts on extra analyses for the trait-pair 8

Replication of Asch (1946)

choice measure and open ended-measure because these measures do not rely on introspection to the same extent. In our view, the descriptions of targets in particular may contain lots of new information on the primacy-of-warmth-effect. Asch did not analyze the open ended-responses, and for 67 years, researchers have focused on easy-to-analyze, top-down measures in impression formation, largely ignoring the rich information contained in open ended-descriptions. We think it is worthwhile to open this treasure chest and explore its contents (even though these analyses will be exploratory in nature).

Proposed Method

For the replication, we selected the experiments that have the strongest bearing on the primacy-of- warmth-effect, namely Studies I, III, and IV of the original publication. Here, we outline the details of the studies that we plan to conduct. Further, we give an overview of the known differences between

Asch's original work and the planned studies, and indicate how these differences could potentially influence the results.

Study Outline

In our replication attempt, we will faithfully replicate Asch's (1946) method by exposing participants to the stimulus lists (as depicted in Table I), and asking them to complete the open-ended measure, trait-pair choice measure, and ranking measure. Asch only presented a frequency table of participants' responses, and did not statistically analyze his results (Mensh & Wishner, 1947, do provide some statistical analysis of their results, but these are very limited, e.g., they do not include effect sizes). As an addition to his study, we will perform statistical analyses on the ranking measures, to test if central traits are indeed ranked as more important than other traits, and to establish the effect size. Moreover, we will examine if any trait is selected as the most important trait determining perceiver's impressions significantly more often than other traits, something that is not possible based on Asch's presentation of his data.

9

Replication of Asch (1946)

In addition to the measures employed by Asch (1946), we will analyze the open-ended responses based on the literature by Gollin (1954). Gollin distinguishes three types of impressions: unitary or unified impressions (in which all the trait information is incorporated in a unified manner), simplified impressions (impression in which part of the trait information is clearly ignored), and aggregated impressions (impressions in which no unity is achieved at all). We will develop clear coding schemas based on the research by Gollin, and will have the responses rated by two independent raters.

Moreover, we will add textual analyses of the open ended-responses to further investigate which traits are central, and how changing one trait in the list can shift the interpretation and/or valence of other traits in the list.

Participants and Design

This study will employ a between-subject design with seven conditions, as depicted in Table 3. We will recruit 1050 participants (150 per condition) to participate in our study through MTurk.

INSERT TABLE III HERE

Procedure

Participants will be randomly assigned to one of the seven stimulus lists (as depicted in Table 3) and will be asked to form an impression of the target described in the list. Next, they will first be asked to write down what their impression of the target is (open-ended question). After this, they will be instructed to select the term in each trait pair (as depicted in Attachment I) that is most applicable to the person (the trait-pair choice measure). Finally, they will once again be exposed to the stimulus list, and will be asked to rank-order the traits from the most important trait in forming their impression to the least important trait.

Analysis Plan

Because the hypotheses in Asch’s paper (1946) are not stated explicitly, we tried to formulate hypotheses that fit with Asch's theorizing and are quantifiable and testable. Because there is a subjective component to our operationalization of Asch's research questions, we formulated multiple

10

Replication of Asch (1946)

hypotheses that cover a range of tests, from very lenient to very stringent. In this section, we will describe the hypotheses for the three different measures: the ranking measures, the trait pair-choice measure, and the open-ended measure. Because the study will be run on MTurk, we will remove any participants who are not native speakers. Moreover, we will remove all participants who fail an instructional manipulation check (Oppenheimer, Meyvis & Davidenko, 2009).

Ranking Measures

For the ranking measure, we propose the following hypotheses (ordered from most lenient to most stringent):

Hypothesis 1: average rankings. The first hypothesis concerns the question whether the average rank administered to a trait (e.g., warmth in Condition 1A) differs from the average rank given to other traits in the same list. It contains the following sub hypotheses:

1a) Warmth and coldness are expected to be central in Conditions 1A and 1B. Thus, on

average, warmth and coldness should have a higher average rank than any of the other traits in

that condition.

1b) Warmth and coldness are expected to be peripheral in Conditions 2A, 2B, and 2C. Thus,

there should be at least one trait with a higher average rank than warmth and coldness in that

condition.

1c) Polite and blunt are expected to be peripheral in Conditions 3A and 3B. Thus, there should

be at least one trait with a higher average rank than politeness and bluntness in that condition.

Hypotheses 1a, 1b and 1c are quite lenient: in this scenario, warmth could be the most highly ranked trait on average, even though warmth may not be the most important trait for many (or even any) of the participants. We will analyze this using two different statistical techniques. First of all, we will use a Wilcoxon signed rank analysis to investigate if the average rank for the focal trait (warmth and coldness in Conditions 1 and 2, politeness and bluntness in Condition 3) differs from the average rank of any of the other traits in a list. Moreover, we will perform an exploded Logit analysis: an analysis

11

Replication of Asch (1946)

that qualitatively yields the same results as the Wilcoxon's test, but will give us an opportunity to look at specific ranks in Hypothesis 2 (Allison & Christakis, 1994).

Hypothesis 2: average rankings for top choices. The second hypothesis concerns the question whether the average rank administered to a trait differs from the average rank given to other traits in the same list. Unlike Hypothesis 1, however, we only take the top ranks into account (Rank 1,

2 and 3). If warmth is indeed primary, this should not just be visible in the average rank over all ranks: it should hold even if we ignore all rankings lower than 3. Thus, this hypothesis rules out the possibility that warmth and coldness are regarded as central but are not well presented in the top ranks

(Rank 1, 2 and 3). Hypothesis 2 entails the following sub hypotheses:

2a) Warmth and coldness are expected to be central in Conditions 1A and 1B. Thus, on

average, warmth and coldness should have higher average ranks than any of the other traits in

that condition, even if we leave all ranks lower than 3 (4 and up) out of the analysis.

2b) Warmth and coldness are expected to be peripheral in Conditions 2A, 2B, and 2C. Thus,

there should be at least 1 trait that has a higher average rank than any of the other traits in that

condition, even if we leave all ranks lower than 3 (4 and up) out of the analysis.

2c) Politeness and bluntness are expected to be peripheral in Conditions 3A and 3B. Thus,

there should be at least 1 trait that has a higher average rank than any of the other traits in that

condition, even if we leave all ranks lower than 3 (4 and up) out of the analysis.

For these analysis, we will employ an Exploded Logit model. However, we will assign ties to all ranks higher than 3, so that only the top ranks are analyzed (based on the recommendations of Allison and

Christakis, 1994).

Trait-Pair Choice Measures

For the trait-pair choice measures, the hypotheses are as follows:

3a) Perceivers in Condition 1A should have more positive inferences of traits than participants

in Condition 1B. Thus, there should be significant difference between conditions in the

number of times each trait is selected.

12

Replication of Asch (1946)

3b) Perceivers in Condition 3A should not have more positive inferences of the target than

participants in Condition 3B. Thus, there should be no significant difference between the

conditions in the amount of times each trait is picked. If there is a significant difference, the

size of the effect should be significantly smaller than the difference between Conditions 1A

and 1B.

3c) Only traits that are related to the same domain as the trait that is changed should be

influenced. Thus, if "warm" is changed to "cold" in conditions 1A and 1B, only traits that are

warmth-related should be influenced.

Hypothesis 3a and 3b will be tested with an ANOVA. To test Hypothesis 3c, we will ask two independent raters to indicate the extent to which the traits of the trait-pair choice lists are related to warmth and competence. We expect that peripheral traits that are warm-related will differ between conditions, but competence-related traits will not.

Open-Ended Measures

For the open-ended measures, the hypotheses are as follows:

4a) All perceivers form unified impressions of personality in all conditions.

4b) Warmth and coldness will be central traits in Conditions 1A and 1B, but not in Conditions

2A and 2B. Politeness and bluntness will not be central traits in Conditions 3A and 3C.

4c) Perceivers attribute different meanings to "warm" in Conditions 1A compared to

Conditions 2A and 2B. Perceivers attribute different meanings to "cold" in Condition 1B

compared to 2C, providing evidence for a change-of-meaning-effect.

To test Hypothesis 4a, we will employ a coding scheme based on Gollin (1954), as described above.

Because analysis of these qualitative data is time-consuming, we will randomly select a third of the responses for every condition, yielding 50 responses per condition (350 responses in total). Two independent raters will rate 10 percent of these responses to establish inter-rater reliability.

To test Hypothesis 4b, we will perform textual analyses of the data. The traits that are used in participant's descriptions are extracted (negations such as "not warm" will be coded separately). We

13

Replication of Asch (1946)

expect that central traits will be mentioned by participants more often (without negation) than peripheral traits and thus should occur more frequently in the descriptions. Second, all traits will be rated on valence and their relevance to competence and warmth (the extent to which they, cognitively, refer to the warmth-domain or the competence-domain). We expect that changing a central trait will lead to an affective shift (e.g., usage of more positive terms when "warmth" is the central trait) as well as a cognitive shift (e.g., usage of more warmth-related terms if "warmth" is the central trait). These analyses are new and exploratory in nature, but have the potential to shed new light on the centrality of traits in perceivers' impressions, as well as on the veracity of a change-of-meaning-explanation for these effects.

To test Hypothesis 4c, two independent raters will receive a short list with synonyms of

"warm". Next, they will be asked to rate which synonym is most applicable to the description. We will employ a chi square-test to test if different options are selected for Condition 1A compared to 2A and

2B, and for Condition 1B compared to 2C.

Power Analysis.

Ranking measure. It is difficult to get a good estimate of the effect sizes in Asch's (1946) studies. Although it is possible to get an idea of Asch's effect sizes based on the results reported in the paper, most of his studies were likely underpowered (with only 20 to 26 participants per cell for

Studies III and IV). If we calculate effect sizes based on Study I, which may be slightly less underpowered for the ranking measure than Studies III and IV, effect sizes seem large (around w =

.50). Because of the small sample sizes and potentially smaller effects for Studies III and IV, we decided to use a slightly more conservative estimate of the effect size as a medium effect instead of a large effect (w = .30 instead of w = .50). With this medium effects size and an alpha level of .80, this would mean that we need approximately 151 participants per group (Cohen, 1992). Power analysis with G*Power (Faul, Erdfelder, Lang & Buchner, 2007) yields a highly similar number of participants

(N = 152). Thus, we expect to need approximately 150 participants per condition, or 1050 participants

14

Replication of Asch (1946)

in total. Because the experiment takes just a couple of minutes, and the analysis of the quantitative data is not labor-intensive, this number seems feasible.

For the Wilcoxon signed rank-analysis, we need 64 participants per cell to pick up on a medium effect size, so 150 participants per cell will suffice for this analysis, too. For the exploded logit analysis, we need approximately 100 participants in total-again, 150 participants per cell will suffice for this analysis.

Trait-pair choice measure. Because Hypothesis 3b is, in essence, a null hypothesis, we would like to adjust the alpha level for this test to .95, in occurrence with Cohen's (1990) recommendations for testing null effects. Again, the effect size is hard to determine;if, to be conservative, we take a low to medium effect size (e.g., Cohen's d = 0.40), power analysis suggests that we would need 136 participants per condition Thus, the sample size we need for the ranking measures (150 participants per group) would suffice for the trait-pair choice analysis.

Open ended -measure. For Hypothesis 4a, Asch (1946) stated that every participant will form a unified impression, and that every participant in his studies did so. Because this universal claim will likely not hold in our data we would like to give a more qualitative view of the data by merely presenting descriptives of the data.

To test Hypothesis 4b, we will perform simple t-tests to estimate the difference in occurrence of traits within the conditions, as well as the difference in valence and domain of the terms mentioned in the different conditions. We will include all participants in these textual analyses.

To test Hypothesis 4c, we will perform a Chi square test. With 50 participants per cell, the effect size for the t-test would need to be on the higher end (Cohen's d = 0.70) for us to have enough power with a sample size of 50 per cell to pick it up. The effect size for the Chi square test would need to be large (w = .50). Although it would be preferable to have more power for these tests, the costs of rating the responses of 350 respondents are already quite large, and we believe rating a large percentage of responses will not be viable from a practical point of view. Moreover, because Asch describes the effects of the open-ended responses as being very striking, and describes the differences

15

Replication of Asch (1946)

between conditions as large, we believe it is reasonable to predict moderately large effect sizes for these particular hypotheses.

Known Differences

Although we will try to replicate Asch (1946) as diligently and precisely as possible, there may nevertheless be differences between our study and the original work. Some of these differences are advantageous, for example:

1) Compared to the original studies, our research has more statistical power.

2) Unlike the original studies, we provide statistical analysis of the results.

3) Unlike the original studies, we can compare the conditions of Studies I, III and IV against

each other.

4) Unlike the original studies, we systematically analyze open-ended responses.

Other differences may be disadvantageous, for example:

5) Unlike the original studies, the present replication attempt will be carried out online. We think it is important to carry out the experiments in English, as translating the materials could lead to important differences in the strength and interpretation of the trait words between Asch's original study and ours. Unfortunately, as we need many participants, and do not have access to sufficiently large samples of English speakers in our lab, we would like to conduct the study online (through MTurk). It is conceivable that Asch's participants (students) were a more homogenous or motivated group than the participants in our study. Moreover, participants in Asch's study did not read the stimulus lists themselves: they received instructions verbally and answered on a sheet of paper. It is possible that a computer-based version of the experiment, in which the experimenter is not present while the participant completes the experiment, will lead to slightly different responses (e.g., less socially desirable responses). However, we have no specific reason to believe that a different mode of administration will profoundly influence the role warmth plays in impression formation.

16

Replication of Asch (1946)

6) Unlike in the original studies, it is 2013. It is possible that words had slightly different connotations in 1946 than they have in 2013, or that societal changes have brought about changes in the way people form impressions of others, or how they report those impressions. For example, there has been a considerable change in racial , or people's willingness to report prejudiced views of racial groups, since 1933 ( Devine & Elliot, 1995; Karlins, Coffman & Walters, 1969), suggesting that these specific impression formation processes have changed over time. Though it is certainly possible that the basic impression formation processes described by Asch in 1946 have also changed, we believe that these processes may be less strongly influenced by social desirability concerns brought forward by societal changes (compared to other impression formation processes, such as those related to racial groups). Moreover, we believe that changes in language use may not be particularly problematic, given that the specific words used in Asch' stimulus lists remain relatively frequent in the

English language.

Coda

Our motivation to replicate Asch's (1946) research might be taken by some as an attempt to discount his work. In fact, nothing could be further removed from the truth. We greatly admire Asch's work, which changed the face of person perception research and introduced an entirely new way of studying first impressions. Exploring his original observations with present-day (statistical) standards in mind, in our view, does not depreciate the brilliance of Asch's work, which lies in its Gestalt.

17

Replication of Asch (1946)

References

Abele, A. E., & Wojciszke, B. (2007). Agency and communion from the perspective of self

versus others. Journal of Personality and Social Psychology, 93, 751-763.

Abele, A. E., & Bruckmüller, S. (2011). The bigger one of the “Big Two”? Preferential

processing of communal information. Journal of Experimental Social Psychology, 47, 935-

948.

Ahlering, R.F., & Parker, L.D. (1989). Need for cognition as moderator of the primacy effect. Journal

of Research in Personality, 23, 313-317.

Allison, P.D., & Christakis, N.A. (1994). Logit models for sets of ranked items. Sociological

Methodology, 24, 199-228.

Anderson, N. H. (1981). Foundations of information integration theory. New York, NY: Academic

Press.

Anderson, N.H., & Barrios, A.A. (1961). Primacy effects in personality impression formation. Journal

of Abnormal and Social Psychology, 63, 346-360.

Asch, S.E. (1946). Forming impressions of personality. Journal of Abnormal and Social

Psychology, 41, 258-290.

Babad, E., Kaplowitz, H., & Darley, J. (1999). A “classic” revisited: Students’ immediate and

delayed evaluations of a warm/cold instructor. Social Psychology of Education, 3, 81-102.

Baron, R.A., & Byrne, D.E. (2004). Social Psychology. Boston, MA: Allyn Bacon.

Bem, D. J. (1987). Writing the empirical journal article. In M. P. Zanna & J. M. Darley (Eds.), The

complete academic: A practical guide for the beginning social scientist (pp. 171-201). New

York, NY: Random House.

Bertamini, M., & Munafo, M.R. (2012). Bite-size science and its undesired side-effects.

Perspectives on Psychological Science, 7, 67-71.

Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304-1312.

Cohen, J. (1992). A power primer. Quantitative Methods in Psychology, 112, 155-159.

18

Replication of Asch (1946)

Cuddy, A. J., Fiske, S. T., & Glick, P. (2008). Warmth and competence as universal

dimensions of : The content model and the BIAS map. Advances

in Experimental Social Psychology, 40, 61-149.

DeLamater, J.D., & Myers, D.J. (2010). Social Psychology. Belmont, CA: Wadsworth Publishing.

Devine, P.G., & Elliot, A.J. (1995). Are racial stereotypes really fading: The Princeton trilogy

revisited. Personality and Social Psychology Bulletin, 11, 1139-1150.

Fiske, S.T., Neuberg, S.L., Beattie, A.E., & Milberg, S.J. (1987). Category-based and

attribute-based reactions to others: Some informational conditions of stereotyping and

individuating processes. Journal of Experimental Social Psychology, 23, 399-427.

Fiske, S.T., Cuddy, A.J.C., & Glick, P. (2007). Universal dimensions of social cognition: Warmth and

competence. Trends in Cognitive Sciences, 11, 77-83.

Faul, F., Erdfelder, E., Lang, A.G., & Buchner, A. (2007). G*Power 3: A flexible statistical power

analysis program for the social, behavioral, and biomedical sciences. Behavior Research

Methods, 39, 175-191.

Franzoi, S.L. (2009). Social Psychology (5th ed.). New York, NY: McGraw-Hill.

Gilbert, D.T. (1998). Ordinary Personology. In D.T.Gilbert, S.T.Fiske, & L. Gardner (Eds.), The

Handbook of Social Psychology (4th ed., Vol.2, pp. 89-150). New York, NY: McGraw-Hill.

Gollin, E.S. (1954). Forming impressions of personality. Journal of Personality, 23, 65-76.

Grace, H.A., & Geenshields, C.M. (1960). Effect of closure on formation of impressions.

Psychological reports, 6, 94.

Hamilton, D. L., & Zanna, M. P. (1974). Context effects in impression formation: Changes in

connotative meaning. Journal of Personality and Social Psychology, 29, 649-654.

Hendrick, C., & Constantini, A.V. (1970). Effects of varying trait inconsistency and response

requirements on the primacy effect in impression formation. Journal of Personality and Social

Psychology, 18, 158-164.

Hewstone, M., Stroebe, W., & Jonas, K. (2007). Introduction to Social psychology: A European

19

Replication of Asch (1946)

Perspective (4th ed.). Hoboken, NJ: John Wiley & Sons.

Hogg, M., & Vaughan, G. (2011). Social Psychology (6th ed.). Upper Saddle River, NJ: Prentice Hall.

Jones, E. E., & Davis, K. E. (1965). From acts to dispositions: The attribution process in person

perception. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 2, pp.

219-266). New York, NY: Academic Press.

Judd, C. M., James-Hawkins, L., Yzerbyt, V., & Kashima, Y. (2005). Fundamental

dimensions of social judgment: understanding the relations between judgments of competence

and warmth. Journal of Personality and Social Psychology, 89, 899-913.

Kaplan, M. F. (1971). Context effects in impression formation: The weighted average versus

the meaning-change formulation. Journal of Personality and Social Psychology, 19,

92-99.

Kaplan, M. F. (1974). Context-induced shifts in personality trait evaluation: A comment on

the evaluative halo effect and meaning change interpretations. Psychological Bulletin,

81, 891-895.

Karlins, M., Coffman, T.L., & Walters, G. (1969). On the fading of social stereotypes: Studies in three

generations of college students. Journal of Personality and Social Psychology, 13, 1-16.

Kassin, S., Fein, S., & Markus, H. R. (2011). Social psychology (8th ed.). Stamford, CT: Cengage

Learning.

Kelley, H.H. (1950). The warm-cold variable in impressions of persons. Journal of Personality, 18,

431-439.

Kervyn, N., Yzerbyt, V., & Judd, C. M. (2010). Compensation between warmth and

competence: Antecedents and consequences of a negative relation between the two

fundamental dimensions of social perception. European Review of Social Psychology,

21, 155-187.

Luchins, A.S. (1948). Forming impressions of personality: A critique. Journal of Abnormal and Social

Psychology, 43, 318-325. 20

Replication of Asch (1946)

Luchins, A.S., & Luchins, E.H. (1986). Primacy and recency effects with descriptions of moral and

immoral behavior. The Journal of General Psychology, 113, 159-177.

Pennington, D.C. (2000). Social Cognition. (1st ed.). London, Great Britain: Routledge.

McCarthy, R.J., & Skowronski, J.J. (2011). You're getting warmer: Level of construal affects the

impact of central traits on impression formation. Journal of Experimental Social Psychology,

47, 1304-1307.

McKelvie, S.J. (1990). The Asch primacy effect: Robust but not infallible. Journal of Social

Behavior & Personality, 5, 135-150.

Mensh, I.N., & Wishner, J. (1947). Asch on "Forming impressions of personality": Further evidence.

Journal of Personality, 16, 188-191.

Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on

mental processes. Psychological Review, 84, 231-259.

Oppenheimer, D.M., Meyvis, T., & Davidenko, N. (2009). Instructional manipulation checks:

Detecting satisficing to increase statistical power. Journal of Experimental Social Psychology,

45, 867-872.

Peeters, G., & Czapinski, J. (1990). Positive-negative asymmetry in evaluations: The

distinction between affective and informational negativity effects. European Review of

Social Psychology, 1, 33-60.

Pettijohn, T.F. II, Pettijon, T.F., McDermott, L.A. (2009). Active learning exercises for teaching classic research on impression formation in social psychology courses.

The Open Education Journal, 2, 78-81.

Richetin, J., Durante, F., Mari, S., Perugini, M., & Volpato, C. (2012). Primacy of warmth versus

competence: A motivated bias? The Journal of Social Psychology, 152, 417-435.

Rosenberg, S., Nelson, C., & Vivekananthan, P. S. (1968). A multidimensional approach to

the structure of personality impressions. Journal of Personality and Social Psychology, 9, 283-

294.

21

Replication of Asch (1946)

Semin, G.R. (1989). The contribution of linguistic factors to attribute inferences and semantic

similarity judgements. European Journal of Social Psychology, 19, 85-100.

Singh, R., Onglacto, M.L.U., Sriram, N., & Tay, A.B.G. (1997) The warm-cold variable in impression

formation: Evidence for the positive-negative asymmetry. British Journal of Social

Psychology, 36, 457-477.

Stainton-Rogers, W. (2011). Social Psychology (2nd ed.). Philadelphia, PA: Open University Press.

Taylor, S. E., Peplau, L. A., & Sears, D. O. (2006). Social psychology (12th ed.). Upper Saddle River,

NJ: Prentice-Hall.

Todorov, A., Said, C.P., Engell, A.D., & Oosterhof, N.N. (2008). Understanding evaluation of faces

on social dimensions. Trends in Cognitive Sciences, 12, 455-460.

Ybarra, O., Chan, E., & Park, D. (2001). Young and old adults' concerns about morality

and competence. Motivation and , 25, 85-100.

Veness, T., & Brierley, D.W. (1963). Forming impressions of personality: Two experiments. British

Journal of Social and Clinical Psychology, 2, 11-19.

Vonk, R. (1994). Trait inferences, impression formation, and person memory: Strategies in

processing inconsistent information about persons. European Review of Social Psychology, 5,

111-149.

Wishner, J. (1960). Reanalysis of "impressions of personality". Psychological Review, 67, 96-112.

Wojciszke, B., Bazinska, R., & Jaworski, M. (1998a). On the dominance of moral categories

in impression formation. Personality and Social Psychology Bulletin, 24, 1245-1257.

Wojciszke, B., Dowhyluk, M., & Jaworski, M. (1998). Moral competence-related traits: How

do they differ? Polish Psychological Bulletin, 29, 283-294.

Wojciszke, B. (2005). Morality and competence in person- and self-perception. European Review of

Social Psychology, 16, 155-188.

Worchel, S., Cooper, J., Goethals, G., & Olson, J. (2000). Social psychology. Belmont, CA:

Wadsworth.

22

Replication of Asch (1946)

Zanna, M. P., & Hamilton, D. L. (1977). Further evidence for meaning change in impression

formation. Journal of Experimental Social Psychology, 13, 224-238.

23

Replication of Asch (1946)

Footnotes

1In the present research, in line with the recommendations by Fiske, Cuddy and Glick (2007), warmth is used as an omnibus term that includes dimensions such as other-profitability (Peeters & Czapinski,

1990), morality (Wojciszke, 2005), and trustworthiness (Todorov, Said, Engell, & Oosterhof, 2008).

2We do not claim to provide a complete and unbiased view of Asch's (1946) complex work; in trying to provide a readable overview, we have omitted countless details that are only accessible in the original work. For a truly complete appreciation of Asch, please read Asch.

3A well-informed reader may notice that Asch writes in his introduction that he tested over a 1000 participants, but the results of 834 (not 1000) are reported in Asch's method-sections.

4Although some authors additionally refer to Study VI or VII about primacy-effects.

5We would like to note that our goal is not to question the veracity of the primacy-of-warmth effect per se (there is a lot of research suggesting that warmth plays a central role in impression formation; for a review, see Fiske, Cuddy, & Glick, 2007; Wojciszke, 2005). Instead, the aim is to investigate to what extent Asch's initial observations can be taken as evidence for a primacy-of-warmth-effect, and to what extent they suggest that there may be limitations or boundary conditions to such an effect (as

Asch hypothesized). We believe this is not merely important from a historical perspective: person perception researchers have been strongly inspired by Asch's research, and we hope that a more complete, interpretable overview of his results, combined with some additional analyses that explain more about the processes underlying the effects can inform new research.

24

Replication of Asch (1946)

Tables

Table 1.

Short Overview of the Methods of the Ten Experiments Reported in Asch (1946), Including the

Conclusions that Asch Draws Based on the Experiments.

Study Included? Stimulus lists Method and Conclusions

I yes group A (N = 90): Participants read the stimulus lists, and wrote down the impression they had of the target intelligent, skillful, (open-ended measure). Next, they had to indicate for a list of traits (see Attachment I) industrious, warm, which was most in accordance with their impression (trait-pair choice measure). Next, determined, practical, they had to rank order which traits of the stimulus list were most important in cautious determining their impressions (ranking measure).

group B (N = 76): Summary of Asch's main conclusions: intelligent, skillful, industrious, cold, a) Based on the open-ended measure, Asch concluded that perceivers formed a unitary determined, practical, impression, that changes radically when "warm" is changed to "cold". cautious b) Based on the trait-pair choice measure, Asch concluded that impressions reversed for certain qualities (e.g., generous, happy), but not others (e.g., honest, strong) when "warm" is changed to "cold".

c) Based on the ranking measures, Asch concludes that some characteristics (such as warmth) were more central than others.

II no (N = 56): intelligent, Participants read the stimulus list and completed the open-ended measure and ranking skillful, industrious, measure. determined, practical, cautious Summary of Asch's main conclusions:

a) Based on the ranking measure, Asch concluded that omitting warm/cold from the stimulus list does not function entirely as an omission. Instead, approximately half of the subjects spontaneously forms a warm impression, the other half a cold one.

III yes group A (N = 20): Participants read the stimulus list and completed the open-ended measure and ranking intelligent, skillful, measure. industrious, polite, determined, practical, Summary of Asch's main conclusions: cautious a) Based on the ranking measure, Asch concluded that changing a peripheral quality, group B (N = 26): such as "polite" to "blunt", influences participant's impression less strongly than intelligent, skillful, changing a central quality, such as "warm" to "cold" industrious, blunt, determined, practical, cautious

IV yes group A (N = 23): Participants read the stimulus list and completed the open-ended measure and ranking obedient, weak, shallow, measure.

25

Replication of Asch (1946)

warm, unambitious, vain Summary of Asch's main conclusions:

group B (N = 21): vain, a) Based on the open-ended measure, Asch concluded that "warm" is interpreted shrewd, unscrupulous, differently in this context than in the context of Study I. warm, shallow, envious b) Based on the ranking measure, Asch concluded that warm/cold is much less important group C (N = 20): in determining impression than it is in Study 1: it has become peripheral and is no longer intelligent, skillful, sincere, central. Thus, a trait can be central or peripheral depending on the context of the cold, conscientious, impression. helpful, modest

V no group A (N = 38): kind, Participants read the stimulus list and completed the open-ended measure and give wise, honest, calm, strong synonyms for calm and strong (synonym measure).

group B (N = 41): cruel, Summary of Asch's main conclusions: shrewd, unscrupulous, calm, strong a) Based on the synonym measure, Asch concluded that people attribute a different (more positive) meaning to calm and strong in group A than group B. Thus, the interpretation of a trait depends on the context.

VI no group A (N = 34): Participants read the stimulus list and completed the open-ended measure, trait-pair intelligent, industrious, choice measure and ranking measure. impulsive, critical, stubborn, envious Summary of Asch's main conclusions:

group B (N = 24): a) Based on the trait-pair choice measure, Asch concluded that more positive qualities envious stubborn, critical, are chosen for group A compared to group B, suggesting that traits early in the list have a impulsive, industrious, stronger influence on impressions (evidence for a primacy-effect). intelligent b) Based on the ranking measure, Asch concluded that traits early in the series are ranked as more important in determining participant's impressions (also evidence for a primacy- effect).

VII no group A (N = 27): Participants read the stimulus list and completed the open-ended measure, trait-pair intelligent, skillful, choice measure and ranking measure. industrious, determined, practical, cautious, evasive Summary of Asch's main conclusions:

group B (N = 30): evasive a) Based on the trait-pair choice measure, Asch concludes that more positive qualities are skillful, industrious, chosen for group A compared to group B, suggesting that traits early in the list have a determined, practical, stronger influence on impressions (evidence for a primacy-effect). cautious, intelligent b) Based on the ranking measure, Asch concluded that traits early in the series are ranked as more important in determining participant's impressions (also evidence for a primacy- effect).

This is a replication of Study VI with similar (though slightly weaker) effects.

VIII no Group A (N=24): Participants either read the stimulus list as if it described one person (group A) or as if it intelligent, industrious, described two people (group B) and had to complete the open-ended and trait-pair choice impulsive, critical, measure. stubborn, envious Participants in group B later heard that the lists described one single person instead of

26

Replication of Asch (1946)

Group B (N = 52): two, and were asked to again form an impression (second open-ended measure).

person1:intelligent, Summary of Asch's main conclusions: industrious, impulsive a) Based on the open-ended questions, Asch concluded that perceivers have trouble person 2: critical, stubborn, forming a unified impression in group B, but not in group A, suggesting that the envious interpretation given to traits depended on the context of other traits, With a sudden change in context, participants' initial interpretations were no longer fitting.

IX no IX part1: intelligent- Participants read the stimulus list and completed the open-ended measure and trait-pair skillful-warm choice measure.

IX part2: group A (N = Summary of Asch's main conclusions: 22): warm a) Based on the trait-pair choice measure, Asch concluded that warmth is central, but that group B (N = 33): cold it has more weight than in Study I, suggesting that even if warmth may have been central in Study I, it was influenced by the context of other, peripheral traits. Thus, the weight and meaning of central traits depends on the context.

X no Participants saw all of the Participants read all four stimulus list and completed the open-ended measure and following four sets: indicated which traits they thought resemble each other (e.g., does "quick" from Set 1 resemble "quick" from Set 2, or "slow" from Set 3?; the similarity measure). Set 1: quick, skillful, helpful Summary of Asch's main conclusions:

Set 2: quick , clumsy, a)Based on the similarity measure, Asch concluded that different traits are sometimes helpful judged as more similar (e.g., quick in Set 1 and slow in Set 3) than similar traits (e.g., quick in Set 1 compared to quick in Set 2). This suggests that the meaning and Set 3: slow, skillful, interpretation of traits depends on the context. helpful

Set 4: slow, clumsy, helpful

He also uses a similar methodology with different stimulus lists but similar results.

Note: Studies that will be included in our replication attempt are marked with "yes" under "included?".

"Stimulus lists" features the lists of stimuli that participants were exposed to.

27

Replication of Asch (1946)

Table II

Original Data of Experiment I (Asch, 1946); Distribution of Ranking of the “Warm-Cold” Qualities in

Order of Their Importance in Forming Impressions.

Warm Cold

Rank n Percentage n Percentage

1 6 14 12 27

2 15 35 8 21

3 4 10 1 2

4 4 10 2 5

5 4 10 3 7

6 3 7 2 5

7 6 14 13 33

Total 42 100 41 100

28

Replication of Asch (1946)

Table III

Conditions Included in our Replication and the Stimulus List Participants are Exposed To.

Condition in our Replication Condition in Asch (1946) Stimulus List

Condition 1 A Study I, Group A intelligent, skillful, industrious, warm, determined, practical, cautious

Condition 1 B Study I, Group B intelligent, skillful, industrious, cold, determined, practical, cautious

Condition 2 A Study IV, Group A obedient, weak, shallow, warm, unambitious, vain

Condition 2 B Study IV, Group B vain, shrewd, unscrupulous, warm, shallow, envious

Condition 2 C Study IV, Group C intelligent, skillful, sincere, cold, conscientious, helpful, modest

Condition 3 A Study III, Group A intelligent, skillful, industrious, polite, determined, practical, cautious

Condition 3 B Study III, Group B intelligent, skillful, industrious, blunt, determined, practical, cautious

29

Replication of Asch (1946)

Attachment I

Checklist (Trait-Pair Choice Measure) as Used in Asch (1946)

"Choose the characteristic that is most in accordance with the view you have formed."

1. generous-ungenerous

2. shrewd-wise

3. unhappy-happy

4. irritable-good natured

5. humorous-humorless

6. sociable-unsociable

7. popular-unpopular

8. unreliable-reliable

9. important-insignificant

10. ruthless-humane

11. good looking-unattractive

12. persistent-unstable

13. frivolous-serious

14. restrained-talkative

15. self centered-altruistic

16. imaginative-hard headed

17. strong-weak

18. dishonest-honest

30

Replication of Asch (1946)

Attachment II

Grant Application

Activity Expected costs

Recruiting 1050 participants through MTurk 1050 * $0.70 = $735

Hiring an RA to rate 350 open-ended responses* $1200

total costs $1935

*Undergraduate Research Assistants at Radboud University Nijmegen costs €21.21 per hour

(including taxes, social security and health insurance requirements) and may not be paid less due to

University regulations. We expect that an RA will be able to rate 10 responses per hour on the dimensions specified in the analysis plan (35 hours in total), and that we will need approximately seven hours extra to have 10% of the responses by an independent rater (another RA), and have the raters discuss cases on which they do not agree. This means that we will need approximately 42 hours in total (€890, 82, which is approximately $1200).

31