Emotion Perception Tasks: a Meta-Analysis of Individual Dif- Ferences

Home , Emotional intelligence, Emotion recognition

Kukin, 10.02.16 Exposé 1

Emotion perception tasks: a meta-analysis of individual differences

1. Theoretical background Emotion perception is a crucial interpersonal ability. Perceiving emotions such as anger, dis- gust, fear, sadness and so forth are important to develop empathy for the counterpart and to make adequate social judgements, e.g. to calm someone because he or she is afraid. Further- more, emotion perception (and identification) is postulated as an ability of emotional intelligence at the lowest level of the model of emotional intelligence (Mayer, Roberts, & Barsade, 2008). Mayer and Salovey (1997) defined emotional intelligence as a set of four abilities: perception of emotion, emotional facilitation, understanding emotions and managing emotions. However, emotion perception is treated distinctly than the other emotional intelligence constructs, since it is the only subscale from the Mayer-Salovey-Caruso Emotional Test (MSCEIT; Mayer, Salovey, & Caruso, 2002) of emotional intelligence that is related to individual performance (Day & Carroll, 2004).

1.1. Emotion perception (EP) Emotion perception refers to the ability, to correctly analyse the structure of facial muscles in order to identify an emotion (Wilhelm, Hildebrandt, Manske, Schacht, & Sommer, 2014). It is necessary to distinguish between typical and maximal behaviour in order to under- stand the measurement of constructs in general. Whereas typical behaviour is associated with tests of personality (often non-cognitive measures) “or what a person is likely to do in a given situation” (Cronbach, 1949, p.29), maximal behaviour is supposed to measure “how well a person can perform at his best” (Cronach, 1949, p. 29) and is therefore referred to ability tests (often cognitive tests). When measuring maximal performance, the person is aware of the appraisal and is motivated to show the best performance he can (Sackett, Zedeck, & Fogli, 1988). In my master thesis, the focus will be on the ability EP.

1.1.1. EP ability measurements There are a lot of tasks to assess EP from faces (including gestures, postures or paralanguage). For instance the Ekman Faces Test (Ekman & Friesen, 1976), the Japanese and Caucasian Brief Affective Recognition Test (JACBART; Matsumoto et al., 2000), the Diagnostic Analysis of Non- verbal Accuracy (DANVA; Nowicki & Ducke, 1994), the Profile of Nonverbal Sensitivity (PONS; Rosenthal, Hall, DiMatteo, Rogers, & Archer, 1979), the Communication of Affect Receiving Ability Test (CARAT; Buck, 1976), a subscale of MSCEIT (Mayer, Salovey, & Caruso, 2002), the Multimodal Emotion Recognition Test (MERT; Bänzinger, Grandjean, & Scherer, 2009), the Geneva Emotion Recognition Test (GERT; Schlegel, Grandjean, & Scherer, 2014) and the task battery BeEmo developed by Wilhelm et al. (2014) are either frequently used in individual differences research or are new promising instruments. There are some tasks (e.g. MSCEIT or BeEmo) that employ additional subscales beyond emotion perception like further facets of Kukin, 10.02.16 Exposé 2

emotional intelligence. In this case, the focus of my master thesis will still be on the emotion perception results of those scales. In most of the EP tasks, faces with different emotional expressions (e.g. happiness, sadness, anger and fear) are presented, which vary either in their intensity of displayed emotion, in their view of targets face or in their dynamic aspect (video vs. picture). The participant’s task is to correctly identify the displayed emotion. Thus, a pro- portion of correctly solved trials can be calculated for each participant and each emotion as a performance indicator.

1.2. Meta- analysis of individual differences A meta-analysis is crucial to integrate research findings across studies into a conclusion (Hunter & Schmidt, 2004). In the field of differential psychology, individual differences are often examined with tests. In order to indicate how precise a test distinguishes between indi- viduals, a reliability of the test scores is estimated. The more reliable tests scores are, the more precise is the measurement (Cronbach, 1947). Furthermore, the reliability is also dependent on current sample (e.g. restricted range in emotion perception performance). One definition of reliability concerns internal consistency, it describes how closely related all items in a test are or to which extend they measure the same construct (Cronbachs α; Cronbach, 1951). Since reliability is an important quality criterion for a test, high reliable test scores are desirable in the examination of individual differences. One possible method to retain cumulative reliabilities of test scores is to conduct a meta-analysis of its psychometric properties (RG: reliability generalization; e.g. Vacha-Haase, 1998; Bonett, 2010). Unfortunately, nearly two out of three studies did not report any reliability and in some studies there was a confusing reference of reliability coefficients (Vacha-Haase, 1998). But in most of the studies, sample means, standard deviations, sample size and number of used items are still reported. The new method elaborated by Doebler and Doebler (in progress) benefits from this information to estimate reliability coefficients and to detect individual differences. Kieffer and Reese (2002) estimated a reliability coefficient in their RG study of the geriatric depression scale for the data that did provide an information about the scale mean, the standard deviation and the number of items. Their kind of estimation presumes that all items difficulties are equal. Bond and De- Paulo (2008) also used means and measurement-corrected standard deviations in their meta- analysis to make a point about individual differences in judging deception. Nevertheless, their assumption about a binomial distribution of percent correct scores (indicating a lie vs. truth) is based on a supposition of absent individual differences in judging deception, meaning all judges in a given sample have the same probability to identify a message as a lie or truth. However, this assumption is not plausible (see next chapter). Additionally, the fact of heterogeneity between studies is ignored in their method.

1.3. Beta-binomial distribution In the new meta-analytical approach by Doebler and Doebler (in progress), a beta-binomial distribution of test data is assumed. Keats and Lord (1962) discussed in their paper the assumption of a beta-binomial distribution for dichotomously scored test data with large sam- ples and its relationship to Cronbachs α. As already mentioned, it is not plausible to use a Kukin, 10.02.16 Exposé 3

binomial-distribution model for test data, in case there are individual differences in sum of scores. This data would be overdispersed, because there is more variation in the data than is predicted in a binomial-distribution model (Albert, 2009). Thus, a better fitting model would be beta-binomial distributed with an additional free parameter. If a variable is binomial distributed with two parameters n and p and p is itself a beta variable with the parameters π and ρ, then the resulting variable is beta binomial distributed (BBD) with the following probability function (Forbes, Evans, Hastings, & Peacock, 2011):

푛 퐵 (v + 푥, 푛 − 푥 + w) 푓(푥) = ( ) . 푥 퐵 (v, w)

N would be the number of items in this context. B is a usual beta function with unknown parameters v and w. It is of interest to estimate these two parameters in order to obtain the mean percent correct score ρ (ρ ∈ (0,1)) of one person and the correlation π (π ∈ (0,1)) between one person’s dichotomy answers. Both parameters can easily be transformed into each other (Doebler & Doebler, in progress). A lot of authors e.g. Wilcox (1979) and Yamamoto and Yanagimoto (1992) employed the method of moments to estimate these two parameters π ∑푘 (푥 −푥)² and ρ. Given a sample mean of 푥 and a sample variance of 푠2 = 푖=1 푖 with a sample of k 푘−1 persons, the moment estimators for π and ρ would be

푥 푛푠2− 푥(푛−푥) π̂ = and ρ̂ = . 푛 (푛−1)푥(푛−푥) Doebler and Doebler also applied a logit transformation to handle large π values in case of easy tasks. Furthermore, a link between Cronbachs α and the parameter ρ was discussed in Keats and Lord paper (1962). As a result, α can be estimated with the same given information using the following equation: 푛ρ̂ α = . 1+(푛−1)ρ̂

As already mentioned, it is very important to obtain reliability coefficients in a meta-analysis in order to provide information about a specific measurement, for instance a measurement for emotion perception data. Since it is also of interest that this measurement detects individual differences in emotion perception, a between sample correlation of π and ρ is needed, which cannot be simply estimated with a usual univariate meta-analysis. For this purpose, an- other statistical approach with a multivariate (in this case a bivariate) meta-analysis as de- scribed in Gasparrinni, Armstrong, and Kenward (2012) is used to estimate both parameters. With their R-package mvmeta, not only a multivariate meta-analysis with random effects can be conducted, but also a meta-regression is applied.

1.4. Research questions The aim of the master thesis would be to test the introduced meta-analytical approach by Doebler and Doebler with existing emotion perception data. The focus will be on the measurement of emotion perception: Kukin, 10.02.16 Exposé 4

1.) On the assumption of a valid beta-binomial distribution of the emotional perception measurement data, how well can the estimation of individual differences be done with the meta-analytical approach of individual differences: A comparison of the reliabilities estimated with this new approach vs. given Cronbach’s alpha calculations in emotion perception data. 2.) Examination of the best measurement for emotion perception data by detecting an in- strument with the highest reliability scores. 3.) In addition, potential moderators of EP tasks difficulty and individual differences will be investigated.

2. Methods 2.1. Research strategies The literature research strategy will be performed on several data bases including PsycINFO, PSYNDEX, PsycARTICLES, Embase, MEDLINE, google scholar and the Cochrane Library. More- over, the ProQuest Dissertation and Thesis will be included to look for unpublished grey literature. In addition, the backward search will be very useful to detect all studies that cited specific emotion perception tasks. Note that emotion recognition and emotion perception are used inconsistent in the literature, wherefore both terms are used in the literature search:

OR AND

 emotion* perce* 1) Ekman Faces Test (pictures of facial affect)  emotion* identif* 2) JACBART  emotion* recogn* 3) CARAT  emotion* intelligence 4) DANVA  nonverbal communication 5) PONS  nonverbal behavio?r 6) MSCEIT  nonverbal accuracy 7) MERT  nonverbal sensitivity  affect recogn* 8) GERT  affect receiv* 9) BeEmo (test battery for measuring the perception…)

Each literature search will be conducted separately for particular EP test. One example for DANVA could be: ((emotion* perce* or emotion* identif* or emotion* recogn* or emotion* intelligence or nonverbal communication or nonverbal behavio?r or nonverbal accuracy or nonverbal sensitivity or affect recogn* or affect receiv*) and DANVA).af. Piloting those search terms in the databases PsycINFO, PSYNDEX, PsycARTICLES, Embase, MEDLINE and the Cochrane Library resulted in (from EP task 1 to 10): 1583, 131, 12, 175, 903, 1335, 65, 37 and 2 outcomes.

Kukin, 10.02.16 Exposé 5

2.2. Inclusion criteria As the planned literature research already implies, only the 9 different emotion perception measures will be included in this meta-analysis in agreement with an expert team of University of Ulm. There is still some worthy of discussion, whether to include the Brief Affect Recogni- tion Test (BART; Ekman & Friesen, 1974), a precursor of the JACBART test (Matsumoto et al., 2000) or whether it would be enough to stay with the Ekman Faces Test by Ekman and Friesen (1976). Besides, only ability measures of emotion perception will be taken into account. The focus of the sample will be on healthy adults between 18 and 60 years, but control group scores of clinical studies will still be included. As previously reported, only scales that focus on EP (e.g. instead of memorizing or more specifically recognizing emotions) will be examined. Not only English, but also German papers will be included in this analysis. Although, EP can be assessed from different body parts, I will only include EP from part-whole and whole faces.

2.3. Power analysis An a priori power analysis is very import to determine a sufficient number of participants or more specifically the number of primary studies that are needed to find a statistical signifi- cance (Pigott, 2012). Although a reviewer cannot plan the available research, a power analysis could help with later decisions about analyses. Assuming a random-effect meta-analysis with a null hypothesis of H0: α= 0.6 (usual cut-off value) and an alternative hypothesis of H1: α≠ 0.6, the calculated number of studies to obtain a power of 0.8 would be 12 for a high degree of heterogeneity.

2.4. Data analysis The data analysis will be implemented with the R-package metafor (Viechtbauer, 2010) and maid (Doebler & Doebler, in progress). Both, reported and estimated Cronbachs αs will be computed as the effect size and a meta-regression will be performed to detect individual differences in emotion perception tasks. Since a random-effect model for this data is suspected, I hope to explain the heterogeneity with assumed moderators, which can be seen in Figure 1. Both moderators, number of items and emotions will be investigated confirmatory, because lower internal consistencies are suspected for shorter versions of EP tests and for emotion specific results. The rest of the possible moderators will be investigated exploratory. Kukin, 10.02.16 Exposé 6

Furthermore, I will analyse whether there is a publication bias in reported reliabilities by using funnel plots and the trill and fill (Duval & Tweedie, 2000) method to estimate the bias.

Figure 1. Logic model of the future meta-analysis with assumed moderators that could impact the reliability of emotion perception tests or subtests.

3. References

Cronbach, L. J. (1947). Test reliability: Its meaning and Albert, J. (2009). Bayesian computation with R. New determination. Psychometrika, 12(1), 1-16. York: Springer. Cronbach, L. J. (1949). Essentials of Psychological Bänziger, T., Grandjean, D., & Scherer, K. R. (2009). Testing. New York: Harper & Brothers. Emotion recognition from expressions in face, Cronbach, L. J. (1951). Coefficient alpha and the inter- voice, and body: the Multimodal Emotion nal structure of tests. Psychometrika, 16(3), Recognition Test (MERT). Emotion, 9(5), 691- 297-334. 704. Day, A. L. & Carroll, S. A. (2004). Using an ability- Bond Jr, C. F. & DePaulo, B. M. (2008). Individual dif- based measure of emotional intelligence to ferences in judging deception: accuracy and predict individual performance, group perfor- bias. Psychological bulletin, 134(4), 477-492. mance, and group citizenship behaviours. Per- Bonett, D. G. (2010). Varying coefficient meta-analytic sonality and individual Differences,36(6), 1443- methods for alpha reliability. Psychological 1458. Methods, 15, 368–385. Duval, S. & Tweedie, R. (2000). Trim and fill: a simple Buck, R. (1976). A test of nonverbal receiving ability: funnel-plot–based method of testing and ad- preliminary Studies1. Human Communication justing for publication bias in meta-analysis. Bi- Research, 2(2), 162-171. ometrics, 56(2), 455-463. Kukin, 10.02.16 Exposé 7

Ekman, P. & Friesen, W. V. (1974). Detecting decep- Test (MSCEIT) user’s manual. Toronto, Ontario, tion from the body or face. Journal of person- Canada: MHS Publishers. ality and Social Psychology, 29(3), 288-298. Nowicki Jr., S. & Duke, M. P. (1994). Individual differ- Forbes, C., Evans, M., Hastings, N., & Peacock, B. ences in the nonverbal communication of af- (2011). Statistical distributions (4nd). John fect: The diagostic analysis of nonverbal accu- Wiley & Sons. racy scale. Journal of Nonverbal Behavior, Gasparrini A., Armstrong, B., & Kenward M. G. 18(1), 9-35. (2012). Multivariate meta-analysis for Pigott, T. (2012). Advances in meta-analysis. New non-linear and other multi-parameter associa- York: Springer. tions. Statistics in Medicine, 31(29), 3821- Rosenthal, R., Hall, J. A., DiMatteo, M. R., Rogers, P. 3839. L., & Archer, D. (1979). Sensitivity to nonverbal Hall, J. A., DiMatteo, M. R., Rogers, P. L., & Archer, D. communication: The PONS test. Baltimore: (1979). Sensitivity to nonverbal communica- Johns Hopkins University Press. tion: The PONS test. Baltimore: Johns Hopkins Sackett, P. R., Zedeck, S., & Fogli, L. (1988). Relations University Press. between measures of typical and maximum Hunter, J. E. & Schmidt, F. L. (2004). Methods of job performance. Journal of Applied Psychol- meta-analysis : correcting error and bias in re- ogy, 73(3), 482-486. search findings (2nd). Thousand Oaks, CA: Schlegel, K., Grandjean, D., & Scherer, K. R. (2014). In- Sage. troducing the Geneva Emotion Recognition Keats, J. A. & Lord, F. M. (1962). A theoretical distri- Test: An example of Rasch-based test develop- bution for mental test scores. Psy- ment. Psychological assessment, 26(2), 666. chometrika, 27(1), 59-72. Vacha-Haase, T. (1998). Reliability generalization: Ex- Kieffer, K. M. & Reese, R. J. (2002). A reliability gener- ploring variance in measurement error affect- alization study of the Geriatric Depression ing score reliability across studies. Educational Scale. Educational and Psychological Measure- and Psychological Measurement, 58(1), 6-20. ment,62(6), 969-994. Viechtbauer, W. (2010). Conducting meta-analyses in Matsumoto, D., LeRoux, J., Wilson-Cohn, C., Raroque, R with the metafor package. Journal of Statisti- J., Kooken, K., Ekman, P., ... & Amo, L. (2000). cal Software, 36(3), 1– A new test to measure emotion recognition 48. http://www.jstatsoft.org/v36/i03/. ability: Matsumoto and Ekman's Japanese and Wilcox, R. R. (1979). Estimating the parameters of the Caucasian Brief Affect Recognition Test beta-binomial distribution. Educational and (JACBART). Journal of Nonverbal Behav- Psychological Measurement, 39(3), 527-535.. ior, 24(3), 179-209. Wilhelm, O., Hildebrandt, A., Manske, K., Schacht, A., Mayer, J. D. & Salovey, P. (1997). What is emotional & Sommer, W. (2014). Test battery for meas- intelligence?. In P. Salovey, & D. Sluyter (eds.), uring the perception and recognition of facial Emotional development and emotional intelli- expressions of emotion. Frontiers in psychol- gence: educational implications (pp. 3–31). ogy, 5(404). New York: Basic Books. Yamamoto, E. & Yanagimoto, T. (1992). Moment esti- Mayer, J. D., Roberts, R. D., & Barsade, S. G. (2008). mators for the beta-binomial distribu- Human abilities: Emotional intelligence. An- tion. Journal of applied statistics, 19(2), 273- nual Review of Psychology, 59, 507-536. 283. Mayer, J. D., Salovey, P., & Caruso, D. R. (2002). Mayer–Salovey–Caruso Emotional Intelligence