School of Psychological Science Reliability THE INTERPERSONAL PERCEPTION TASK (IPT): VALIDITY OF A PERFORMANCE MEASURE ≠ Rafael Robles, Amber Fultz, & Frank Bernieri, PhD Validity Figure 1 RESULTS CONCLUSIONS RESEARCH QUESTION • Both the IPT and the IPT-15 had poor internal consistency • The Interpersonal Perception Task (IPT) and (< .30; see Table 1) Interpersonal Task 15 (IPT-15) have Can a task with poor reliability • Both the IPT and IPT-15 showed convergent validity with the “unacceptable” reliability according to MSCEIT, DANVA2, PONS (see Table 2) still be a valid psychological classical test theory (Furr & Bacharach, 2008). • The IPT alone significantly correlated with the TAS-20. • Both the IPT and IPT-15 show convergent measure? • Neither the IPT nor the IPT-15 significantly correlated with fully written measures of emotional intelligence (STEU and validity with other measures of interpersonal STEM) or four out of the five BIG-5 personality traits sensitivity Table 1 (exception agreeableness). Published and Sampled Reliability Measures for Two • Both the IPT and IPT-15 show discriminant Versions of the Interpersonal Perception Task Figure 3 validity with personality traits. • The IPT offers a measure with objective Selected item from Costanzo and Archer’s (1989) Interpersonal Perception Task. IPT IPT-15 answers rather than consensus or theoretical n BACKGROUND Published judgments increasing its ecological validity. The Interpersonal Perception Task (IPT) was created by Mark Reliability • Use of IRT analysis on the IPT and IPT-15 Costanzo and Dane Archer (1989) as a naturalistic measure of KR-20 438/530 0.52 0.38 interpersonal sensitivity. The original task included 30 spontaneous show the latter task is, in fact, easier and recordings of two or more people interacting accompanied by a multiple Test-retest 46/52 0.70 0.73 does not gather as much information choice question about the interaction (see Figure 1). The IPT was found Current to have poor reliability so the Interpersonal Perception Task-15 (IPT-15) Reliability regarding participants with above average was later created as a shorter, easier measure with better psychometric abilities in interpersonal perception properties (Archer, Costanzo, & Akert, 2001). KR-20 181 0.14 0.26 Cronbach’s • Despite its low internal reliability, the IPT is The use of classical test theory often pushes psychometrists to use a measure’s reliability as a proxy assessment of its validity (see Figure 2). Alpha 181 0.14 0.25 a valid measure. We argue that even an unreliable measure can be a valid, informative Average ICC 181 0.01 0.02 • IRT analysis of the IPT shows the items within the task have the • Reliability IS NOT an appropriate proxy measure. ability to discriminate people with abilities ranging from θ = -2.53 Table 2 up to those with an ability of θ = 2.15 with a mean difficulty of measure for validity when selecting Figure 2 Pearson Correlations Between Two Versions of 0.65 (see Figure 3). psychological measures. the Interpersonal Perception Task and Related Constructs IPT IPT-15 Figure 4 REFERENCES -Costa, P. T., & McCrae, R. R. (2008). The Revised NEO Personality Inventory (NEO-PI-R). In The SAGE Handbook of Personality n r r Theory and Assessment: Volume 2 — Personality Measurement and Testing (pp. 179–198). 1 Oliver’s Yard, 55 City Road, London EC1Y 1SP United Kingdom: SAGE Publications Ltd. https://doi.org/10.4135/9781849200479.n9 -Costanzo, M., & Archer, D. (1989). Interperting the expressive behavior of others: The Interpersonal Perception Task. Journal of MSCEIT 181 0.17* 0.16* Nonverbal Behavior, 13(4), 225–245. https://doi.org/10.1007/BF00990295 Furr, R. M., & Bacharach, V. R. (2008). Empirical estimates of reliability. In Psychometrics: An Introduction (pp. 104-139). Los Angeles, CA: SAGE Publications, Inc. DANVA2 98 0.21* 0.24* -Furr, R. M., & Bacharach, V. R. (2008b). Validity: Estimating and evaluating convergent and discriminant validity evidence. In Convergent validity equation equating validity to a function of reliability Psychometrics: An Introduction (pp. 191-235). Los Angeles, CA: SAGE Publications, Inc. -Leising, D., Grande, T., & Faber, R. (2009). The Toronto Alexithymia Scale (TAS-20): A measure of general psychological distress. PONS 181 0.21** 0.19** Journal of Research in Personality, 43(4), 707–710. https://doi.org/10.1016/j.jrp.2009.03.009 -M. Costanzo, & B. Archer. (1994). The Interpersonal Perception Task 15 (IPT–15): A guide for researchers and teachers. Berkeley: University of California Center for Media and Independent Learning. TAS-20 84 -0.22* -0.11 -MacCann, C., & Roberts, R. D. (2008). New paradigms for assessing emotional intelligence: Theory and data. Emotion, 8(4), 540– METHOD 551. https://doi.org/10.1037/a0012746 -Mayer, J. D., Salovey, P., Caruso, D. R., & Sitarenios, G. (2003). Measuring emotional intelligence with the MSCEIT V2.0. Riggio SSI 181 -0.01 -0.02 Emotion, 3(1), 97–105. https://doi.org/10.1037/1528-3542.3.1.97 Participants: 181 (113 female) undergraduate students from Oregon State -Riggio, R. E. (1986). Assessment of basic social skills. Journal of Personality and Social Psychology, 51(3), 649–660. https://doi.org/10.1037/0022-3514.51.3.649 University (M = 22 years old, SD = 5.4 years) completed multiple STEU 99 0.14 0.12 -Robert Rosenthal, Judy A. Hall, M. R. DiMatteo, P. L. Rogers, & Dane Archer. (1979). Sensitivity to Nonverbal Communication: The PONS Test. Baltimore: Johns Hopkins University Press. psychological measures over 10-weeks for course credit. STEM 99 0.13 0.14 -S. Nowicki, & M. P. Duke. (2001). Nonverbal receptivity: The Dianostic Analysis of Nonverbal Accuracy (DANVA). In J. A. Hall & F. J. Bernieri (Eds.), Interpersonal sensitivity: theory and measurement (pp. 183–198). Mahwah, NJ: L. Erlbaum Procedure: Participants enrolled in a 10-week course about psychological Neuroticism 181 -0.09 -0.01 Associates. assessment. Throughout the course participants completed multiple behavioral, self-report, and performance measures of personality traits, Extraversion 181 0.05 -0.04 intelligence, empathy, and nonverbal decoding/encoding skills. Openness 181 0.00 -0.03 • IRT analysis of the IPT-15 show items within the task discriminate Psychometric evaluations of these measures were conducted using can discriminate between people with abilities ranging from θ = - methods stemming from classical test theory and item response theory. Agreeableness 181 0.21* 0.17* Conscientiousness 181 0.06 0.06 2.64 to θ = 0.97 with a mean difficulty of -0.94 (see Figure 4). *p < .05; **p < .01 Contact Information: [email protected].