Self-Administered Test of Social Intelligence 1
Total Page:16
File Type:pdf, Size:1020Kb
SELF-ADMINISTERED TEST OF SOCIAL INTELLIGENCE 1 The Social Shapes Test as a Self-Administered, Online Measure of Social Intelligence: Three Studies with Typically Developing Adults and Adults with Autism Spectrum Disorder Matt I. Brown1, Christopher F. Chabris1, & PatriCk R. HeCk1 1Geisinger Health System, Lewisburg, PA Running Head: SELF-ADMINISTERED TEST OF SOCIAL INTELLIGENCE Version 1 submitted for peer-review at the Journal of Autism and Developmental Disorders, 29 July 2021. This paper has not been peer reviewed. Please do not copy or cite without author's permission *Address correspondence to: Matt I. Brown Geisinger Autism and Developmental MediCine Institute 120 Hamm Drive, Suite 2A, MC 60-36 Lewisburg, PA 17837 [email protected] Acknowledgments: We thank Lauren Walsh for her research assistance and Antoinette DiCrisCio, Cora Taylor, and Vanessa Troiani for their helpful feedbaCk on our manusCript. We also thank Jamie Toroney, Kiely Law, and the SPARK Consortium for their assistance in reCruiting partiCipants for this study. This work was supported by funds from the National Institutes of Health (grant U01MH119705). SELF-ADMINISTERED TEST OF SOCIAL INTELLIGENCE 2 AbstraCt The Social Shapes Test (SST) was designed as a measure of social intelligence, theory of mind, or mentalizing without using human faCes or relying on extensive verbal ability and is Completely self-administered online. Despite promising validity evidence, there have been no studies of whether this task is suitable for adults with autism speCtrum disorder (ASD). ACross three studies, we find that the SST is an equally reliable test for adults with ASD. We deteCt a modest difference in SST performance due to ASD and no evidence for differential item functioning. We also provide a ten-item version for whenever testing time is limited. The SST is a suitable measure of social intelligence, espeCially for studies where remote, online assessment is needed. Keywords: autism speCtrum disorder, social cognition, social intelligence, theory of mind; online testing SELF-ADMINISTERED TEST OF SOCIAL INTELLIGENCE 3 Social and emotional skills are important for adaptive functioning in everyday life (Soto et al., 2020). These abilities have been Categorized or labeled using a variety of terms, including social intelligence (SI), perspeCtive-taking, mentalizing, and Theory of Mind (Olderbak & Wilhelm, 2020). CliniCal researchers have developed an array of psychologiCal assessments to assess these abilities (e.g., Abrahams et al., 2019) and to investigate the phenotypes of various psychologiCal conditions including autism speCtrum disorder (ASD; Morrison et al., 2019) and sChizophrenia (Pinkham et al., 2017a). Cognitive neurosCience researchers have also used these tools to identify areas of brain associated with social functioning (SChaafsma et al., 2015). Research on the development and implementation of social-Cognitive measures can therefore Contribute to our understanding of these important etiologies (Casaletto & Heaton, 2017) however most existing tools have focused on reCognizing emotion in human faCial expressions or interpreting written desCriptions of social interaCtions. Due in part to the COVID-19 epidemiC, many cliniCal and research praCtiCes have needed to shift from administering in-person assessments to implementing measures whiCh can be administered remotely online or during telemediCine visits (Türközer & Öngür, 2020). This shift to online, non-proctored assessment has already beCome the predominant mode in employment testing (e.g., Tippins, 2015) prior to the pandemiC. In contrast, many of the assessments used in CliniCal or developmental research are designed for in-person administration. Although reCent work has helped establish new, web-based cognitive ability assessments (e.g., Biagianti et al., 2019; Wright, 2020), few have focused on designing remote measures of SI or Theory of Mind. This is important given that these forms of social cognition are generally affeCted by ASD or similar, developmental disorders (Velikonja et al., 2019). Presently, many of the SI measures whiCh can be administered online rely on self- or observer-reports such as the Social SELF-ADMINISTERED TEST OF SOCIAL INTELLIGENCE 4 Responsiveness SCale (SRS; Constantino et al., 2003), the Autism Quotient (AQ; Baron-Cohen et al., 2001b), or the Broad Autism Phenotype Questionnaire (BAPQ; Hurley et al., 2007). In Contrast, many of the validated, objeCtive SI tests are designed to be completed in-person and administered by a proctor or trained cliniCian. This makes them ill-suited for remote administration and presents a challenge given the growing need for remote, web-based assessments. To this end, the Social Shapes Test (SST; Brown et al., 2019) was developed as a simple, self-administering SI test using distinct materials from many other SI measures and with promising evidence for Convergent validity. To date, however, the SST has only been used in samples of typiCally developing adults. Therefore, we conducted the present study to examine whether the SST can be used to provide reliable yet remote measurement of SI for adults with or without an ASD diagnosis. Measuring SI using Animated Shape Tasks Heider and Simmel (1944) famously observed that research partiCipants often desCribed the movements of animated, geometriC shapes in human psychologiCal terms. This pioneering work inspired research by Klin (2000), who found that individuals with autism or Asperger’s disorder made fewer social attributions to the Heider and Simmel video Compared to members of an unaffeCted control group. At the same time, Abell and colleagues (2000) reported similar results using animations based on the Heider and Simmel video. Subsequent research inspired by these findings has largely observed group differences due to ASD among children and adults in performing different types of animated shape tasks (Burger-Caplan et al., 2016). Moreover, researchers have also used shape tasks to quantify differences in SI or social attribution due to sChizophrenia (Bell et al., 2010; Johannesen et al., 2018; KoelkebeCk et al., 2010) and to identify SELF-ADMINISTERED TEST OF SOCIAL INTELLIGENCE 5 neurophysiologiCal anteCedents, including differences in eye movement (ZwiCkel et al., 2010), and regions of brain aCtivation (Weisberg et al., 2012). One benefit of these animated shape tasks is that they rely less on reading skill or verbal knowledge and Comprehension compared to other measures of SI. More verbally-loaded tasks may allow individuals to use their verbal skills to compensate for low SI (Livingston & Happé, 2017), or may be poorly-performed not beCause of low SI but beCause of low verbal ability. This is important given that one of the few SI tests whiCh can be freely administered online, the Reading the Mind in the Eyes test (RMET; Baron-Cohen et al., 2001a), has been critiCized for its reliance on verbal content (Peterson & Miller, 2012). For example, the RMET presents emotion words that are not all Commonly used in everyday language, implying that one’s own vocabulary skill may affeCt RMET performance (Kittel et al., 2021). In addition, tasks involving human faCes tend to display sCore differences due to raCe or ethniCity among cliniCal and noncliniCal populations (Dodell-Feder et al., 2019; Pinkham et al., 2017b). In contrast, animated shape tasks have displayed few if any group differences in past research (Brown et al., 2019, Brown et al., 2021; Lee et al., 2018). Despite these advantages, many animated shape tasks are not well-suited for remote testing. Nearly all existing tasks require proctoring from a cliniCian or an administrator. For example, both tasks used by Klin (2000) and Abell and colleagues (2000) require an administrator to ask questions and to reCord and sCore verbal responses from partiCipants. Not only Could this introduce Confounding effeCts of verbal ability or rater bias, but it also increases administration time and financial costs (Livingston et al., 2019). Some tasks use a combination of free response and multiple choiCe questions (e.g., the Frith-Happé animations; White et al., 2011) or only a fixed set of multiple choiCe questions (e.g., Social Attribution Test – Multiple SELF-ADMINISTERED TEST OF SOCIAL INTELLIGENCE 6 ChoiCe or SAT-MC; Bell et al., 2010). However, both examples require an administrator to be present in order to operate the speCifiC video segments for eaCh question. This is problematiC beCause it prevents partiCipants from being able to complete these tasks remotely. These barriers will likely prevent researchers from using animated shape tasks as research studies increasingly shift from being conducted in-person to online. The Social Shapes Test (SST) The SST is a 23-item multiple choiCe social intelligence test. EaCh SST item Consists of a short, 13–23 seCond animated video whiCh includes a standard set of colored, geometriC shapes. These animations were designed to represent a broader range of emotional and mental states Compared to the original Heider and Simmel video and have been found to eliCit a similar degree of social attributions in narrative desCriptions compared to those reported by Klin (Ratajska et al., 2020). EaCh video features a different social plot where the shapes display a variety of behaviors including bullying, helping, comforting, deCeption, and playing. All videos are Controlled by the partiCipant and can be viewed as many times as desired. Before starting the SST, all partiCipants are given a sample item followed by feedbaCk