Running head: SYSTEMATIC ERROR IN THE DETECTION
Systematic Error in the Detection of Serious Mental Illness in the Signing Deaf
Suzannah Jackson1, Louise Munro2, & Nerina Caltabiano3
1Psychology, James Cook University, 2QUT Faculty of Health, Queensland University of Technology, & 3 Department of Psychology, College of Healthcare Sciences, James Cook University
Contact details:
1 [email protected]: ORCID iD: 0000-0002-7989-2077 2 [email protected]; ORCID iD: 0000-0001-8737-4360 3 [email protected]: ORCID iD: 0000-0002-3625-8236
______
Abstract
A limitation of existing sign language translations of clinical assessments of serious mental illness (SMI) is the presence of systematic error against the language preferences of signing Deaf people. The error occurs in the presentation of responses that invariably occur in written language format, rather than in a signed language, a contravention of international translation guidelines. The present study aims to modernise the field of Deafness and Mental
Health using a web-based interface to improve the valid and reliable detection of SMI in signing
Deaf people. In this study, the psychometric properties of an Australian Sign Language (Auslan) translation of both versions of the Kessler Psychological Scales of Distress (K10 and K6), were examined. The scales were translated according to World Health Organisation World Mental
Health Survey protocols featuring graphics interchange format (GIF) technology as the response modality, then administered to 30 signing Deaf participants. Results of internal consistency showed that the K10-Auslan and K6-Auslan have acceptable reliability as measures of SMI in SYSTEMATIC ERROR 2
Australian Deaf people. The present study resolves the issue of systematic error in the detection of SMI in the signing Deaf, by introducing a translated clinical scale with the technological capacity to support sign language as the response modality.
Keywords: Deafness and Mental Health, Sign Language Test Translation, Australian
Sign Language, Kessler Psychological Scale of Distress, Graphics Interchange Format, Sign
Language Technology
SYSTEMATIC ERROR 3
Systematic Error in the Detection of Serious Mental Illness in the Signing Deaf
In Australia, there are currently no linguistically and culturally appropriate assessment tools that can measure serious mental illness (SMI) in signing Deaf people. As a result, inequality in access to appropriate mental health assessment, treatment and evidence-based intervention, places Deaf people at higher risk of developing mental disorders. Epidemiological research conducted internationally shows an increased prevalence of psychiatric illnesses in Deaf adult populations compared to hearing people (Brigman et al., 2000; de Graaf & Bijl, 2002;
Fellinger et al., 2005; Ridgeway, 1997). However, the body of research comprises studies that contain a design flaw, where the presentation of responses is displayed in written language format i.e. English, rather than a signed language, which is a contravention of international translation guidelines (Harkness et al., 2008). Moreover, the research includes comparisons of
Deaf results against hearing norms, which is problematic given the disparate communication modalities, sensory and sociocultural experiences that exist between the two groups. Deafness is heterogeneous with individuals differentiated by factors such as degree of loss, type of loss, age of onset, communicate mode and cultural identification. The prevalence of deafness worldwide is estimated to be at approximately 15-26%, with the highest rates found in low-income countries
(Fellinger, Holzinger, & Pollard, 2012). Deaf communities generally comprise individuals who are pre-lingual deaf, experience severe to profound loss and communicate via a signed language.
Signing Deaf people represent roughly seven per 10000 people worldwide and are the focus of the current report.
The Australian Deaf Community
Members of the Australian Deaf community generally identify as Culturally Deaf and regard Australian Sign Language (Auslan) as their preferred mode of communication. Auslan is SYSTEMATIC ERROR 4 the primary language of some 6500 Deaf community members and is officially recognised by the federal government as a community language (Johnston, 2004). Deaf people have varying degrees of fluency in both Auslan and English, but primarily use Auslan as their native or preferred language (Deaf Australia, 2013). According to peak advocacy body Deaf Australia, the community is an organised network of citizens with many represented on executive boards of finance, health, advocacy, arts, sporting and social organisations at national and international levels.
Yet in healthcare settings, the contemporary understanding of deafness continues to reflect an historic notion of a disability that requires amelioration. Sociomedical definitions posit deafness as a state of “abnormality”, contrasted against the duplicitously “normal” ability to hear and speak (Ferndale, Munro, & Watson, 2016). The definition was popularised in the 1800’s by proponents of the oralist approach, resulting in an international ban on the use of sign language in the education of Deaf children. The impact of sign language prohibition was so substantial that it reportedly triggered the sociolinguistic and cultural genocide of functioning Deaf communities at the time (Lane, 1993a; Ladd, 2003). The modern term ‘audism’ captures the ongoing process of structural subordination of Deaf communities, sign language and culture (Bauman, 2004). The implications at the individual level is a working assumption that a Deaf person’s happiness is dependent on acquiring fluency in the language of the dominant hearing culture. To this day, organisations that promote audism derive significant economic benefit and community support for oralist programs that discourage (and often ban) sign language. As a consequence, such programs divert resources away from the social, structural and environmental requirements of signing communities (Branson & Miller, 2002; Obasi, 2008). Hence Deaf people in need of SYSTEMATIC ERROR 5 medical support are systematically disadvantaged with respect to gaining full access to resources according to language preference.
Deafness and Mental Health in Australia
The Deafness and Mental Health Statewide Consultation Service in Queensland is the only psychiatric facility that caters to the cultural and linguistic diversity of Deaf people (The
State of Queensland, 2016). The service was established in response to a 1998 report conducted by Dianne Briffa into Australian mental health service providers that found treating psychiatrists, therapists and psychologists were not providing access to the Deaf community effectively. Yet health professionals still believe that a lack of appropriate therapeutic tools in sign language, interpreters and availability of expertise prevents them from adequately assessing SMI in Deaf people (Ferndale, Watson, & Munro, 2016). Ensuring that services accommodate language preference increases access to healthcare information (Napier & Kidd, 2013), whereas failure to do so may impact a Deaf consumer’s understanding of a range of concepts including mental health, drug and alcohol dependence and sexual health (Terry, Le, & Nguyen, 2016). Recent evidence shows mental health systems that privilege hearingness influence Deaf people to internalise disempowerment (Ferndale, Watson, & Munro, 2016) and exacerbate the course and severity of SMI (Fellinger, Holzinger, & Pollard, 2012).
The Epidemiology of Deafness and Mental Health
Epidemiological studies conducted internationally report an increased prevalence of psychiatric illnesses in Deaf adult populations. Studies in Great Britain, New Zealand, Holland and Austria involving administration of signed language versions of the General Health
Questionnaire (GHQ) found significantly higher scores in Deaf participants indicating the presence of psychiatric illness than hearing comparison groups (Brigman et al., 2000; de Graaf & SYSTEMATIC ERROR 6
Bijl, 2002; Fellinger et al., 2005; Ridgeway, 1997). More recently, a comprehensive review of the international literature on the mental health of Deaf people stated that there was ample evidence for higher rates of mental health problems in people who are Deaf than in hearing individuals (Fellinger, Holzinger, & Pollard, 2012). In Australia, Deaf adult epidemiological data is limited to one existing study that examined the psychometric properties of an Auslan version of the Outcome Rating Scale (ORS-Auslan; Munro & Rodwell, 2009). In the study, Deaf participants scored higher on the ORS-Auslan compared to hearing comparisons, yet rather than conclude that deafness is linked to increased prevalence of psychiatric illness, the authors instead implicated cultural factors to explain the disparity. For instance, they found that the clinical threshold for what was characterised an illness was lower in the Deaf group indicating that Deaf people may hold unique values and beliefs about mental illness. These diverse cultural norms might affect validity and reliability of routine psychometric instruments, like the hearing-normed
ORS-Auslan, making them clinically over-sensitive to Deaf people. This finding prompted the authors to instead focus on within-group comparisons and to date, the ORS-Auslan is the only measure able to effectively distinguish between clinical and non-clinical presentations of mental illness in the Australian Deaf community.
Clinical Assessments that Measure SMI in Deaf Communities
The conclusions that deafness is linked to an increased risk of psychiatric illness potentially disregards the greater sociolinguistic context. Survey methodologists contend that both written and signed languages are culturally incompatible (Mason, 2005), each language exploits different sensory-motor modalities, e.g., spoken English is auditory-oral whereas
Auslan, is visuo-spatial. Modality and cultural differences can impact the utility of clinical assessments normed on the auditory-oral population that are then administered to signing Deaf SYSTEMATIC ERROR 7 people and is a practice that has implications on ethical diagnosis and the provision of evidence- based intervention (Black & Glickman, 2006). If Deaf people experience higher levels of SMI, then the increase is more likely to be associated with a lack of culturally and linguistically accessible mental health information, resources, and services and not a function of deafness.
International researchers have outlined the effects of disregarding the cultural and linguistic needs of Deaf clients in a mental health context. Poor communication is known to result in diagnostic and medical errors, with most Deaf people in need of treatment either misdiagnosed or not diagnosed at all (Black & Glickman, 2006; Landsberger, Sajid, Schmelkin,
Diaz, & Weiler, 2013). Clinical assessment tools translated into signed languages are now standard practice for measuring SMI in Deaf people with many utilising advances in video, graphics and online social interfaces like Skype or Zoom. Existing online assessments include the use of webhosting interfaces featuring uploaded video-taped translations, and include: British
Sign Language versions of the Patient Health Questionnaire (PHQ) (Rogers et al., 2012), Clinical
Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) (Rogers, Evans, Campbell,
Young, & Lovell, 2014) and EuroQol (EQ-5D-5L) (Rogers et al., 2016). The Outcome Rating
Scale (ORS-Auslan) (Munro & Rodwell, 2009) uses Youtube as the hosting platform while all others are translated into video clips, like American Sign Language versions of the Rosenberg
Self-Esteem Scale (Crowe, 2002) and the Multi-dimensional Health Locus of Control Scales
(Athale et al., 2010); Austrian Sign Language versions of the WHO Quality of Life (WHO-
QOL), the Brief Symptom Inventory (BSI) and the GHQ (Fellinger et al., 2005); or alternatively, translated in real time like the Norwegian Sign Language version of The Mini International
Neuropsychiatric Interview (MINI) (Øhre, Saltnes, von Tetzchner, & Falkum, 2014) and are not available for use freely over the internet. SYSTEMATIC ERROR 8
A limitation of existing sign language translations of clinical assessments of SMI is that the measurement of responses is modality dependent. Despite presentation of items usually occurring in a visuospatial format using video clips, response options are dependent on written lexical units, for instance “All of the time”. Responses are made by way of either clicking on a written scale option or in written language format using pen and paper. This is excepting translations conducted in real time by professionals experienced in both sign language and psychiatry like the MINI (Øhre, Saltnes, von Tetzchner, & Falkum, 2014). Constraining response options to written format is concerning for three reasons: firstly, respondents are not afforded the privilege of making selections in their preferred language and must “translate” responses into written form, which secondly, constitutes measurement error, that thirdly, contravenes international mental health translation guidelines requiring that response options match presentation language (Harkness et al., 2008). In clinical scale design, the presence of measurement error can impact the integrity of a scale by obscuring the true outcome variable, should scale error pervade every study design in the literature, then the level of SMI may be systematically understated or overstated, which has clear epidemiological implications.
Rationale for the Present Study
The current lack of statistically valid and reliable clinical assessment tools represents a significant gap in the ability of mental health services to access the Deaf community effectively.
This study aimed to produce a clinical assessment tool that can measure SMI in signing Deaf people using webhosting, video and Graphic Interchange Format (GIF) innovation. The K10 is used widely in Australia as an outcome measure for SMI in both clinical and epidemiological applications (Furukawa, Kessler, Slade, & Andrews, 2003; Slade, Grove, & Burgess, 2011). K10 and K6 versions have been translated into 26 different languages and are suitable for use with SYSTEMATIC ERROR 9 vulnerable refugee (Lillee, Thambiran, & Laugharne, 2015) and Indigenous Australian populations (Nagel, Robinson, Condon, & Trauer, 2009). Deaf people may be increasingly exposed to the K10 through contact with their General Practitioner given the K10 is an outcome screener for establishing mental health plans. Without the use of an Auslan Interpreter, the measure would invariably be presented in written English, a discouraged practice given complex literacy, communication modality and sociocultural factors (Napier & Kidd, 2013; Terry, Le, &
Nguyen, 2016). Furthermore, developing translated versions is preferential to using Interpreters due to within-group standardisation procedures eliminating any confound in variation of meaning between interpreters (Rogers, Young, Lovell, & Evans, 2012). Given the lack of clinical screeners available in Auslan and the increased expectation that Deaf people will come into contact with the K10, it is clear that this study presents a suitable opportunity to measure SMI in
Deaf people and to trial GIF Auslan response technology. The aims of the present study were to:
(1) Perform translations of the K10 and K6 into Auslan according to World Health Organisation
(WHO) World Mental Health Survey (WMHS) protocols (Harkness et al., 2008), (2) Test the translated K10-Auslan and K6-Auslan versions on a sample of Deaf bilinguals to investigate statistical agreement between the two versions, and (3) Conduct online interviewing of the Deaf bilinguals’ responses to the instruments.
Methods
Participants
Participants were 30 signing nonclinical Deaf people (11 male, 19 females, 18-77 years).
All participants self-reported bilingual competency in Auslan and written English. Table 1 shows the demographic characteristics for the Deaf bilingual participants.
SYSTEMATIC ERROR 10
Table 1.
Demographic Data of the Participants of the K10 Translation Study (n =30)
Demographic Characteristics N (%) Gender Male 11 (36.7) Female 19 (63.3) Age Group 18-29 2 (6.7) 30-39 1 (3.3) 40-49 14 (46.7) 50-59 6 (20) 60+ 7 (23.3) Language Preference Auslan 19 (63.3) No Preference 11 (36.7) Identify as Culturally Deaf Yes 20 (66.7) Sometimes 10 (33.3) Degree of Hearing Loss Profound (91+ dB) 24 (80) Severe (66-90 dB) 4 (13.3) Unsure 2 (6.7) Highest level of Education Pre-Tertiary 17 (56.7) Tertiary 13 (43.3) Psychological distress in last 30 days? No 21 (70) Yes 9 (30)
The rationale for limiting the study sample to nonclinical bilinguals was to focus analyses on cultural and linguistic equivalence and to enable both within and between comparisons between all versions of the K10 and K6, a similar method was used in Rogers et al., (2016).
Advertisements, word of mouth, emails and video publication on the Facebook page Auslaners –
Deaf Community Australia were used to access the sample. Advertising included the terms “Deaf bilinguals in the languages of Auslan and English”, produced in signing format in a video clip featuring the principal investigator, making it clear that the target audience were required to be SYSTEMATIC ERROR 11 proficient across both Auslan and written English. Participants who wished to be involved in the study provided consent and confirmed they were over 18 years of age before entering the questionnaire. All information regarding participation, informed consent and the ability to withdraw at any stage were provided in Auslan video format. Not all data were usable, one respondent did not complete the survey and one self-identified English as their language preference together with indicating a severe degree of deafness hence both were excluded from demographic and further analyses. Ethics approval was granted by both James Cook University and Queensland Health Ethics Committees.
Materials
The K10 is a scale developed to measure and monitor nonspecific psychological stress experienced over a 30-day recall period (Kessler et al., 2003). The K10 is short (10 questions and
2-3 minutes to complete) and screens broadly for psychological distress, effectively discriminating DSM-IV disorders from non-cases. The K6 is a condensed version of the K10 and is presented in the same format, the only difference is that questions 1.a, 1.c,1.f and 1.g are omitted limiting the scale to six questions (See Appendix A for the K10 & Appendix B for the
K6). In Australia, the K10 has been used in state-based population surveys and Australian K10 norms are now available (Slade, Grove, & Burgess, 2011). In addition to these norms, large scale screening in Australia comparing the K10 and its counterpart, the shortened K6 version demonstrates that both scales have better discriminatory power than the GHQ-12 in detecting
SMI (Furukawa, Kessler, Slade, & Andrews, 2003). K10 and K6 versions have been translated into 26 different languages and are suitable for use with vulnerable refugee (Lillee, Thambiran,
& Laugharne, 2015) and Indigenous Australian populations (Nagel, Robinson, Condon, &
Trauer, 2009). The nature of task instructions is brief and simple, for instance, “The following SYSTEMATIC ERROR 12 questions ask about how you have been feeling during the past 30 days. For each question, please circle the number that best describes how often you had this feeling”. Respondents are then expected to consider the phrase “During the last month, how often did you feel…?” applied to each of the ten questions, for instance, Q1b. “… nervous?”.
Procedure
Sign language test translation. Test translation and interpreting are areas of study that focus on the quality of the translated terms. The process is not simply about interpreting from one language into another but involves repeated iterations to produce a version that is conceptually and cross-culturally equivalent (Harkness et al., 2008). The development of translations follows a rigorous psychometric methodology to ensure construct validity between the source and target texts (Angelelli & Jacobson, 2009). Translation inaccuracies occur when items in the source language are translated directly into the target language, a process known as formal equivalence (Nida, 1993). Formal equivalence can produce differential item functioning, which occurs when cultural groups have a different probability of endorsing an item (Stolk et al.,
2014). In test translation, even subtle or nuanced changes in item connotation can produce differential item functioning, because responses in target readers are not equivalent to responses in readers of the original source text. By contrast, dynamic equivalence ensures that translations effectively meet the linguistic, social and cultural norms of the context intended for application.
Auslan K10 and K6 versions were developed with permission of Dr Ronald Kessler prior to commencement of the project. Translation protocols were applied in accordance with the
Harvard National Comorbidity Study and the World Health Organisation (WHO) World Mental
Health Survey (WMHS) (Harkness et al., 2008) and the Sign Language Communities’ Terms of
Reference (SLCTR) (Harris, Holmes, & Mertens, 2009). The SLCTR principles highlight the SYSTEMATIC ERROR 13 worth and validity of Deaf cultures, the right of expression of sign language community realities, self-determination and self-management, the right of sign language groups to work and make decisions within their own cultural terms, sign language community control, the recognition and acceptance of sign language community diversity, reconciliation of competing interests among people who use sign language, and the worth of the group. The principal investigator is a hearing native signer, in accordance with SLCTR principles, Deaf academics and laypeople were consulted for the duration of the study.
Forward translation. A dynamic forward translation was developed by two hearing native Auslan and English bilinguals, who were both raised from birth by signing Deaf parents.
The bilinguals included the principal investigator, a Provisional Psychologist and an accredited
Auslan Para-professional Interpreter, along with a Professional Level Interpreter, who has a
Master of Education and is enrolled in a Doctorate of Philosophy. An expert panel was convened by the principal investigator for the purpose of identification and resolution of inadequate expressions or concepts of the forward translation. The panel included two Deaf native bilinguals proficient in the languages of Auslan and English, a native Auslan and English bilingual who is employed as a Deafness and Mental Health Interpreter, a native English speaker who is employed in Deafness and Mental Health, and finally a survey methodologist who has undertaken previous research with Deaf participants. Criticisms of the forward translation included suggestions for alternate sign selection, alternate choice of idioms, modifications in facial expression and recommendations for improving the quality of lighting and film.
Modifications of task instructions were required to enhance viewing of the translation in video format, the phrase “During the last month, how often did you feel…?” preceded each question
(A-J), which is in contrast to the original form where it is only stated once at the top. The SYSTEMATIC ERROR 14 rationale for this decision was to accommodate the video interface, a similar method of presenting task instructions was reported in a British Sign Language translation of the Clinical
Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) (Rogers, Evans, Campbell,
Young, & Lovell, 2014) in order to enhance the culturally appropriate conveyance of instructions. GIF and online presentation of the Auslan K10 was well-received by all panel members. All criticisms were considered with modifications included in the production of an updated version.
Back translation. A back translation of the updated K10 was then carried out by an independent Auslan and English Interpreter. The Interpreter holds a Doctorate of Philosophy and was qualified at the Conference Interpreter Level, currently the highest level in Australia. The back translation included a written English translation of the Auslan K10 version (which includes the K6). The principal investigator then compared the back translation to the original English
K10 version and found that the back translation adequately captured the semantic and cultural equivalence of the original and no further modifications were required.
Video, GIF production and web platform development. The principal investigator directed, produced and edited all video clips and GIFs, managed software and web hosting. The translation was performed by the professional Interpreter and filmed in a classroom using a green screen as a background on a video camera. Once completed, video clips and GIFS were edited on a desktop computer using film software, exported to MP4 format and uploaded to a password protected YouTube account.
The online survey tool Google Forms was selected as the web hosting platform. All information about the study including instructions, purpose, ethics and informed consent information were interpreted into Auslan and available for viewing as video clips on the web SYSTEMATIC ERROR 15 platform. Entire versions of all scales in the following order: the K10-Auslan and K6-Auslan and
English K10 and K6, followed by qualitative questions, were uploaded to Google Forms
(Questions 1-6). Each version was presented on its own webpage with respondents required to answer all questions before clicking “next” to continue onto the next section. Both Auslan versions were presented with written English instructions limited to: “Please click on the video” and “Now click on an answer”, none of the video clips or GIFS contained any written English subtitles to control for contamination of the results. Figure 1 shows the presentation modality for the K10-Auslan, question 1A.
SYSTEMATIC ERROR 16
Pre-testing and online cognitive interviewing. Pre-testing and online cognitive interviewing of representative respondents of the nonclinical target population was carried out through the online survey. Respondents were asked to systematically rate agreement of the translated K10-Auslan questions to clarify any misunderstanding of signs, phrases or idioms using a 5-point Likert scale rated from strongly agree to strongly disagree and provided extra opportunity to include qualitative responses. Respondents were also asked a series of questions to investigate attitudes ranging from the online presentation, GIF technology and use of
Interpreters. All questions were presented both in Auslan video format and English. Results of pre-testing were encouraging hence are not included in the current report. All respondents were provided with an option to participate in face to face cognitive interviewing with the principal investigator via Skype, none took up this opportunity.
Final version. Once the entire translation process was completed, a final version was produced representing the first stage of cultural and linguistic equivalence. A link to a sample version of the K10-Auslan is here: https://goo.gl/forms/9R2qDyBbjhXAAPhb2
Plan for Statistical Analysis
The analytic plan for the current study was to use Cronbach’s alpha to measure internal consistency of the scales used in the study. Statistical agreement would be examined using
Pearson’s product-moment correlation coefficient (r) to measure the strength of associations between dependent variables and t-tests would be applied to all other within and between group comparisons. To investigate cut-off scores for the Auslan K10, a Chi-Square analysis would be conducted. An apriori power analysis was calculated using the program G*Power3 (Faul,
Erdfelder, Buchner, & Lang, 2009) and revealed that in order for an effect to be detected (80% chance, one-tailed), significant at the 5% level, a sample of 67 participants would be required. SYSTEMATIC ERROR 17
Results
Statistical Analysis
Data were analysed using SPSS (IBM Corp, 2017). There were no missing data hence total scores were calculated for all fully completed forms. All data met the assumptions of normality, thus equal variances could be assumed.
Internal reliability. Table 2 shows internal consistency for each scale assessed using
Cronbach’s alpha.
Table 2.
Scale Items Cronbach’s alpha
K10-Auslan 10 .77
K10 English 10 .81
K6-Auslan 6 .65
K6 English 6 .79
Statistical agreement. Statistical agreement was assessed by comparing the strength of associations between questions of the Auslan and English versions of the K10 and K6. Table 3 shows the bivariate correlation for K10-Auslan and K10-English scores, all except for the final question “worthless” were positive and strong.
SYSTEMATIC ERROR 18
Table 3.
Pearson Product-Moment Correlations for Items of the Auslan and English Versions of the K10
K10 English
Q1A Q1B Q1C Q1D Q1E Q1F Q1G Q1H Q1I Q1J
K10
Auslan
Q1A .808** .378* .271 .620** .239 .062 .418* .132 .341 .226
Q1B .478** .403* .293 -.059 -.014 .047 .022 -.201 .474**
Q1C .870** .440* .122 .217 .148 .174 -.152 .360
Q1D .483** .493** .255 .424* .334 .175 .230
Q1E .474** .461* .240 .274 .175 .230
Q1F .413* .170 .654** .014 .203
Q1G .704** .368* .626** .046
Q1H .537** .267 .324
Q1I .521** .137
Q1J .315 Note. * p ≤ .05, **p≤ .001.
Table 4 shows the K6-Auslan and K6-English bivariate correlation, all six questions on each language test were positive and strong.
SYSTEMATIC ERROR 19
Table 4.
Pearson Product-Moment Correlations for Items of the Auslan and English Versions of the K6
K6 English
Q1A Q1B Q1C Q1D Q1E Q1F
K6 Auslan
Q1A .712** -.126 .128 .403* -.029 .295
Q1B .481** .544** .279 .439* .346
Q1C .765** .150 .101 .338
Q1D .678** .174 .162
Q1E .544** .313
Q1F .718**
Note. * p ≤ .05, ** p≤ .001.
K10-Auslan and K10 English scores. Table 5 shows the means and standard deviations for the all versions of the K10 and K6 with comparison to the Australian national averages
(Slade, Grove, & Burgess, 2011). All between group differences were found to be statistically significant.
SYSTEMATIC ERROR 20
Table 5.
Means and Standard Deviations (SD) for all Versions of the K10 and K6
Scale N M Australian t df 95% d (Effect Size) (SD) National Confidence Average Interval
K10 30 18.97 14.5 4.679** 29 (6.42, 2.51) 0.85 (large) Auslan (5.23)
K6 30 10.37 8.7 3.180** 29 (2.74, 0.59) 0.58 (medium) Auslan (2.87)
K10 30 18.67 14.5 4.095** 29 (6.25, 2.09) 0.85 (large) English (5.57)
K6 30 10.2 8.7 2.447* 29 (2.75, 0.25) 0.58 (medium) English (3.36)
Note. * p ≤ .05, **=p≤ .001.
K10-Auslan scores in relation to gender. There was no significant difference between gender and K10 or K6-Auslan scores. An independent samples t test was used to compare the average Auslan K10 and K6 scores of male (n = 11) and female (n = 19) Deaf bilinguals in this study. For the K10-Auslan, the t test was not statistically significant t(28) = -0.546, p = 0.589, two-tailed, CI (3.012, -5.204) with the male group scores (M = 18.27, SD = 4.88) being 1.096 points lower than the female group scores (M = 19.37, SD = 5.51). For the K6-Auslan, the t test was not statistically significant t(28) = -1.342, p = 0.19, two-tailed, CI (0.757, -3.638) with the male group scores (M = 9.45, SD = 2.464) being 1.44 points lower than the female group scores
(M = 10.89, SD =0.692).
K10-Auslan cut-off scores. K10 results are usually presented as cut-off scores using four levels of psychological distress to indicate prevalence and severity (Slade, Grove, & Burgess, SYSTEMATIC ERROR 21
2011). Table 6 shows the cut-off scores applied in the present study (Australian Bureau of
Statistics, 2001; Health Department of Western Australia, 2001).
Table 6.
K10 Cut-off Scores Used for the 2000 Health and Wellbeing Survey and the 2001
National Health Survey to Estimate the Prevalence of Levels of Psychological Distress
K10 Score Level of Psychological Distress
10-15 Low 16-21 Moderate 22-29 High 30-50 Very High
Table 7 shows the cut off scores for Auslan K10 results in the Deaf bilingual nonclinical sample which were not significant x2(3, N = 30) = .70, p = .87. The Auslan cut-off scores did not differ according to age range for males x2(8, N = 11) = 10.0, p = .26 nor females x 2(9, N = 19) =
6.7, p = .67.
Table 7.
Levels of Psychological Distress According to Auslan K10 in the Bilingual Sample
Level of Psychological Distress Auslan K10 Scores % Low (10-15) 26.7 Moderate (16-21) 46.7 High (22-29) 23.3 Very High (30-50) 3.3 Total 100.0 Note: Results are nonsignificant. K10 Cut-off Scores According to the Australian Bureau
of Statistic (2001) and the Health Department of Western Australia (2001).
Table 8 presents the Auslan K10 cut off scores in relation to age range and gender.
SYSTEMATIC ERROR 22
Table 8.
Level of Psychological Distress using Auslan K10 Cut-off Scores According to Age Group and
Gender
Level of Psychological Age Group Distress 18-29 30-39 40-49 50-59 60+ Total % % % % % % Male Low (10-15) 33.3 0.00 0.00 33.3 33.3 100 Moderate (16-21) 0.00 20.0 20.0 60.0 0.00 100 High (22-29) 0.00 0.00 66.7 0.00 33.3 100 Very High (30-50) 0.0 0.00 0.00 0.00 0.00 100
Female Low (10-15) 0.00 0.00 60.0 20.0 20.0 100 Moderate (16-21) 11.1 0.00 44.4 0.00 44.4 100 High (22-29) 0.00 0.00 75.0 25.0 0.00 100 Very High (30-50) 0.0 0.00 100.0 0.00 0.00 100 Note: Differences are not significant
Discussion
Internal Reliability of the Auslan K10 and K6
The present study examined the psychometric properties of a clinically appropriate
Auslan translation of the K10 and K6 psychological distress scales. Results of internal consistency showed that the K10-Auslan and K6-Auslan have acceptable reliability as culturally and linguistically appropriate measures of SMI in signing Australian Deaf people. The K6-
Auslan resulted in a slightly lower Cronbach’s alpha, however the prevailing view is that an internal reliability approaching .70 will suffice for scales that are still in the early stages of research and development (Tavakil & Dennick, 2011).
Statistical Agreement between English and Auslan Versions
Closer examination of the statistical agreement between the two scales shows strong and significant correlations between items of the K10-Auslan and K6-Auslan by comparison with SYSTEMATIC ERROR 23
English versions. This is with the exception of the final scale item on the K10-Auslan indicating the possibility of differential item functioning (Stolk, Kaplan, & Szwarc, 2014). The tenth item on the K10 incorporates the term “worthless” and has been implicated in other multicultural research as having divergent cultural connotations (Australian Institute of Health and Welfare,
2009). In Auslan, the sign for “worthless”, if translated directly, shares conceptual similarity to the English word “hopeless”, which is a separate question on the K10. In the present study, results on both the Auslan K10 and K6 showed weak correlation between “hopeless” and
“worthless” indicating that the nuanced dynamic translation of each item produced differentiated concepts in Auslan. The weak correlation of “worthless” on the K10-Auslan was not found in the
K6-Auslan version, limited statistical power due to the modest sample size in the present study
(n=30) may have played a role hence more testing is required with an increased sample (n=67+) to tease out whether this is a statistical or conceptual phenomenon.
Systematic Error in the Literature of SMI in Signing Deaf People
The overall aim of the present study was to highlight the issue of systematic error in clinical assessments of SMI in the signing Deaf. International translation guidelines state that the presentation of scale response options must match presentation language to prevent the influence of measurement error (Harkness et al., 2008). The current study reveals that all existing signed language translations of mental illness assessments contain a measurement artefact involving the use of written language expressions to represent scale response options. The repeated application of this artefact in clinical trials with signing Deaf participants may have implications for the body of literature regarding the true level of SMI in this population. Validity and reliability of sign language tests translations is crucial to obtaining a clear epidemiological picture of the prevalence of SMI, which for the signing Deaf population, may have been systematically SYSTEMATIC ERROR 24 understated or overstated. Pollard (2002) has previously cautioned that erroneous data gathering methods can produce statistical error, in this instance, limitations in technology may have hitherto prevented the inclusion of sign languages as the appropriate modality. This issue is resolved in the current study given the K10-Auslan and K6-Auslan both contain this technological capacity. Future directions for the field of study might include a systematic review of the existing literature examining whether measurement error has produced statistical deviation of the true value of SMI scores in the signing Deaf population.
Serious Mental Illness and the Signing Deaf Community
This is the first sign language translation of the K10 and K6 conducted worldwide hence there are currently no international Deaf norms of either scale available for comparative purposes. In this study, comparison to the Australian (Hearing) National Average showed that the
Deaf bilingual nonclinical sample scored significantly higher on both Auslan and English versions of the K10 and K6. There are several potential explanations for this difference.
Firstly, a general lack of clinical assessment measures normed on Deaf people makes population evaluation problematic. Moreover, systematic bias of previous assessments and the practice of making comparisons between norms from socio-linguistically dissimilar groups further complicate the epidemiological picture. Comparing Deaf people to hearing norms on standardised psychological tests is generally discouraged due to differences in the communication preferences, fund of information, socialisation experiences and cultural affiliation of each group (Connolly, Rose, & Austen, 2006; Pollard, 2002). Another issue purports to the use of potentially arbitrary categorical cut offs (low, moderate, high, very high) employed in statistical analyses in the present study given the unknown nature of the construct of
SMI in the Deaf population (Sunderland et al., 2011). K10 categorical cut offs in hearing samples SYSTEMATIC ERROR 25 are reportedly effective given high concordance rates with psychiatric diagnoses and applicability to epidemiological and clinical settings. For instance, cut offs reveal population prevalence trends and support lay clinicians to make decisions about levels of psychiatric distress in clients. Deaf population-based studies in future may employ cut offs in this manner to support the understanding of clinical thresholds and population trends of SMI using Deaf representative samples.
Secondly, previous research has shown that the clinical threshold for what is considered
“mental illness” is lower in Deaf samples due to cultural differences and beliefs about SMI
(Munro & Rodwell, 2009). Thus, discrepancies in how Deaf and hearing people experience distress versus mental health disorders may explain higher levels of SMI in Deaf people. It is understood that the stressors associated with living in a world that privileges hearingness precipitates more transient forms of distress in Deaf people rather than eliciting chronic and impairing mental health disorders (Landsberger, Diaz, Spring, Sheward, & Sculley, 2014).
International and local research demonstrates that Deaf children and adults are diagnosed at the same rates as hearing children and adolescents with internalising disorders (e.g., depressive and anxiety disorders) where clinical judgment may moderate the presence of SMI and reduce the probability of detecting transient distress (Brown & Cormes, 2015). By contrast, short sign language translated clinical screeners may be insensitive to the divergent cultural beliefs and experiences of Deaf and hearing people thereby erroneously eliciting higher levels of mental illness in Deaf samples.
Thirdly, it is possible that the hearingness of the principal investigator represented a source of bias contributing to statistical error in production of the translation. Deaf communities may have a distinctive fund of information as a result of cumulative sensory and sociocultural SYSTEMATIC ERROR 26 experiences. Living as a Deaf person is a cultural artefact unavailable to hearing researchers despite any mitigating level of fluency in a signed language. Thus, to address potential sources of bias, hearing researchers working in signing communities are expected to abide by the SLCTR principles (Harris et al, 2009). In the present study, a conscientious effort was made to comply, for instance, the forward translation was developed by native signers in collaboration with both
Deaf academics and laypeople, instructions were modified to include sign language, tests were readily available using online webhosting and sign language constituted the response modality.
Nevertheless, the issue raises ethical questions about the degree of involvement that hearing researchers have in the production of translations in the field of Deafness and Mental Health.
Finally, the importance of establishing Deaf norms is further supported in the results of the present study. There were no significant gender differences between Deaf bilinguals in this sample despite almost every Australian English K10 study demonstrating that females experience greater levels of SMI than males (Furukawa et al., 2003; Australian Institute of
Health and Welfare, 2009; Department of Human Services, 2001; Slade et al., 2011). This provides some indication that Deaf men and women construct their knowledge of SMI in a similar fashion and that such construction is culturally, rather than gender bound. However, investigation in the current study was limited to establishing cultural and linguistic equivalence, item and scalar equivalence and internal reliability. Other forms including construct and concurrent validity were not investigated, study is required that incorporates a larger sample and clinical norms.
Technology, Innovation and the Future of Deaf Mental Health Assessments
The current study combines sign language technology with clinical assessment to progress the field of International Deafness and Mental Health research. The K10-Auslan applies SYSTEMATIC ERROR 27 technological innovation to clinical scale design to address systematic error in the detection of
SMI in the signing Deaf by using video interfaces to support Auslan clips as the response modality. Results of the present study investigating Deaf participants’ responses showed that there was overwhelming support for use of online webhosting and GIF technology in the presentation of mental health assessments. This corroborates previous research conducted by
Deaf researchers highlighting the benefit of online formats for psychological scales given that sign languages are visuo-spatial in nature and that Deaf people tend to prefer access to online content (Rogers et al., 2012). A final recommendation of this paper is that Deaf-friendly mental health clinical assessments be freely available online with accompanying mental health resource information presented in signed languages.
SYSTEMATIC ERROR 28
References
Angelelli, C., & Jacobson, H. (2009). Testing and assessment in translation and interpreting
studies: A call for dialgogue between research and practice. Amsterdam: John Benjamins
Publishing Company.
Athale, N., Aldridge, A., Malcarene, V., Nakaji, M., Samady, W., & Sadler, G. (2010). Validity of
the multi-dimensional health locos of control scales in American Sign Language. Journal
of Health Psychology, 15(7), 1064-1074. doi: 10.1177/1359105309360427
Australian Bureau of Statistics. (2001). 2001 National health survey: Summary of results.
Canberra: Australian Bureau of Statistics.
Australian Institute of Health and Welfare. (2009). Measuring the social and emotional
wellbeing of Aboriginal and Torres Strait Islander peoples. Canberra: AIHW.
Bauman, H. (2004). Audism: Exploring the metaphysics of oppression. Journal of Deaf Studies
and Deaf Education, 9(2), 239-246. doi: 10.1093/deafed/enh025
Black, P., & Glickman, N. (2006). Demographics, psychiatric diagnoses, and other
characteristics of North American Deaf and hard-of-hearing inpatients. Journal of Deaf
Studies and Deaf Education, 11(3), 303-321. doi: 10.1093/deafed/enj042
Branson, J., & Miller, D. (2002). Damned for their difference: The cultural construction of Deaf
people as disabled: A sociological history. Washington DC: Gallaudet University Press.
Brigman, G., MacPherson, B., Rako, M., Campbell, J., V, M., & Norman-Kelly, T. (2000). A
national epidemiological survey of mental illness in the New Zealand Deaf community.
In T. Hjortsö, L. von der Lieth, C. Carlsen, & (Eds), Mental health services for deaf
people: a worldwide perspective (pp. 216-234). Devon, UK: European Society for Mental
Health and Deafness. SYSTEMATIC ERROR 29
Brown, M., & Cormes, A. (2015). Mental health of deaf and hard-of-hearing adolescents: What
the students say. Deaf Studies and Deaf Education, 20(1), 75-81.
doi:10.1093/deafed/enu031
Connolly, C. M., Rose, J., & Austen, S. (2006). Identifying and assessing depression in
prelingually Deaf people: A literature review. American Annals of the Deaf, 151(1), 49-
60. Doi: 10.1093/deafed/enw061
Crowe, T. (2002). Translation of the Rosenberg self-esteem scale into American Sign Language:
A principal components analysis. Social Work Research, 26(1), 57-63. doi:
/10.1093/swr/26.1.57 de Graaf, R., & Bijl, R. (2002). Determinants of mental distress in adults with a severe auditory
impairment: Differences between prelingual and postlingual deafness. Psychosomatic
Medicine, 40, 61-70.
Deaf Australia. (2013, 01 01). Deaf Australia home page. Retrieved 01 13, 2017, from Deaf
Australia: deafaustralia.org.au
Department of Human Services. (2001). Victorian population health survey. Melbourne:
Victorian Department of Human Services.
Faul, F., Erdfelder, E., Buchner, A., & Lang, A-G. (2009). Statistical power analyses using
G*Power 3.1: Tests for correlation and regression analysis. Behavior Research Methods,
41, 1149-1160. doi: 10.3758/BRM.41.4.1149
Fellinger, J., Holzinger, D., & Pollard, R. (2012). Mental health of deaf people. Lancet, 379,
1037-44. Doi: 10.1016/S0140-6736(11)61143-4 SYSTEMATIC ERROR 30
Fellinger, J., Holzinger, D., Dobner, U., Gerich, J., Lehner, R., Lenz, G., & Goldberg, D. (2005).
Mental distress and quality of life in a deaf population. Social Psychiatry and Psychiatric
Epidemiology, 40, 737-742. doi: 10.1007/s00127-005-0936-8
Ferndale, D., Munro, L., & Watson, B. (2016). A discourse of "abnormality": Exploring
discussions of people living in Australia with deafness or hearing loss. American Annals
of the Deaf, 483-495.
Ferndale, D., Watson, B., & Munro, L. (2016). An exploration of how health care providers
understand experiences of deafness. Critical Public Health, 1-13.
Furukawa, T., Kessler, R., Slade, T., & Andrews, G. (2003). The performance of the K6 and K10
screening scales for psychological distress in the Australian National Survey of Mental
Health and Well-being. Psychological Medicine, 33, 357-362. doi: 10.1002/mpr.351
Harkness, J., Pennell, B., Villar, A., Gebler, N., Aguilar-Gaxiola, S., & Bilgen, I. (2008).
Translation procedures and translation assessment in the World Mental Health Survey
initiative. In R. Kessler, & T. (. Ustun, The WHO World Mental Health Surveys: Global
Perspectives on the Epidemiology of Mental Disorders (pp. 91-113). Cambridge:
Cambridge University Press.
Harris, R., Holmes, H. M., & Mertens, D. M. (2009). Research ethics in sign language
communities. Sign Language Studies, 9(2), 104-131. doi: 10.1353/sls.0.0011
Health Department of Western Australia. (2001). Health and Wellbeing Survey 2000:
Psychological distress in the Western Australian population. Western Australia: Health
Department of Western Australia.
IBM Corp. (2017). IBM Statistics for Windows, Version 25. Armonk: NY: IBM Corp. SYSTEMATIC ERROR 31
Johnston, T. (2004). W(h)ither the Deaf community? Population, genetics and the future of
Australian Sign Language. American Annals of the Deaf, 148(5), 358-375. Doi:
10.1353/aad.2004.0004
Kessler, R., Baker, P., Colpe, L., Epstein, J., Gfroerer, J., Hiripi, E., Zaslavsky, A. (2003).
Screening for serious mental illness in the general population. Archives of General
Psychiatry, 60(2), 184-189. doi: 10.1001/archpsyc.60.2.184
Ladd, P. (2003). Understanding Deaf culture: In search of Deafhood. Clevedon, UK:
Multilingual Matters Ltd.
Landsberger, S., Diaz, D., Spring, N., Sheward, J., & Sculley, C. (2014). Psychiatric diagnoses
and psychosocial needs of outpatient deaf children and adolescents. Child Psychiatry
Human Development, 45, 42-45. doi: 10.1007/s10578-013-0375-9
Landsberger, S., Sajid, A., Schmelkin, L., Diaz, D., & Weiler, C. (2013). Assessment and
treatment of deaf adults with psychiatric disorders: A review of the literature for
practitioners. Journal of Psychiatric Practice, 19(2), 87-97. doi:
10.1097/01.pra.0000428555.48588.f9
Lane, H. (1993a). The mask of benevolence. New York: Random House.
Lillee, A., Thambiran, A., & Laugharne, J. (2015). Evaluating the mental health of recently
arrived refugee adults in Western Australia. Journal of Public Mental Health, 14(2), 56-
68. doi: 10.1108/JPMH-05-2013-0033
Mason, T. (2005). Cross-cultural instrument translation: Assessment, translation and statistical
applications. Amercian Annals of the Deaf, 150(1), 67-72. doi: 10.1353/aad.2005.0020 SYSTEMATIC ERROR 32
Munro, L., & Rodwell, J. (2009). Validation of an Australian Sign Language instrument of
outcome measurement for adults in mental health settings. Australian and New Zealand
Journal of Psychiatry, 43, 332-339.
Nagel, T., Robinson, G., Condon, J., & Trauer, T. (2009). Approach to treatment of mental illness
and substance dependence in remote Indigenous communities: Results of a mixed
methods study. Australian Journal of Rural Health, 17, 174-182. doi: 10.1111/j.1440-
1584.2009.01060.x.
Napier, J., & Kidd, M. (2013). English literacy as a barrier to health care information for deaf
people who use Auslan. Australian Family Physician, 42(12), 896-899.
Nida. (1993). Language, culture and translating. Shanghai: Shanghai Foreign Lannguage
Education Press.
Obasi, C. (2008). Seeing the Deaf in "Deafness". Journal of Deaf Studies and Deaf Education,
13(4), 455-465. doi: 10.1093/deafed/enn008
Øhre, B., Saltnes, H., von Tetzchner, S., & Falkum, E. (2014). Psychometric properties of a sign
language version of the Mini International Neuropsychiatric Interview (MINI). BMC
Psychiatry, 14(148), 1-10. doi: 10.1037/pas0000541
Pollard, R. (2002). Ethical conduct in research involving deaf people. In V. Gutman, & (Ed),
Ethics in mental health and deafness (pp. 162-178). Washington DC: Gallaudet
University Press.
Ridgeway, S. (1997). Deaf people and psychological health - some preliminary findings. Deaf
Worlds, 13, 9-18.
Rogers, K., Evans, C., Campbell, M., Young, A., & Lovell, K. (2014). The reliability of British
Sign Language and English versions of the Clinical Outcomes in Routine Evaluation - SYSTEMATIC ERROR 33
Outcome Measure with d/Deaf populations in the UK: An initial study. Health and Social
Care in the Community, 22(3), 278-289. doi: 10.1111/hsc.12078
Rogers, K., Pilling, M., Davies, L., Belk, R., Nassimi-Green, C., & Young, A. (2016).
Translation, validity and reliability of the British Sign Language (BSL) version of the
EQ-5D-5L. Quality of Life Research, 25, 1825-1834. doi: 10.1007/s11136-016-1235-4
Rogers, K., Young, A., Lovell, K., & Evans, C. (2012). The challenges of translating the Clinical
Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) into British Sign
Language. Journal of Deaf Studies and Deaf Education, 18(3), 287-298. doi:
10.1093/deafed/ent002
Rogers, K., Young, A., Lovell, K., Campbell, M., Scott, P., & Kendal, S. (2012). The British Sign
Language Versions of the Patient Health Questionaire, the Generalised Anxiety Disorder
7-Item Scale, and the Work and Social Adjustment Scale. Journal of Deaf Studies and
Deaf Education, 18(1), 110-122. doi: 10.1093/deafed/ens040
Slade, T., Grove, R., & Burgess, P. (2011). Kessler Psychological Distress Scale: Normative data
from the 2007 Australian National Survey of Mental Health and Wellbeing. Australian
and New Zealand Journal of Psychiatry, 45, 308-316. doi:
10.3109/00048674.2010.543653
Stolk, Y., Kaplan, I., & Szwarc, J. (2014). Clinical use of the Kessler psychological distress
scales with culturally diverse groups. International Jounral of Methods in Psychiatric
Research, 23(2), 161-183. doi: 10.1002/mpr.1426
Tavakil, M., & Dennick, R. (2011). Making sense of cronbach's alpha. International Journal of
Medical Education, 2, 53-55. doi: 10.5116/ijme.4dfb.8dfd SYSTEMATIC ERROR 34
Terry, D., Le, Q., & Nguyen, H. (2016). Moving forward with dignity: Exploring health
awareness in an isolated Deaf community of Australia. Disability and Health Journal, 9,
281-288. doi: 10.1016/j.dhjo.2015.11.002
The State of Queensland. (2016). Guidelines for working with people who are Deaf or hard of
hearing (Version 3). Brisbane: The State of Queensland (Metro South Hospital and
Health Service) Metro South Addiction and Mental Health Services Deafness and Mental
Health Statewide Consultation and Liaison Service.
SYSTEMATIC ERROR 35
Appendix A
K10 Self Administration Q1
The following questions ask about how you have been feeling during the past 30 days. For each question, please circle the number that best describes how often you had this feeling.
All Most Some A little None Q1. During that month, how often did you of the of the of the of the of the feel … time time time time time a. … tired out for no good reason? 1 2 3 4 5
b. …nervous? 1 2 3 4 5
c. …so nervous that nothing could calm 1 2 3 4 5 you down?
d. …hopeless? 1 2 3 4 5
e. …restless or fidgety? 1 2 3 4 5
f. …so restless that you could not sit still? 1 2 3 4 5
g. …depressed? 1 2 3 4 5
h. …so depressed that nothing could cheer 1 2 3 4 5 you up?
i. …that everything was an effort? 1 2 3 4 5
j. …worthless? 1 2 3 4 5
SYSTEMATIC ERROR 36
Appendix B
K6 Self Administration Q1
The following questions ask about how you have been feeling during the past 30 days. For each question, please circle the number that best describes how often you had this feeling.
All Most Some A little None During the past 30 days, about how of the of the of the of the of the often did you feel … time time time time time …nervous? 1 2 3 4 5
…hopeless? 1 2 3 4 5
…restless or fidgety? 1 2 3 4 5
…so depressed that nothing could cheer 1 2 3 4 5 you up?
…that everything was an effort? 1 2 3 4 5
…worthless? 1 2 3 4 5