Deaf Adults' Health Literacy and Access to Health Information: Protocol for a Multicenter Mixed Methods Study

JMIR RESEARCH PROTOCOLS McKee et al

Protocol Deaf Adults' Health Literacy and Access to Health Information: Protocol for a Multicenter Mixed Methods Study

Michael M McKee1, MD, MPH; Peter C Hauser2, PhD; Sara Champlin3, PhD; Michael Paasche-Orlow4, MD, MPH, MA; Kelley Wyse1, MA; Jessica Cuculick5, EdD; Lorraine R Buis1, PhD; Melissa Plegue1, MA; Ananda Sen1, PhD; Michael D Fetters1, MD, MPH, MA 1Department of Family Medicine, University of Michigan, Ann Arbor, MI, United States 2Department of American Sign Language and Interpreter Education, Rochester Institute of Technology, Rochester, NY, United States 3Mayborn School of Journalism, University of North Texas, Denton, TX, United States 4Section of General Internal Medicine, Boston University, Boston, MA, United States 5Center on Cognition and Language, National Technical Institute for Deaf, Rochester Institute of Technology, Rochester, NY, United States

Corresponding Author: Michael M McKee, MD, MPH Department of Family Medicine University of Michigan 1018 Fuller Street Ann Arbor, MI, 48104 United States Phone: 1 734 998 7335 Email: [email protected]

Abstract

Background: Deaf American Sign Language (ASL) users often struggle with limited health literacy compared with their hearing peers. However, the mechanisms driving limited health literacy and how this may impact access to and understanding of health information for Deaf individuals have not been determined. Deaf individuals are more likely than hearing individuals to use the internet, yet they continue to report significant barriers to health information. This study presents an opportunity to identify key targets that impact information access for a largely marginalized population. Objective: This study aims to elucidate the role of information marginalization on health literacy in Deaf ASL users and to better understand the mechanisms of health literacy in this population for the purpose of identifying viable targets for future health literacy interventions. Methods: This is an exploratory mixed methods study to identify predictors and moderators of health literacy in the Deaf population. These predictors of health literacy will be used to inform the second step that qualitatively explains the findings, including how Deaf individuals access and understand Web-based health information. Multiple interviewer- and computer-based instruments underwent translation and adaptation, from English to ASL, to make them accessible for the Deaf participants in our study. A planned sample of 450 Deaf ASL users and 450 hearing native English speakers, aged 18 to 70 years, will be recruited from 3 partnering sites: Rochester, NY; Flint, MI; and Chicago, IL. These individuals will participate in a single data collection visit. A subset of participants (approximately 30) with key characteristics of interest will be invited for a second data collection visit to observe and inquire more about their ability to directly access, navigate, and comprehend Web-based health information. The study will help assess how the ways health literacy and information are visualized may differ between Deaf individuals and hearing individuals. The study will also survey participants' ownership and use of computer and mobile devices and their level of Web-based information use, including health information. Results: Adaptation and translation of protocols and instruments have been completed and are now in use for the study. Recruitment is underway and will continue until late 2020. Results from this study will be used to provide a guide on how to structure Web-based health information in a way that maximizes accessibility and improves health literacy for Deaf individuals. Conclusions: The results from this mixed methods proposal will advance what is known about health literacy and health information accessibility for Deaf individuals. This innovative study will generate rich data on how to formulate health information and health literacy interventions more accurately to take advantage of visual learning skills.

https://www.researchprotocols.org/2019/10/e14889 JMIR Res Protoc 2019 | vol. 8 | iss. 10 | e14889 | p. 1 (page number not for citation purposes) XSL·FO RenderX JMIR RESEARCH PROTOCOLS McKee et al

International Registered Report Identifier (IRRID): PRR1-10.2196/14889

(JMIR Res Protoc 2019;8(10):e14889) doi: 10.2196/14889

KEYWORDS deaf; hearing loss; consumer health information; health literacy

self-management of health care needs in these individuals. What Introduction is needed is to understand how to design technology and Background Web-based health information in a way that may benefit these types of consumers [18,19]. As Deaf individuals communicate Deaf American Sign Language (ASL) users are nearly 7 times through a visual language, the proposed study provides an more likely than their hearing peers to have inadequate health opportunity to determine optimal visual-based information literacy [1]. Deaf ASL users (henceforth, Deaf) rarely receive sources, providing another avenue to help those with lower language concordant health care services and are at highest risk health literacy. Individuals with low health literacy struggle in for miscommunication with their health care providers. Deaf locating relevant Web-based health information and may fixate individuals understand less than 30% of what is being said on irrelevant aspects of displayed information [20]. Such a through lipreading [2,3]. Furthermore, prose literacy poses a phenomenon may be important for populations such as the Deaf challenge for Deaf individuals; the average Deaf individual who are dependent on visual mechanisms for communication reads English at the fifth- to sixth-grade level [1,4-7]. The use and for overcoming information gaps. of interpreters is the standard of care for Deaf ASL users, yet they are infrequently provided [8]. The majority of Deaf This study describes the methods used to study the role of individuals (approximately 95%) have hearing family members information marginalization on health literacy for Deaf who do not sign. Thus, many experience the dinner table individuals. The authors describe approaches for instrument syndrome, where they have encountered years at the dinner adaptations and measures applicable for Deaf individuals who table watching close family members and friends converse with use ASL. Finally, a description of the unique aspects of our each other while being unable to understand what is being said, proposed data collection method is shared in this paper. The depriving them of incidental learning opportunities that many research objectives are two-fold: (1) to elucidate the role of hearing individuals take for granted [9]. The loss of incidental information marginalization on health literacy in Deaf signers learning opportunities and information marginalization for Deaf and (2) to better understand the mechanisms of health literacy individuals occurs daily in a broad range of work, schools, in this population for the purpose of identifying viable targets friends, families, government, media, and health care contexts for future health literacy intervention development. [1]. Many Deaf ASL users learn language, health information, Study's Theoretical Framework and even culture via peers rather than family [10] and struggle to identify and correct misinformation [11]. Due to this There are few mechanism-based studies in health literacy communication-depleted milieu, inadequate health literacy may research and none involving Deaf and hard of hearing be an important cause of the lower level of health-related individuals. A review of the literature helped with developing knowledge and worse health outcomes that have been observed ideal conceptual models to guide the project's proposal. The among Deaf individuals. authors modified and created a hybrid of 3 widely used health literacy and diffusion theory conceptual models: (1) the attitudes, Deaf individuals, because of their social and language skills, and knowledge (ASK) model [21], (2) the health literacy marginalization, appear to use Web-based health information model [22], and (3) the diffusion theory [23] for Deaf more frequently than the general population [12,13]. However, individuals. Figure 1 illustrates constructs that are relevant to it is unclear if Deaf individuals access Web-based health health literacy and health-related information. information or other information sources effectively and what Sociodemographics, cognition, educational achievement degree of impact this may have on their health literacy. In the including reading skills and task performances, and general general population, there is evidence that health literacy impacts health knowledge have all been shown to be related to each mortality, yet the connection remains inconsistent when looking other with respect to health literacy in the general population at other clinical outcomes [14]. In multiple studies, positive [21]. Additional aspects of health literacy that need to be better associations with both health literacy and electronic health understood in the Deaf community include the role of attitudes (eHealth) literacy and improved quality of life, better use of (ie, the feeling and trust one has toward information), skills (ie, health care services, and improved health behaviors among the tools that allow one to seek, obtain, and understand individuals with different types of health conditions were information), and knowledge (ie, demonstrates comprehension). demonstrated [15-17]. An area of concern is that disadvantaged It is hypothesized that the factors outlined in the model are key populations, notably those with chronic health conditions or elements of health literacy for Deaf individuals. As a result, our disabilities, may struggle with eHealth literacy. This may Deaf Health Literacy conceptual model regarding information promote additional inequalities in their ability for Web-based includes 3 main concepts that lead to health outcomes and are health information to influence positive outcomes such as being used in this study to inform data collection and analysis.

https://www.researchprotocols.org/2019/10/e14889 JMIR Res Protoc 2019 | vol. 8 | iss. 10 | e14889 | p. 2 (page number not for citation purposes) XSL·FO RenderX JMIR RESEARCH PROTOCOLS McKee et al

Figure 1. Deaf health literacy conceptual model for information.

To help explain how and why certain characteristics may impact Hurley Medical Center (HMC). The study is also registered at health literacy, including ability to access, navigate, and the ClinicalTrials.gov under ID NCT03093779. understand Web-based health information, a qualitative interview along with thematic analysis will be performed. This Study Design will explain the interrelation in outcomes of interest (eg, what The study uses an explanatory sequential mixed methods design characteristics are associated with adequate health literacy), [24] with extensive quantitative data collection procedures to intermediary outcomes (eg, what backgrounds, skills, and identify predictors and moderators of health literacy. We will contextual factors allow for better navigational ability on the first conduct a cross-sectional study to collect predictors of Web), and personal conditions that need to be explored further health literacy. These predictors of health literacy will be used (eg, cognitive issues). to inform the subsequent phase of qualitative assessment. For example, elicitation interviews with selected participants Methods returning for the second data collection visit will be used to help explain the quantitative results and to help elucidate how Deaf Overview individuals access and understand Web-based health For this study, there are 3 proposed aims. Aim 1 is to evaluate information. The study will use a stratified sampling approach differences in ASK, regarding health information for both Deaf to invite participants of certain characteristics to return for signers and hearing English speakers. Aim 2 plans to assess further data collection. Characteristics of interest will be based hearing status as an effect modifier of the association between on quantitative analyses that show significant associations health literacy and ASK with health information. Aim 3 will between health literacy and any of the measured variables (eg, qualitatively assess the impact of health information accessibility educational attainment, age, or reading literacy). Approximately and patterns of use on health literacy with one-on-one elicitation 15 Deaf and 15 hearing participants with inadequate health interviews. This will be based on actual Web-based health literacy and another 15 Deaf and 15 hearing participants with information searches for 4 clinical vignettes using an observed adequate health literacy will return for a second visit to conduct task fulfillment experiment as a prompt. The goal of this aim a task performance on a desktop computer and participate in a is to explore differences with Web-based health information follow-up interview designed to learn more about their decision search ability and health knowledge acquisition in a subsample making and preferences. The return visit will involve a of Deaf and hearing individuals recruited from aims 1 and 2. qualitative assessment using elicitation interviews to help The University of Michigan institutional review board (IRB) explain any of the quantitative results. The goal is to learn how approved the study and provides coordination with the following and why Deaf individuals access and understand health IRBs: Sinai Hospital, Rochester Insitute of Technology, and information (Figure 2) and how this may differ from hearing individuals.

https://www.researchprotocols.org/2019/10/e14889 JMIR Res Protoc 2019 | vol. 8 | iss. 10 | e14889 | p. 3 (page number not for citation purposes) XSL·FO RenderX JMIR RESEARCH PROTOCOLS McKee et al

Figure 2. Overall study design. ASL-NVS: American sign language-newest vital sign.

The project uses a community advisory board (CAB) with identify themselves as Deaf and communicate primarily in ASL. representation from study sites. The CAB is designed to provide For the hearing comparison group, the hearing study participants community oversight and guidance on the research protocol also must be aged at least 18 years but self-report normal hearing development and interpretation of findings. The board consists and communicate in spoken English (ie, English monolinguals). of 6 Deaf community members, 2 from each of the research Participants will be excluded if they have any cognitive data collection sites. impairments (eg, because of dementia, delirium, or intoxication), inability to consent to the study, or limited vision. Study Setting and Participants The study will recruit 450 Deaf and 450 hearing participants at Sample Size 3 primary data collection sites. The sites include HMC at Flint, To detect differences between correlations at 1% level of MI (selected because of its diverse Deaf patient population, significance with 80% power, based on a one-sided Z-test of along with previous research collaborations with the University comparison of Fisher Z-transformed correlations, we plan to of Michigan); The National Technical Institute of the Deaf recruit 450 participants in each group that will allow us to detect (NTID), which is housed at the Rochester Institute of a minimum difference of 20%. The sample size is more than Technology in Rochester, NY (selected because of NTID's adequate in detecting the differences between the groups with experience in research and Rochester's highly educated Deaf respect to the accessibility, acquisition, and attitude measures community); and The Sinai Deaf Health program in Chicago, in aim 1 as well as to assess deafness as an effect modifier of IL (selected because of its long-standing commitment to Deaf the association between health literacy and ASK with health health education and research and its minority community). information. Recruitment Measures Recruitment occurs at each site through the use of flyers, email The constructs outlined in the Deaf Health Literacy conceptual listserves, social media postings and vlogs (video-based blogs), model will be measured through a collection of chosen tools and community outreach. Once a potential participant is deemed (see Table 1). These tools were selected primarily because of eligible, a research staff educates him or her on the study, and their accessibility (prior use with past projects involving Deaf if interested in proceeding, an in-person research visit is participants) and their key constructs that they measure. There scheduled. The informed consent is reviewed and signed at the were several tools that required translation and adaptation before in-person visit before proceeding with the study. The research they could be used in the study. A translation work group staff who worked with the Deaf participants are fluent in ASL. (TWG) was formed for this purpose. ASL is a language with its own rules of semantics and syntax similar to other languages. Eligibility Criteria To be eligible, study participants must be aged at least 18 years. For the Deaf study group, eligible Deaf participants must

https://www.researchprotocols.org/2019/10/e14889 JMIR Res Protoc 2019 | vol. 8 | iss. 10 | e14889 | p. 4 (page number not for citation purposes) XSL·FO RenderX JMIR RESEARCH PROTOCOLS McKee et al

Table 1. Key variables and their measures. Key variables Measures Defintion Attitudes Self-efficacy One's belief in one's ability to succeed in specific situa- Self-efficacy for information seeking [26]. tions. Skills Cognition Ability to process information, metacognition, and behav- Kaufman Brief Intelligence Test, Second Edition (assesses ioral regulation, which all are important for learning and nonverbal cognition), and the Color Trails Test (assesses social behavior executive function) [27] provide standard scores. Both have been used extensively in Deaf populations [28,29]. Reading literacy Level of reading proficiency reading grade level from the Test of Silent Contextual Reading Fluency, Second Edition [30]. This instrument is visual and provides a suitable tool for both Deaf and hearing populations. Electronic health Ability to seek out, find, evaluate and appraise, integrate, eHealth literacy Scale (validated measure of Web-based and (eHealth) literacy and apply what is obtained in an electronic or Web-based eHealth literacy) [31]. Adapted further for the Deaf ASLa environment to solving a health problem. users that includes whether Web-based information is accessible in ASL. Visualization of health Visualization patterns of the presented health information. Actual through eye tracking software, which assesses the information visualization of health-related information on 4 health topics [20,32]. Knowledge Health knowledge Knowledge of cardiovascular health. Wagner et al's cardiovascular health knowledge measure [33].

Language fluency Level of expressive and receptive language proficiency. (1) ASL-SRTb [34] and (2) Speaking Grammar Subtest of the Test of Adolescent and Adult Language, Third Edition [35]. Both will assess reception and expressive fluency in both ASL and spoken English. The ASL-SRT has been used extensively in Deaf populations [36,37].

aASL: American Sign Language. bASL-SRT: American Sign Language Sentence Reproduction Test. The shape, placement, and movement of hands as well as facial expertise within their content area or translation work. expressions and body movements all play important parts in Moreover, the team members have Deaf cultural issues as they conveying important information. There is no written form of are members of the Deaf community. The TWG members, ASL, so the use of ASL gloss is a way to keep note of how the including Deaf and hearing individuals, worked collaboratively ASL concepts should be signed to convey the meaning of the in small groups to prepare, review, and perform quality checks. question or instructions accurately. Furthermore, the use of In addition to multiple intrateam reviews during the translation gloss has to do with the fact that the target language may not and filming process (videos used for internal reviews and have equivalent words to represent the original language. The back-translation steps), external reviewers, who were not majority of the instruments were already available in ASL before members of the translation team, provided an added layer of the study's funding, but there was translation and adaptation review and validation to ensure the quality and integrity of the work needed for the following instruments: eHealth literacy work. A final blind back-translation from the ASL videos back Scale (eHEALS; 8 questions), Pew Research Center's Health to the English text was done by 2 bilingual Deaf experts (not Online Survey questions related to online technology and involved with the TWG) to ensure translation accuracy. The information use (41 questions) [25], certain sociodemographic above steps, including selection of the team, were done to questions, cardiovascular knowledge assessment (25 questions), establish appropriate language and cultural translations. and self-efficacy instrument (3 questions). Health Literacy Assessment The TWG consisted of the principal investigator (PI), a faculty The study will use an accessible and validated health literacy experienced in translation of research instruments for Deaf instrument for Deaf individuals that the PI derived from the individuals, 1 ASL interpreter skilled in medical terminology, Newest Vital Sign (referred to as the ASL-NVS) [38]. The and 3 native Deaf signers experienced with translation work. ASL-NVS also offers other benefits as it assesses numeracy This translational workgroup approach was successful in the and reading literacy, has a short administration time with only adaption of prior measures (eg, American Sign 6 questions, and has been created and validated for other Language-Newest Vital Sign, ASL-NVS) [1]. All TWG team languages (eg, Spanish). The ASL-NVS is the first, and currently members are bilingually fluent in English and ASL, including

https://www.researchprotocols.org/2019/10/e14889 JMIR Res Protoc 2019 | vol. 8 | iss. 10 | e14889 | p. 5 (page number not for citation purposes) XSL·FO RenderX JMIR RESEARCH PROTOCOLS McKee et al

only, health literacy instrument available for the Deaf Health Information Evaluation Skill population. The instrument is available online [39]. Actual behaviors will be assessed for each participant through Attitudes Assessment the use of the eye tracking software (Tobii; Tobii Technology) on 4 standardized health topic pages (2 with a picture and 2 Self-Efficacy without a picture). Eye tracking technology will be used to The Self-efficacy for Information Seeking scale is based on a observe their ability to visualize and understand standardized scaled 3-item questionnaire assessing beliefs about ability to health information with and without pictures. For eye tracking, effectively perform specific information-related behaviors, such we will use the Tobii eye tracker and software for data collection as finding information about health, evaluating the accuracy of on fixation duration and fixation count on the presented health information, and using the internet to find information information and the visualization pattern on both websites. This [26]. eye tracker provides extremely detailed eye tracking metrics (in milliseconds), including fixation counts, visit duration, time Skills Assessments to first fixation, and percentage of viewing time fixated/clicked Nonverbal Cognitive Ability on a certain area of interest. It also can be used in either static (eg, print) or dynamic (eg, video or website scrolling) to assess The Kaufman Brief Intelligence Test, Second Edition (KBIT2) each participant's areas of interest. [40], Matrices Subtest (nonverbal intelligence quotient [IQ]) is computed based on one's performance on a single Knowledge Assessments multiple-choice subtest that does not require language skills. There are 46 items composed of several types of items involving Health Knowledge visual stimuli, both meaningful (people and objects) and abstract A validated, general health knowledge test called the Heart (designs and symbols). All items require understanding of Disease Fact Questionnaire [33] will be used. It assesses relationships among the stimuli and require the participant to knowledge of cardiovascular health in a true/false/do not know point to the correct response. The KBIT2 nonverbal IQ has a questionnaire with 25 questions. There is a paucity of general 0.82 correlation with the Wechsler Adult Intelligence Scale, health knowledge assessments that are validated. This tool is Third Edition, nonverbal composite [40]. This test has recently one of the few that provides a range of topics to assess become commonly used in cognitive science and participants' knowledge and has been used in a previous study psycholinguistic studies that have included Deaf participants involving Deaf individuals. [21,22,29] to control for individual cognitive differences. Language Fluency Executive Function Ability As Deaf individuals reside in communities that use English as The Color Trails Test (CTT) [27] is a measure of executive the lingua franca (or de facto language), they are frequently function abilities where participants are asked to rapidly connect bilingual. Fluency will be assessed in both ASL and English numbered circles in sequence by alternating between pink and for the Deaf participants. English will be assessed for the hearing yellow circles [27]. The time to complete the task is recorded. participants. ASL fluency will be tested with the American Sign The CTT has a test-retest reliability of 0.79 and has good Language Sentence Reproduction Test (ASL-SRT) [34], which factorial and criterion-related validity [27] with individuals with is a brief global measure of individuals'receptive and expressive frontal lobe injuries. One of the coinvestigators (PCH) has used ASL skills. Participants are required to watch videos of 20 ASL this test in cognitive science studies involving Deaf participants sentences and correctly reproduce the sentences after each one [28,29]. is presented. Correct reproductions are awarded 1 point, and the maximum total score is 20. ASL-SRT has been used in other Reading Literacy cognitive science and psycholinguistic studies [36,37] and has This will be assessed through the Test of Silent Contextual been adapted to other signed languages. The ASL-SRT interrater Reading Fluency, Second Edition [30]. This tool provides an reliability coefficient is 0.83, with an internal consistency efficient standardized literacy assessment for both Deaf and (Cronbach alpha) of .88. CConstruct validity was established hearing participants. by illustrating that Deaf adults perform better on this test than Electronic Health Literacy Deaf children (P<.001; partial eta sq=.042) and native signers perform better than nonnative signers (P<.001; partial eta sq= eHEALS was adapted to make it more culturally and 274). An equivalent and parallel test is available that assesses linguistically appropriate for Deaf ASL users. The 8-item the receptive and expressive spoken English abilities that will questionnaire inquires about the participants' knowledge, be used for hearing participants. The test is called Speaking comfort, and perceived skills at finding, evaluating, and applying Grammar Subtest of the Test of Adolescent and Adult Language, eHealth information to health problems [31]. An additional 41 Third Edition (TOEL3) [35], and has been widely used for questions derived from the Pew Research Center's Health language fluency assessments. The TOEL3 has internal Online Survey [25] related to online technology and information consistency (Cronbach alpha) of .95, with test-retest reliability will assess the participant's level of information access and use. coefficient of 0.80. This also was expanded to inquire about ownership of computer and mobile-based devices. Sociodemographic information will be based on an existing shortened version of the Behavioral Risk Factor Surveillance Systems [41]. This will be used to evaluate key demographic

https://www.researchprotocols.org/2019/10/e14889 JMIR Res Protoc 2019 | vol. 8 | iss. 10 | e14889 | p. 6 (page number not for citation purposes) XSL·FO RenderX JMIR RESEARCH PROTOCOLS McKee et al

information for each participant and adjust for covariates in nature of websites accessed to address the clinical vignettes will health literacy. For Deaf participants, additional questions will be recorded and assessed through the use of the Tobii usability be asked to provide information on the following: (1) presence testing software already available in our facilities [44]. The data of Deaf family members, (2) ownership of a hearing aid or will include video footage of the participant (including eye cochlear implant, (3) hearing status with and without a hearing movements and facial expression), websites visited, duration aid or cochlear implant, (4) onset of hearing status, (5) any of page visited, and number of clicks and cursor activity. The cultural identity with their hearing status, (6) Deaf school use of the Tobii provides several benefits: (1) the tool provides attendance, and (7) socialization preferences based on hearing real-time dual-screen recording and display of both the status (Deaf, hearing, or both). We will also perform a Shoebox participant's actions online and webcam recordings of their Audiometry [42] on all participants to assess their unaided facial expressions and eye movements; (2) the interviewer can hearing levels (HLs) from (−)10 dB to 90 dB at 500 Hz, 1000 watch the actions of the participant on the interviewer's Hz, 2000 Hz, and 4000 Hz for each ear. Shoebox is a leading computer screen through the use of the Tobii Pro Studio tablet-based audiometer, which is a class 2 medical device software program in real time; (3) the interviewer can tag registered and listed with the Food and Drug Administration. different-colored annotations (eg, search queries, websites It offers a clinically validated, self-administered, automated accessed, and challenges encountered) to the video data being testing platform optimized for use outside of a sound booth; recorded, which is visible on the time bar; (4) the tool can can be completed in 5 min or less; and is completely instantaneously compute summary statistics such as the total configurable. HL will be defined by the standard greater than search time in seconds, the number of unique websites and pages or equal to 25 dB average loss across the 4 frequencies in the visited, and the number and characteristics of queries issued for better ear [43]. This will allow us to control for hearing loss the interviewer to discuss with the participant; and (5) the variations among participants, especially because they may interviewer can scroll quickly to the relevant time of the video influence the level of access to information. to help the participant observe and remember their actions but, more importantly, allow for the interviewer to ask more detailed Procedures questions on how and why specific actions were chosen and the Once the participants are eligible, the research team will participant's thought process related to the online information. schedule an in-person visit to review both the informed consent A figure demonstrating the different components and how it and administer the different measures. This was done in the works can be accessed at Tobii website. same fashion at each performance site. The research staff review the consent form and the brief study information directly in the The recordings from the task performance above will serve as participant's preferred language (eg, ASL if Deaf). Once consent a prompt for the elicitation interview. It will help inform the is obtained, the staff then proceeds with the data collection subsequent qualitative assessment to help explain the protocol. The measures are grouped into the following quantitative results (of aims 1 and 2) and elucidate how and categories: (1) demographics/background information (includes why Deaf individuals access and understand health information. Shoebox Audiometry), (2) interviewer-administered Similarly, the same approach will be conducted with a questionnaires (ie, cardiovascular knowledge assessment, subsample of hearing individuals. This will be done to provide self-efficacy, Web-based health information use, and e-literacy), robust comparisons. Video-based and semistructured qualitative (3) computer-administered measures (ASL-SRT, TOEL-3, and assessments will be used to determine both how and why Deaf the ASL-NVS), (4) booklet testing (ie, CTT, KBIT2, and and hearing participants of different health literacy strata TOSCRF-2), and (5) Tobii eye tracker experiment. To avoid (inadequate and adequate; n=15 each) may differ in their any ordering bias, the categories (except for the performances with Web-based health information searches. The demographic/background information) are randomized to each findings here will be used to help explain results from aims 1 participant (see Multimedia Appendix 1). The and 2. The elicitation interview questions will be generated computer-generated randomization was completed before any initially from the findings in aims 1 and 2 and complemented participant recruitment. The counterbalance checklist designed with additional questions following completion of the task with 4 blocks (A, B, C, and D) was randomly assigned to each performance on a computer. The interviewer will playback the participant (eg, 1 participant may get a checklist C that starts dual-screen recording of the participant's actions to facilitate with booklet testing). recall of his or her own actions and thoughts with each task undertaken. The interviewer will encourage the participant to Procedures for Return Visit (for Those to Be Invited) make his/her thoughts transparent, that is, telling how and why There will be 2 parts when the participant returns for their the choices were made in a narrative format. The dual-screen second visit (see Table 2). The first step will observe how recording with performance metrics will serve as stimulus participants search and acquire Web-based health information. material for the video elicitation interview. As the dialog This task performance will include 4 brief clinical vignettes (ie, between the interviewer and the participant will be in ASL pneumonia, deep vein thrombosis, migraines, and appendicitis) (visual language), a separate external video camera will be used and multiple-choice answers will serve as a prompt. This will to record this interaction for later transcription. The interview be given to all invited aim 3 participants to examine their will reveal the complex cognitive and decision-making processes abilities to search for health information on the Web and acquire that may occur with each individual, which are not adequately health knowledge. Participants' query formulations, navigation revealed in earlier statistical analysis. The incorporation of both patterns, cursor activity, total search times, and number and steps provides better integration of the findings of all 3 aims and generates robust data to guide future intervention

https://www.researchprotocols.org/2019/10/e14889 JMIR Res Protoc 2019 | vol. 8 | iss. 10 | e14889 | p. 7 (page number not for citation purposes) XSL·FO RenderX JMIR RESEARCH PROTOCOLS McKee et al

development to address inadequate health literacy and these task- and user-oriented factors are important in their roles information marginalization for Deaf populations. A strength in the overall eHealth context. The proposed step will help of this step is the ability to explore contextual factors (ie, task- examine how these intrapersonal and system-based factors may and user-oriented factors) that could not be explained with the shape the Web-based health experience and abilities for both first visit. As stated by a newer eHealth literacy model [45], Deaf and hearing individuals.

Table 2. Aim 3 approaches. Definition Purpose Approach Data collection Expected outcomes Web-based health infor- To assess Web-based Use of Tobii video/on- Navigation patterns of Deaf Assess Web-based health information mation navigation and health information line use recording soft- individuals; preferences of search ability of Deaf individuals (un- comprehension (step 1) search ability, acquisi- ware; 4 clinical vi- health websites, including known); determine if Deaf individuals' tion, and patterns gnettes with questions search engines (n=15 Deaf health literacy determines the quality of to prompt Web-based and n=15 hearing for compar- the navigation and comprehension of the search ison); scoring on questions information (unknown); generate areas related to vignettes of weaknesses and strengths for interventions to focus (unknown) Elicitation interviews To explain and expand Use of elicitation inter- Explore how and why partici- Assess how Deaf individuals visualize (step 2) the understanding of the views about video pants decided to use the web- and learn Web-based health information findings demonstrated recordings of the partic- sites they chose, including the (unknown); determine the preferences in aims 1 and 2 ipants and their Web- types of queries and what as- of health information tools and dissemi- based searches pects of the Web-based infor- nation (unknown) and understand why mation was useful to them they have those preferences (unknown)

respectively. The health literacy indicator (ASL-NVS) ranges Statistical Analyses for Aim 1 in value from 0 to 6. To understand whether the association Frequency statistics will be used to summarize categorical data, between health literacy and ASK related to health information whereas means (SDs) were computed to describe continuous differs between Deaf and hearing individuals, we shall compute data. Correlations between main study variables will also be in the Deaf and hearing groups Fisher Z-transformed partial analyzed using Pearson r correlation coefficients. Differences correlations between health literacy and each of the constructs, between the Deaf and hearing participants in terms of ASK adjusted for the socioeconomic factors and potential site related to health information will be assessed using linear differences. Subsequently, 2-sample Z-test of these correlations regression models. To control for the effects of demographic will be carried out for each construct. We shall supplement this and physiological covariates on the dependent variables of ASK analysis with a regression approach where health literacy will related to health information, sociodemographic (eg, race) and be regressed on the group and each of the constructs along with physiological factors (ie, hearing loss severity and laterality) their interaction controlling for the socioeconomic factors and will be entered in model 1 as potential confounding factors site. Although the interaction term is of primary focus, such a affecting each variable of interest. As we are primarily interested model will provide an overall appraisal of the association in the additional effects of ASK related to health information between health literacy and various covariates. On the basis of on health literacy and eHealth, predictor variables will be PI's previous work involving the ASL-NVS, one can subsequently entered in model 2. Differences between the Deaf alternatively characterize health literacy as a categorical variable and hearing participants in terms of ASK related to health with categories of adequate (NVS score of 5-6), indeterminate information will be assessed using linear regression models. (2-4), and inadequate (0-1). As a secondary analysis, we will The scales for each modality described in Table 1 will be used conduct an ordinal logistic regression with the categorical as a separate outcome with Deaf versus hearing as the primary outcome of health literacy using the same covariates in the covariate. A propensity-adjusted analysis will also be conducted model as mentioned above. with the propensities defined as the predicted probability of having adequate health literacy as a function of Analyses for Aim 3 sociodemographic factors estimated through a logistic regression Building on the methodology used in previous Web-based search model. For observational data, the propensity-matched methods behavior studies [46], participants' search skills and success in are often deemed to be a better alternative to the usual multiple addressing the clinical scenario tasks will be assessed using a linear regression controlling for potential confounders. Strata combination of methods. First, based on the browser log and based on propensity quantiles will be used as a categorical interaction data gathered by the usability software, we will covariate in the linear regression model. Furthermore, the compute summary statistics, such as the total time (in minutes) variability across sites will be controlled for in the model. Usual spent in searching, the number of unique websites and pages model diagnostics will be performed, followed by corrective visited, and the number and characteristics of queries issued, actions as necessary. involved in completing each task. Second, we will include time-based retrieval measures [47] that can characterize and Statistical Analyses for Aim 2 quantify how a participant's search progress evolves over time. The ASL-NVS and the English-NVS will be used to assess the Knowledge acquisition will be measured by the number of health literacy score in both Deaf and hearing participants, correct responses based on the clinical vignettes (score range

https://www.researchprotocols.org/2019/10/e14889 JMIR Res Protoc 2019 | vol. 8 | iss. 10 | e14889 | p. 8 (page number not for citation purposes) XSL·FO RenderX JMIR RESEARCH PROTOCOLS McKee et al

between 0 and 12). All measures representing navigation skill interventions that more accurately take advantage of visual and knowledge acquisition will be compared across levels of learning skills, even for those who have normal hearing. The health literacy (adequate vs intermediate/inadequate) on the findings will also be potentially beneficial for website editors. subgroup of the preselected 30 Deaf participants using 2 sample On the basis of the findings, recommendations for improving t tests. Although it is anticipated that power will be low to accessibility to health information for those who are non-English observe statistical significance, the primary objective for this speaking and for those with inadequate health literacy will be comparison is to identify trends. Search progress for the clinical provided. Results will be published in peer-reviewed journals scenarios will be defined in terms of the quality of health once completed. information pages found during the search task. To assess the quality of health information pages, the research team will Discussion evaluate the Web pages/resources returned by participant searches or visited by the participant via browser navigation in Principal Findings terms of accuracy and coverage of the target facts in the This project will advance knowledge on the origins of reference set. Any differences will be discussed until a inadequate health literacy among Deaf individuals, including consensus is obtained. how effectively they access Web-based health information. In Analysis of the Elicitation Interviews this paper, methods developed for the study will help assess the role of information marginalization on health literacy in the For the iterative analysis of the qualitative data obtained with Deaf population. We also describe our approach to adapt the elicitation interview process, we will follow multiple steps: instruments and measures applicable for Deaf ASL users. This (1) thoroughly read the transcribed data, (2) generate initial project is the first to systematically explore health literacy for codes, (3) search for themes and patterns among the codes and Deaf individuals. Most Deaf studies focus on Deaf health across participants, (4) review themes, and (5) define and name knowledge gaps and disparities, but our goal is to provide themes. Initial data queries will focus on questions that emerge tangible recommendations going forward, especially for those from aims 1 and 2 and navigability and knowledge acquisition involved with health information dissemination and website patterns as measured in step 1. As we learn more through the editors. iterative elicitation interview/analysis process, emerging themes and questions will be explored. Video-recording data of the Direct Impact one-on-one interviews will be transcribed into English by a Improvement in translation work and their processes can have bilingual transcriber [10]. Field notes will be taken after each an important impact as it affects other funded research projects encounter to describe the context, primary content, and emerging and those that are being proposed. Furthermore, existing concepts. Participants' data in aim 3 will also be linked with measures (eg, ASL-NVS) will be shared with other investigators the data collected in aims 1 and 2 that helped inform questions interested in working with this population. We hope that the in the elicitation interviews. This will be done through the use measures can be incorporated into community and clinical of joint displays, a state-of-the-art integration procedure in surveillance to help them manage their own health and determine mixed methods studies used for visually matching results by how health information may or may not be reaching these domains and themes from the quantitative and qualitative data individuals. Research with these communities are highly to link the relevant information. All raw data video files will underfunded. This project provides an example of feasible be uploaded onto a secure server, transcribed at the PI's office, research with the Deaf community. In addition, little is known and analyzed by the qualitative team. The use of Dedoose about the predictors of health literacy in Deaf populations and software [48] will allow transcription and detailed annotation how this affects their ability to use Web-based health to be tagged with video data of Deaf signers. All qualitative information. This is key because of their social and information data (transcripts and field notes) will be coded independently marginalization and their documented risk for inadequate health by multiple investigators. Codes will be developed using literacy. Web-based health information, if accessible, can consensus. A codebook will be created initially using navigation provide a strategic approach to address the community's health and knowledge acquisition and will be expanded with emergent knowledge gaps. codes. All transcripts will be coded by the research team (shown above) using the codebook in Dedoose. The results of this mixed methods proposal will significantly advance what is known about health literacy and health Results information accessibility for the Deaf population. This innovative study will generate rich data on how to formulate The project received human participants approval from the health information and health literacy interventions more University of Michigan IRB (HUM00132918) to provide accurately to take advantage of visual learning skills. The study oversight to the 3 other research sites. Data collection will be results will be provided once data collection is complete completed in late 2020. Findings from this study will be used (anticipated in 2020). Once finalized, findings will be used to to advance what is known about health literacy and health develop a white paper on how to structure Web-based health information accessibility for the Deaf population. The findings information in a way that maximizes accessibility and will help explain how Deaf individuals access, navigate, and comprehension for Deaf individuals and also includes strategies comprehend Web-based information. This study will help guide that adhere to universal health literacy precautions because of formulation of health information and health literacy the Deaf population's documented risk of inadequate health literacy.

https://www.researchprotocols.org/2019/10/e14889 JMIR Res Protoc 2019 | vol. 8 | iss. 10 | e14889 | p. 9 (page number not for citation purposes) XSL·FO RenderX JMIR RESEARCH PROTOCOLS McKee et al

Timeline and Challenges required for the study. Furthermore, to avoid participant fatigue It is important to establish adequate time and resources to ensure with multiple measures, we plan to offer refreshments and ample adequate translation and adaptation of instruments needed in opportunity for breaks during the data collection visits. the project. This process took longer than anticipated. It also Conclusions requires a diverse team to ensure good comprehension. We also Research involving Deaf individuals requires careful needed to do 1 additional step and elicit feedback from the consideration of instruments and measures that will not cause advisory board and the staff member at each site to make sure measurement bias, cultural discordance, and inaccessibility there were no issues (eg, regional variations for certain signs). issues. Furthermore, dedicated time for adaptation, translation, It was decided that interviewer-administered questionnaires and, if needed, validation steps should be factored when were preferred because of participant's different levels of calculating the appropriate research time and funds needed to language fluency. Although this did not yield a computerized implement the project. Adequate training time is needed when self-administered tool, it was done to reduce any potential data conducting a multisite study involving several research staff. collection errors or language/cultural discordance with Deaf The results from this mixed methods proposal will advance participants. In addition to the TWG, our research team required what is known about health literacy and health information extensive training to ensure strict protocol adherence, given the accessibility for the Deaf population. This study will also be multiple study measures. In preparation for going into the field, useful to develop best practices in improving the accessibility we conducted both a site-specific workshop and an in-person and usefulness of Web-based health for individuals at risk for workshop with all staff and investigators together to standardize inadequate health literacy. interviews, administration of instruments, and data entry

Acknowledgments This study is funded by the National Institute on Deafness and Other Communication Disorders at the National Institutes of Health (1 R01 DC014703-03). The funding agencies had no role in the study design, data collection, analyses, interpretation of the findings, or decision to submit this manuscript for publication. These contents are solely the responsibility of the authors and do not necessarily reflect the views of funding sources. The authors wish to thank Judy Connelly, Lilly Pritula, and Rania Ajilat for their administrative support and the CAB for their grant assistance and input.

Conflicts of Interest None declared.

Multimedia Appendix 1 Random ordering of measures. [DOCX File 19 KB-Multimedia Appendix 1]

Multimedia Appendix 2 Peer-reviewer report from the National Institutes of Health. [PDF File (Adobe PDF File)233 KB-Multimedia Appendix 2]

References 1. McKee MM, Paasche-Orlow MK, Winters PC, Fiscella K, Zazove P, Sen A, et al. Assessing health literacy in deaf American sign language users. J Health Commun 2015;20(Suppl 2):92-100 [FREE Full text] [doi: 10.1080/10810730.2015.1066468] [Medline: 26513036] 2. The US Government Accountability Office (GAO). 1988. Toward Equality: Education of the Deaf - A Report to the President and the Congress of the United States URL: http://archive.gao.gov/t2pbat17/135760.pdf [accessed 2019-03-01] 3. Nicholls GH, Ling DM. Cued Speech and the reception of spoken language. J Speech Hear Res 1982 Jun;25(2):262-269. [Medline: 7120965] 4. Allen T. Patterns of academic achievement among hearing impaired students: 1974 and 1983. In: Schildroth A, Karchmer M, editors. Deaf Children in America. San Diego, CA: College Hill Press; 1986:161-206. 5. Singleton JL, Morgan D, DiGello E, Wiles J, Rivers R. Vocabulary use by low, moderate, and high ASL-proficient writers compared to hearing ESL and monolingual speakers. J Deaf Stud Deaf Educ 2004;9(1):86-103. [doi: 10.1093/deafed/enh011] [Medline: 15304404] 6. Traxler CB. The Stanford achievement test, 9th edition: national norming and performance standards for deaf and hard-of-hearing students. J Deaf Stud Deaf Educ 2000;5(4):337-348. [doi: 10.1093/deafed/5.4.337] [Medline: 15454499] 7. Zazove P, Meador HE, Reed BD, Gorenflo DW. Deaf persons© english reading levels and associations with epidemiological, educational, and cultural factors. J Health Commun 2013;18(7):760-772. [doi: 10.1080/10810730.2012.743633] [Medline: 23590242]

https://www.researchprotocols.org/2019/10/e14889 JMIR Res Protoc 2019 | vol. 8 | iss. 10 | e14889 | p. 10 (page number not for citation purposes) XSL·FO RenderX JMIR RESEARCH PROTOCOLS McKee et al

8. Alexander A, Ladd P, Powell S. Deafness might damage your health. Lancet 2012 Mar 17;379(9820):979-981. [doi: 10.1016/S0140-6736(11)61670-X] [Medline: 22423872] 9. Hauser PC, O©Hearn A, McKee M, Steider A, Thew D. Deaf epistemology: deafhood and deafness. Am Ann Deaf 2010;154(5):486-92; discussion 493-86. [Medline: 20415284] 10. McKee MM, Schlehofer D, Cuculick J, Starr M, Smith S, Chin NP. Perceptions of cardiovascular health in an underserved community of deaf adults using American sign language. Disabil Health J 2011 Jul;4(3):192-197 [FREE Full text] [doi: 10.1016/j.dhjo.2011.04.001] [Medline: 21723526] 11. McEwen E, Anton-Culver H. The medical communication of deaf patients. J Fam Pract 1988 Mar;26(3):289-291. [Medline: 3346631] 12. Valentine G, Maruyama T, Levy P, Skelton T, Correll J. University of Sheffield. The Role of the Internet in D/Deaf People©s Inclusion in the Information Society URL: https://www.sheffield.ac.uk/polopoly_fs/1.268538!/file/d-deaf_report.pdf [accessed 2019-03-01] 13. Valentine G, Skelton T. ©An umbilical cord to the world© -the role of the internet in D/deaf people©s information and communication practices. Inf Commun Soc 2009 Feb;12(1):44-65. [doi: 10.1080/13691180802158573] 14. Neter E, Brainin E. Association between health literacy, ehealth literacy, and health outcomes among patients with long-term conditions. Eur Psychol 2019 Jan;24(1):68-81. [doi: 10.1027/1016-9040/a000350] 15. Mitsutake S, Shibata A, Ishii K, Oka K. Associations of ehealth literacy with health behavior among adult internet users. J Med Internet Res 2016 Jul 18;18(7):e192 [FREE Full text] [doi: 10.2196/jmir.5413] [Medline: 27432783] 16. Stellefson M, Paige SR, Alber JM, Chaney BH, Chaney D, Apperson A, et al. Association between health literacy, electronic health literacy, disease-specific knowledge, and health-related quality of life among adults with chronic obstructive pulmonary disease: cross-sectional study. J Med Internet Res 2019 Jun 6;21(6):e12165 [FREE Full text] [doi: 10.2196/12165] [Medline: 31172962] 17. Luo YF, Yang SC, Chen AS, Chiang CH. Associations of ehealth literacy with health services utilization among college students: cross-sectional study. J Med Internet Res 2018 Oct 25;20(10):e283 [FREE Full text] [doi: 10.2196/jmir.8897] [Medline: 30361201] 18. Neter E, Brainin E. eHealth literacy: extending the digital divide to the realm of health information. J Med Internet Res 2012 Jan 27;14(1):e19 [FREE Full text] [doi: 10.2196/jmir.1619] [Medline: 22357448] 19. van Deursen AJ, van Dijk JA, Peters O. Rethinking internet skills: the contribution of gender, age, education, internet experience, and hours online to medium- and content-related internet skills. Poetics 2011 Apr;39(2):125-144. [doi: 10.1016/j.poetic.2011.02.001] 20. Mackert M, Champlin SE, Pasch KE, Weiss BD. Understanding health literacy measurement through eye tracking. J Health Commun 2013;18(Suppl 1):185-196 [FREE Full text] [doi: 10.1080/10810730.2013.825666] [Medline: 24093355] 21. Ownby RL, Acevedo A, Waldrop-Valverde D, Jacobs RJ, Caballero J. Abilities, skills and knowledge in measures of health literacy. Patient Educ Couns 2014 May;95(2):211-217 [FREE Full text] [doi: 10.1016/j.pec.2014.02.002] [Medline: 24637163] 22. Paasche-Orlow MK, Wolf MS. The causal pathways linking health literacy to health outcomes. Am J Health Behav 2007;31(Suppl 1):S19-S26. [doi: 10.5555/ajhb.2007.31.supp.S19] [Medline: 17931132] 23. Green LW, Ottoson JM, García C, Hiatt RA. Diffusion theory and knowledge dissemination, utilization, and integration in public health. Annu Rev Public Health 2009;30:151-174. [doi: 10.1146/annurev.publhealth.031308.100049] [Medline: 19705558] 24. Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs-principles and practices. Health Serv Res 2013 Dec;48(6 Pt 2):2134-2156 [FREE Full text] [doi: 10.1111/1475-6773.12117] [Medline: 24279835] 25. Pew Research Center. 2018. Internet and Technology URL: https://www.pewresearch.org/areas-of-expertise/technology-use/ [accessed 2019-03-01] 26. Kalichman SC, Cherry C, Cain D, Pope H, Kalichman M, Eaton L, et al. Internet-based health information consumer skills intervention for people living with HIV/AIDS. J Consult Clin Psychol 2006 Jun;74(3):545-554. [doi: 10.1037/0022-006X.74.3.545] [Medline: 16822111] 27. D©Elia LF, Satz P, Uchiyama CL, White T. Color trails test. In: Brown S, Bowen E, Prescott D, editors. The Forensic Psychologist©s Report Writing Guide. Abingdon, United Kingdom: Routledge; 1996. 28. Contreras J, Hauser PC, DeNaples G, Israel E. Executive Function Development in Deaf Unimodal Bilinguals. In: Proceedings of the Conference on the Association of Psychological Science. 2014 Presented at: APS©14; May 22-25, 2014; San Francisco, CA. 29. Hauser PC, Contreras J. Impact of Language Exposure on Executive Function Development. In: Proceedings of the Conference on the Association of College Educators for the Deaf and Hard of Hearing. 2014 Presented at: ACE-DHH©14; February 7-9, 2014; Washington, DC. 30. Hammill DD, Wiederhold JL, Allen EA. LinguiSystems Inc. 2006. TOSCRF-2: Test of Silent Contextual Reading Fluency±Second Edition URL: https://www.linguisystems.com/Products/13805/ toscrf2-test-of-silent-contextual-reading-fluencysecond-edition.aspx [accessed 2019-03-01]

https://www.researchprotocols.org/2019/10/e14889 JMIR Res Protoc 2019 | vol. 8 | iss. 10 | e14889 | p. 11 (page number not for citation purposes) XSL·FO RenderX JMIR RESEARCH PROTOCOLS McKee et al

31. Norman CD, Skinner HA. eHEALS: the ehealth literacy scale. J Med Internet Res 2006 Nov 14;8(4):e27 [FREE Full text] [doi: 10.2196/jmir.8.4.e27] [Medline: 17213046] 32. Strandvall T. Tobii Pro: Eye Tracking Technology for Research. 2014. Introduction to Eye Tracking Research URL: https:/ /www.tobiipro.com/news-events/events/introduction-to-eye-tracking-research-aug8/ [accessed 2019-03-01] 33. Wagner J, Lacey K, Chyun D, Abbott G. Development of a questionnaire to measure heart disease risk knowledge in people with diabetes: the heart disease fact questionnaire. Patient Educ Couns 2005 Jul;58(1):82-87. [doi: 10.1016/j.pec.2004.07.004] [Medline: 15950840] 34. Hauser PC, Paludneviciene R, Supalla T, Bavelier D. Rochester Institute of Technology Research. 2006. American Sign Language - Sentence Reproduction Test: Development & Implications URL: https://scholarworks.rit.edu/cgi/ viewcontent.cgi?referer=https://www.google.com/&httpsredir=1&article=1599&context=other [accessed 2019-08-28] 35. Hammill D, Brown V, Larsen S, Wiederholt JL. Test of Adolescent and Adult Language. Third edition. Austin, TX: PRO-ED Inc; 1994. 36. Corina DP, Lawyer LA, Hauser P, Hirshorn E. Lexical processing in deaf readers: an FMRI investigation of reading proficiency. PLoS One 2013;8(1):e54696 [FREE Full text] [doi: 10.1371/journal.pone.0054696] [Medline: 23359269] 37. Morford JP, Wilkinson E, Villwock A, Piñar P, Kroll JF. When deaf signers read English: do written words activate their sign translations? Cognition 2011 Feb;118(2):286-292 [FREE Full text] [doi: 10.1016/j.cognition.2010.11.006] [Medline: 21145047] 38. Weiss BD, Mays MZ, Martz W, Castro KM, DeWalt DA, Pignone MP, et al. Quick assessment of literacy in primary care: the newest vital sign. Ann Fam Med 2005;3(6):514-522 [FREE Full text] [doi: 10.1370/afm.405] [Medline: 16338915] 39. Deaf Health Literacy Study. 2019. Newest Vital Sign Health Measure URL: http://www.mckee-research.org/nvs/asl-intro. php [accessed 2019-03-01] 40. Bain SK, Jaspers K. Test review: review of Kaufman brief intelligence test, second edition. J Psychoeduc Assess 2010 Mar 21;28(2):167-174. [doi: 10.1177/0734282909348217] 41. Centers for Disease Control and Prevention. 2014. Behavioral Risk Factor Surveillance System Questionnaire - 2015 URL: https://www.cdc.gov/brfss/questionnaires/pdf-ques/2015-brfss-questionnaire-12-29-14.pdf [accessed 2019-03-01] 42. Shoebox Audiometry. 2019. Products URL: https://tinyurl.com/yyb6bprk [accessed 2019-03-01] 43. World Health Organization. 2019. Deafness and Hearing Loss - Key Facts URL: https://www.who.int/news-room/fact-sheets/ detail/deafness-and-hearing-loss [accessed 2019-03-01] 44. Tobii Pro: Eye Tracking Technology for Research. 2017. Tobii Pro Studio URL: https://www.tobiipro.com/learn-and-support/ learn/tobii-pro-studio/ [accessed 2017-04-03] 45. Paige SR, Stellefson M, Krieger JL, Anderson-Lewis C, Cheong J, Stopka C. Proposing a transactional model of ehealth literacy: concept analysis. J Med Internet Res 2018 Oct 2;20(10):e10175 [FREE Full text] [doi: 10.2196/10175] [Medline: 30279155] 46. Eysenbach G, Köhler C. How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. Br Med J 2002 Mar 9;324(7337):573-577 [FREE Full text] [doi: 10.1136/bmj.324.7337.573] [Medline: 11884321] 47. Smucker MD, Clarke CL. Time-Based Calibration of Effectiveness Measures. In: Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2012 Presented at: SIGIR©12; August 12-16, 2012; Portland, Oregon, USA p. 95-104. [doi: 10.1145/2348283.2348300] 48. Dedoose. Features URL: https://www.dedoose.com/home/features [accessed 2019-04-29] [WebCite Cache ID 77ztLdZIB]

Abbreviations ASK: attitudes, skills, and knowledge ASL: American Sign Language ASL-NVS: American Sign Language-Newest Vital Sign ASL-SRT: American Sign Language Sentence Reproduction Test CAB: community advisory board CTT: Color Trails Test eHEALS: eHealth literacy Scale eHealth: electronic health HL: hearing level HMC: Hurley Medical Center IQ: intelligence quotient IRB: institutional review board KBIT2: Kauffman Brief Intelligence Test, Second Edition NTID: National Technical Institute of the Deaf NVS: Newest Vital Sign PI: principal investigator

https://www.researchprotocols.org/2019/10/e14889 JMIR Res Protoc 2019 | vol. 8 | iss. 10 | e14889 | p. 12 (page number not for citation purposes) XSL·FO RenderX JMIR RESEARCH PROTOCOLS McKee et al

TOEL3: Speaking Grammar Subtest of the Test of Adolescent and Adult Language, Third Edition TWG: translation work group

Edited by G Eysenbach; submitted 31.05.19; peer-reviewed by V Manchaiah, E Neter; comments to author 01.07.19; revised version received 12.08.19; accepted 13.08.19; published 09.10.19 Please cite as: McKee MM, Hauser PC, Champlin S, Paasche-Orlow M, Wyse K, Cuculick J, Buis LR, Plegue M, Sen A, Fetters MD Deaf Adults' Health Literacy and Access to Health Information: Protocol for a Multicenter Mixed Methods Study JMIR Res Protoc 2019;8(10):e14889 URL: https://www.researchprotocols.org/2019/10/e14889 doi: 10.2196/14889 PMID: 31599730

©Michael M McKee, Peter C Hauser, Sara Champlin, Michael Paasche-Orlow, Kelley Wyse, Jessica Cuculick, Lorraine R Buis, Melissa Plegue, Ananda Sen, Michael D Fetters. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 09.10.2019. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.

https://www.researchprotocols.org/2019/10/e14889 JMIR Res Protoc 2019 | vol. 8 | iss. 10 | e14889 | p. 13 (page number not for citation purposes) XSL·FO RenderX