Florida State University Libraries

Electronic Theses, Treatises and Dissertations The Graduate School

2012 United States Federal Health Care Websites: A Multimethod Evaluation of Website Accessibility for Individuals with Disabilities John . Brobst

Follow this and additional works at the FSU Digital Library. For more information, please contact [email protected] THE FLORIDA STATE UNIVERSITY

COLLEGE OF COMMUNICATION AND INFORMATION

UNITED STATES FEDERAL HEALTH CARE WEBSITES:

A MULTIMETHOD EVALUATION OF WEBSITE ACCESSIBILITY

FOR INDIVIDUALS WITH DISABILITIES

By

JOHN L. BROBST

A Dissertation submitted to the School of Library and Information Studies in partial fulfillment of the requirements for the degree of Doctor of Philosophy

Degree Awarded: Spring Semester, 2012

John L. Brobst defended this dissertation on February 20, 2012.

The members of the supervisory committee were:

Charles . McClure Professor Directing Dissertation

Stephen . McDowell University Representative

Paul . Marty Committee Member

Charles . Hinnant Committee Member

The Graduate School has verified and approved the above-named committee members and certifies that the dissertation has been approved in accordance with university requirements.

ii

I dedicate this dissertation to my parents, Mr. Leander . Brobst and Mrs. Charlotte . Brobst.

Our parents walk in our lives, and too quickly leave. They fill our hearts, our souls, and our minds, giving us our start from which to explore and grow. In thanks, I dedicate this work. It could not have come about without their outstanding love, support, and inspiration. May God bless and keep you.

iii ACKNOWLEDGEMENTS

I wish to express my heartfelt appreciation to my committee members, Dr. Charles R. McClure, Dr. Stephen D. McDowell, Dr. Paul F. Marty and Dr. Charles C. Hinnant, for their support and input to this project. I also appreciate the support provided by Dr. John Carlo Bertot who also served on my committee during the project. Without their guidance, experience, and honored commitment to the high ideals of academic achievement, this success could not have been achieved. A very special thank you goes to Dr. Charles R. McClure, who not only served as my doctoral committee chair but also served as my mentor throughout the doctoral program. It was his enthusiasm, talent, and dedication that inspired me to pursue an advanced degree, making this research effort possible. Lastly, I wish to express my gratitude to all of my friends and colleagues for their support and assistance through the development and execution of this project. May success and happiness follow you throughout your lives.

iv TABLE OF CONTENTS

List of Tables vi List of Figures viii Abstract ix

1. INTRODUCTION 1

2. LITERATURE REVIEW 37

3. METHODOLOGY 76

4. SIDE-BY-SIDE POLICY ANALYSIS 133

5. ANALYSIS OF WEB CONTENT REVIEW 181

6. ANALYSIS OF AUTOMATED TESTING 204

7. ANALYSIS OF EXPERT REVIEW 224

8. CONCLUSION 248

APPENDICES 285

A. WEB ACCESSIBILITY CRITERIA FROM SECTION 508 285 . AUTOMATED ACCESSIBILITY TESTING PRODUCTS 292 C. RESOURCES ON ACCESSIBILITY 293 D. WEB CONTENT REVIEW PROTOCOL 294 E. EXAMPLE OF AN AUTOMATED TESTING SOFTWARE REPORT 295 F. TEXT OF SECTION 508 OF THE REHABILITATION ACT OF 1973 297 . TRAINING PROGRAM ELEMENTS 310

REFERENCES 312

BIOGRAPHICAL SKETCH 331

LIST OF TABLES

Table 1.1 Summary of the key elements of this research study. 6

Table 1.2 Relationship of research questions to study objectives. 7

Table 1.3 Project timeline. 34

Table 2.1 Summary of key issues. 74

Table 3.1 Tasks and schedule of pre-data collection steps. 77

Table 3.2 Timeline of data collection and presentation activities. 77

Table 3.3 Research design scheduled tasks. 78

Table 3.4 Sequence order for the research methods. 79

Table 3.5 Relationship of research questions to methods. 81

Table 3.6 Relationship of methods to study objectives. 82

Table 3.7 Populations for each methodology used in this study. 84

Table 3.8 Web content review goals and measures. 90

Table 3.9 Data collection worksheet – Web content review. 91

Table 3.10 Websites linked to WWW.HEALTH.GOV. 97

Table 3.11 Data collection worksheet – Presence of an accessibility statement. 98

Table 3.12 Automated testing goals, measures, and definition of measures. 99

Table 3.13 Identification of the URLs. 101

Table 3.14 Identification of the website tested. 102

Table 3.15 Automated testing - Identification of the web pages tested. 103

Table 3.16 Worksheet – Automated testing data collection. 104

Table 3.17 Worksheet to select an automated testing product. 107

Table 3.18 Data collection instrument – Automated testing. 108

vi Table 3.19 Expert testing goals, measures, and definition of measures. 111

Table 3.20 Identification of the web pages tested in expert review. 113

Table 3.21 Expert testing data collection instrument. 114

Table 3.22 Relationship of Section 508 criteria to the data collection worksheet. 117

Table 4.1 Key stakeholder groups. 144

Table 4.2 Key findings: Side-by-side analysis. 147

Table 4.3 Analysis of policy options. 169

Table 4.4 Analysis of policy options – Scoring table. 170

Table 5.1 Findings: Does the website have an accessibility statement? 185

Table 5.2 Findings: What is the quality of the accessibility statement? 190

Table 5.3 Types of content found in the web site accessibility statements. 195

Table 6.1 Identification of the errors found within the websites tested. 207

Table 6.2 Relationship between accessibility statement and error counts. 213

Table 6.3 How does this study compare? 217

Table 6.4 Relationship of findings to research questions. 219

Table 7.1 Expert testing data collection instrument. 225

Table 7.2 Websites ranked by expert review scores. 239

Table 7.3 Key findings as part of the scoring effort. 240

Table 7.4 Key findings that were not scored. 240

Table 8.1 Summary of implications. 249

Table 8.2 Relationship of research questions to methods. 258

vii LIST OF FIGURES

Figure 1.1 Technology Acceptance Model. 26

Figure 1.2 Conceptual framework diagram. 27

Figure 1.3 Overview of research proposal. 36

Figure 7.1 Unfiltered version of the web page (http://health.gov). 227

Figure 7.2 Web page simulated as monochromatic (http//health.gov). 227

Figure 7.3 Healthfinder.gov: Page as it appeared in browser. 230

Figure 7.4 Healthfinder.gov: Page under the deutanopia simulation. 230

Figure 7.5 HealthyPeople web page as viewed in the browser. 232

Figure 7.6 HealthyPeople web page as viewed in the tritanopia simulation. 232

Figure 7.7 Healthcare.gov: Website under normal browser viewing. 235

Figure 7.8 Healthcare.gov: Website under grayscale simulated viewing. 235

Figure 7.9 Healthcare.gov: Normal browser view of web page. 236

Figure 7.10 Healthcare.gov: Browser view with scripting disabled. 236

Figure 8.1 Conceptual framework diagram. 265

viii ABSTRACT

The problem addressed by this study is the observed low levels of compliance with federal policy on website accessibility. The study examines the two key federal policies that promote website accessibility, using a side-by-side policy analysis technique. The analysis examines the Americans with Disabilities Act of 1990 and Section 508 of the Rehabilitation Act of 1973, which together have the intent of establishing a legal mandate for equality regarding website accessibility. In comparing key aspects of these two policies, the findings indicate several areas were the policies were inconsistent or unclear in providing guidance on attaining accessible websites. The study performed an accessibility analysis that included automated testing, an expert review, and a content review of six federal health care websites. Recommendations call for the establishment of federal programs that would provide federal website managers with additional training in website accessibility, and provide support for ongoing research towards developing guidance and tools that would serve to promote websites that are more accessible.

ix CHAPTER 1

INTRODUCTION

Problem Statement

The problem this study addressed was to determine how well federal health care websites comply with Section 508 of the Rehabilitation Act of 1973 (29 U..C. § 794d.). This research effort conducted an examination for compliance with the web accessibility criteria identified within Section 508 of the Rehabilitation Act of 1973 (Section 508). For this research study, web accessibility refers to providing equal access and equal opportunity to the Internet for people with disabilities. Web accessibility means that people with disabilities can perceive, understand, navigate, and interact with the web, and that they can contribute to the web (W3C, 2008). A major element of this research effort was to identify the problems and barriers that prevent federal webmasters from attaining accessible websites. A better understanding of the term webmaster is available from the International Webmaster Association (IWA). The IWA is a non-profit professional association, recognized as being the industry's leading provider of educational and certification standards for web professionals (IWA, 2010). Pells (1998, p. 46) indicates that the IWA provides the following definition for webmaster: ―a webmaster is a professional who has primary responsibility for managing websites.‖ A webmaster is the employee who has the primary responsibility for the design, implementation, and maintenance of an organization's website. Typically, the webmaster is a staff person or an outside contractor tasked with overseeing the website, whether or not the term webmaster appears officially in a title or job description (Pells, 1998). This research assessed the implementation of Section 508 of the Rehabilitation Act of 1973 (29 U.S.C. § 794d.) as it applies to health care websites provided by the federal government of the United States. Section 508 has the intent of assuring that all federal websites are accessible to individuals with disabilities, and provides specific criteria towards assuring equal access by those individuals with disabilities. There is a small body of knowledge addressing this general topic area, but the knowledge that does exist indicated poor accessibility (Johnston & Helms, 2008; Jaeger, 2006; Jaeger, 2004; West, 2002, 2004). Individuals with disabilities comprise over

1 54 million individuals in the United States, and represent an increasingly large as a segment of the population (Jaeger & Bowman, 2005). This means that these websites may be inaccessible to a large group of citizens and may be in violation of federal law. The research described in this study provides a detailed evaluation of the accessibility of selected federal government health care websites for individuals with disabilities in the terms or the standards identified in Section 508.

Definition of Key Terms

This section defines the key terms used throughout this document. Accessibility – The term accessibility refers to web or website accessibility, where web accessibility refers to providing equal access and equal opportunity to the Internet for people with disabilities. More specifically, web accessibility means that people with disabilities can perceive, understand, navigate, and interact with the web, and that they can contribute to the web (W3C, 2009a). Accessibility error – An indication of a violation of the standards and criteria that are mandated for federal websites by Section 508 of the Rehabilitation Act of 1973, as amended in 1998 (29 U.S.C. § 794d.). Compliance – The achievement of the accessibility standards as defined within Section 508 of the Rehabilitation Act of 1973 (herein referred to as Section 508). Disability – This study used the definition as provided within Americans with Disabilities Act of 1990 (42 U.S.C. § 12102[2]), which is consistent with the definition used by the United Stated Census Bureau. That definition is: The term ―disability‖ means, with respect to an individual— (A) a physical or mental impairment that substantially limits one or more of the major life activities of such individual; (B) a record of such an impairment; or (C) regarded as having such an impairment. Federal government health care websites – Official websites of the federal government have the designation of .GOV or .MIL as a domain name within the website‘s Universal Resource Locator (URL) address (GSA, 2009; FWC, 1997). The health care identification or

2 association exists if these sites are included on the homepage of the official federal government web portal for health care information, identified as www.health.gov (www.health.gov, 2009). ―www.health.gov is a portal to the websites of a number of multi-agency health initiatives and activities of the U.S. Department of Health and Human Services (HHS) and other Federal departments and agencies‖ (www.health.gov, 2009). Federal information policy - The statement of a specific goal set by the U.S. government to regulate information related activities—both within the government and for society (Thompson, McClure, & Jaeger, 2003). Homepage - The opening page encountered first at a World Wide Web site that usually contains introductory information about the website and may provide links to the other pages within the website. Information policies - Policies instituted to affect the manner and degree to which people have access to the products and services through electronic media (Thompson, McClure, & Jaeger, 2003). Policy - Policy at any level is the set of government directives intended to shape decisions and actions of individuals, organizations, and government agencies. Policy is the guidance derived from a variety of sources that may include legislation, directives, guidelines, court decisions, presidential statements, agency circulars, and other official statements (McClure, 1999). Webmasters – webmasters are those individuals that have primary responsibility for web maintenance, development, or management of a website (Pells, 1998).

Significance of the Problem

The Internet has evolved to the point where it provides the global community with extensive communication capabilities and access to an overwhelming number of information resources. This international interaction amongst diverse senders and receivers serves to elevate the importance of attaining website accessibility. Improved accessibility helps enssure that the products and services proffered by this medium are available, usable, and readily understandable by all individuals. The power of the Internet is international, and access by everyone regardless of disability is paramount.

3 Within the United States, people with disabilities comprise over 54 million individuals, representing a significant group within the population, and growing even larger as the baby boomers grow older (Jaeger & Bowman, 2005). Even though the slow development of assistive devices (used to facilitate usability and accessibility of websites) has impeded use of the Internet by individuals with disabilities, that group comprises over 12% of all Internet users (Pew Internet and American Life Project, 2003). As the entire population continues to age, while living longer in their golden years, the number of users (with disabilities) of federal websites will continue to increase. The growing size of this group, combined with the overall expanding use of the Internet (Pew Internet and American Life Project, 2004), makes this group of website users increasingly important. This important societal group of federal website users has a very real human centric concern, which is the desire to have full accessibility to the health care services and resources available via government websites. Several major legislative efforts have established a solid legal framework for requiring website accessibility (Jaeger, 2004; 2006). That legal requirement for web accessibility applies to all federal government websites through the provisions of Section 508 of the Rehabilitation Act of 1973. However, the mere act of passing a law has not made website accessibility a reality. Several broad based studies indicate that overall efforts to attain federal website accessibility have fallen short (Jaeger, 2006; Kruse & Hale, 2003; Kruse & Schur 2003; Wells 2001; Hignite, 2000). The low accessibility found in federal websites indicates a weak policy environment, with ineffective legislation at the core of the problem (Johnston & Helms, 2008; Jaeger, 2004, 2006). Contributing to the low accessibility within that weak policy environment, may be such factors as lack of enforcement and the perception of significant expense associated with providing accessibility. The failure of existing federal policy, combined with an ethical commitment to provide equal access, the significance of this effort (Jaeger, 2004; 2002). Failing to make accessibility a reality could result in the disenfranchising of a major segment of society. Ethical concerns and a sense of social responsibility for any under-served group compel action to assure compliance with the intent of the law. ―Unequal access is also a violation of the general notions of equality and justice that are inherent in a democratic society…‖ (Jaeger, 2002, para. 19).

4 Significance of Accessible Federal Health Care Websites

Jaeger (2006) stated that many studies have used automated testing to determine the accessibility of the websites, and then rehashed the tools and techniques that are available to webmasters to fix the problem. Yet website accessibility remains low. This study determined what realistic and pragmatic steps would make federal health care websites more compliant with the criteria identified in Section 508. The Internet is an increasingly important source of health care related information. Health conscious consumers rely on the web for supplemental health related information (Harris Interactive, 2006; Rogerson & Fairweather, 2001). A survey by Harris Interactive covering the United States found that 74% of all adults used the Internet, and of these 80% or 160 million go online to seek health care information (Harris Interactive, 2006). These users seek web based health care information on average five times a month (Harris Interactive, 2006). The ability to obtain accurate medical information quickly, conveniently, and privately presents consumers with the opportunity for improved decision-making and allows an increased level of participation in their personal medical care (Fox, 2006). This growing importance of Internet based health care information requires that these resources are usable and accessible to everyone. Despite the large amount of health related information available on the Internet, there is little knowledge about the accessibility of that health information (Berland, et al., 2001). The Internet has the potential to eliminate substantially those barriers that inhibit access to information for patients, but only if that information is obtainable by all individuals, including those persons with disabilities. This research provides a better understanding of the accessibility of federal health care websites, and results in recommendations for improved access to those websites. These recommendations address the areas of training, technology, managerial practices, and makes suggestions for improving federal accessibility policy. Improving the access to this group of federal websites will help assure the provision of quality health care information to all individuals, including those with disabilities. These improved and more accessible websites will better communicate important health and medical information to all users, and may thereby improve the health related decisions made by the individual users.

5 Study Purpose, Goal, and Objectives

The purpose of this research study was to conduct an evaluation of the accessibility of federal health care websites for individuals with disabilities, and to understand the factors that may impede attaining accessible websites. The intent of this study was to develop a better understanding of the accessibility of federal health care websites and to determine what barriers may prevent attaining full accessibility. Table 1.1 presents a summary of the key elements that guided the development of this research study. Using these elements to shape the assessment of the web accessibility, the study gained insight as to the impact of any inaccessibility upon individuals with disabilities in obtaining federally provided health care information, products, and services. Prior research indicated that federal websites have generally exhibited poor levels of accessibility (Jaeger, 2006; West, 2004, 2002).

Table 1.1: Summary of the key elements of this research study. Summary of the Key Elements of this Research Study The problem this study addresses is to determine how well federal health Problem care websites comply with Section 508 of the Rehabilitation Act of Statement 1973(29 U.S.C. § 794d.). The purposes of this research study are to conduct an evaluation of the accessibility of federal health care websites for individuals with Purpose disabilities, and to understand the factors that may impede attaining accessible websites. To improve the accessibility of the federal websites that provide health care Goal information. 1. Evaluate compliance of federal health care websites using Section 508 criteria. Objectives 2. Identify the issues and barriers that affect attaining compliance. 3. What changes would promote websites that are more accessible?

The goal of this research was to improve the accessibility of the federal websites that provide health care information. Identifying the barriers and understanding the steps taken to overcome these problem areas supports the development of practical recommendations regarding how webmasters within federal agencies can mitigate or overcome the challenges of providing websites that are fully accessible to individuals with disabilities. Objectives for this goal included:

6 1. Evaluate compliance of federal health care websites using Section 508 criteria. 2. Identify the issues and barriers that affect attaining compliance. 3. Identify practical recommendations to promote websites that are more accessible. These objectives served to guide this study‘s research questions. These objectives, combined with the nature of the research questions, contribute to the determination of the data collection and data analysis techniques employed within this study.

Research Questions

The research questions used within this study align with and support the stated objectives, as indicated in Table 1.2. The questions, along with the data collection and data analysis methods, reflected the intent to address the lack of knowledge regarding the problems faced by federal webmasters in developing fully accessible websites. The study determined the

Table 1.2: Relationship of research questions to study objectives. Relationship of Research Questions to Study Objectives Research Research Research Objective Objective Objective 1. 2. 3.

Evaluate the Identify issues Identify practical Research Research compliance of and barriers recommendations federal health care that affect to promote more Questions Methods websites using attaining accessible Section 508 compliance. websites. criteria.

Are federal health care web Expert testing sites compliant with Section Automated testing 508 requirements?

What are the issues and Expert testing barriers to attaining Automated testing accessibility? Side-by-side policy X analysis

What changes would Expert testing promote websites that are Automated testing more accessible? Side-by-side policy X analysis Web content review

7 accessibility of federal health care websites, and evaluated the accessibility of that group of websites by comparing the number of websites without errors (as determined by Section 508 criteria) to the number of websites examined within that studied group. (Brobst, 2009; Cullen, 2008; Jaeger, 2006; West, 2002). Additional knowledge resulted by posing the following research questions: 1. Are federal health care websites compliant with Section 508 requirements? 2. What are the issues and barriers that affect attaining accessibility? 3. What changes would promote websites that are more accessible? The nature of these research questions provides an indication as to the more useful methods of conducting the data collection and analysis. Table 1.2 indicates the relationship between the research questions, methodology, and the stated objectives for this study.

Measures

Within this research effort, accessibility relates to the ability of a user with a disability to interact with a website. This interaction includes evaluating the access to content elements and functions found within a website. This study measured accessibility in terms of the standards provided within Section 508 of the Rehabilitation Act. Section 508 established specific accessibility standards for federal government websites, making those standards an ideal set of measures for evaluating the accessibility of government health care websites. As such, when this study discusses accessibility, it is in terms of the Section 508 standards. The number of errors found is the measure used to indicate the accessibility for a website, where errors are instances where the website violates the standards established by Section 508. Section 508 details specific criteria to achieve accessibility, and this law applies to federal health care websites. These criteria are identified in Section 508 (§ 1194.22) and are presented in Appendix A. These standards provided a set of requirements suitable for use in an evaluation of the accessibility of federal health care websites. Another means of measuring accessibility includes identifying the types of errors (Section 508 violations), which can be easily accomplished by use of automated accessibility testing. Many automated software products are available for conducting accessibility testing of websites. Appendix B identifies a few of the more commonly used software packages designed

8 for testing the accessibility of any website. This type of software product can quickly examine a website, methodically exploring each web page within the site. The software generates a report that identifies the types of errors (Section 508 violations) and the count of occurrences of that type of error. Each error identified is a definite case of non-compliance with the accessibility standards and specified criteria defined in Section 508. Errors so identified are indications that one or more groups (i.e., individuals having a similar type of disability) will find it impossible to access information in the web document (W3C, 2009a). Correcting these errors is a basic requirement to assure that these groups have a minimum level of accessibility to the website (Section508.gov, 2010). Automated software testing, expert testing, and user based testing are typical methods used to examine a website for accessibility errors. Each method brings specific strengths to the analysis effort, and when used in combination they can provide a more comprehensive and accurate determination of the accessibility of a website. Regardless of the method used, the standard for making the determination of error relates to the criteria established in Section 508. Measures of accessibility for this study refer to measures that indicate the degree to which federal agencies provide websites that are accessible as defined by statuary provisions identified within Section 508 of the Rehabilitation Act. Measures of accessibility provide quantifiable data on the individual‘s ability to effectively use the websites and understand the content provided therein. For this study, the user group under consideration is a group comprised of individuals with disabilities. Measures of website accessibility typically stem from three separate areas: 1) the policy environment, 2) the webmaster‘s perspective of providing accessibility features on the website, and 3) the user‘s perspective in attempting to access and comprehend the information presented within the website. The conceptual framework that served to guide this research incorporates each of these areas. This study used multiple methods that provide information related to all three areas. The evaluation methods employed by this study provided measures of access that include: 1. Side-by-side policy analysis - analyzed the key legislative acts related to web accessibility and provided a policy focused qualitative assessment of selected issues identified in the literature review.

9 2. Web content review – examined website content for accessibility related policy statements. This method provided a qualitative measure of the relationship between accessibility of a website and the provision of accessibility related policy statements on a website. 3. Expert testing – provided a user centric focus on quantitative and qualitative measures related to the accessibility of the website, as counts of instances of non-compliance (errors) using the criteria identified in Section 508 and the types (or categories) of errors found. 4. Automated testing – provided quantitative measures related to the accessibility of the websites, as counts of errors and the types of errors found. Results obtained from these methods guided the development of recommendations to improve policies, organizational processes, tools, and techniques used to attain accessible websites.

Key Issues

Web technology has been developing and evolving at a rapid pace, and will continue to do so (Peters, 2009). This changing environment poses a considerable challenge to the webmasters who must implement measures to assure accessibility, in step with each new technological development. While this constantly changing technology is a major challenge to providing accessible websites, that environment of rapid change also drives concerns related to other issue areas as ethics, policy, and legal liability.

Technology and Accessibility

The web provides users with access to a wealth of information. That user population is a diverse group that includes individuals of all ages, socio-economic classes, and educational levels. Each of those sub-groupings is comprised of individuals with varying levels of technical computing skills (Lazar, Dudley-Sponaugle, & Greenidge, 2004; Shneiderman, 2000). In looking at the characteristics that describe the larger population, many users of the web are individuals with disabilities (Pew Internet and American Life Project, 2004). Examples of such disabilities include sensory (e.g. hearing and vision), motor (e.g. limited use of hands), and

10 cognitive impairments (e.g. learning disabilities) (Lazar et al., 2004). These users with disabilities may employ various forms of assistive technology that enhance their ability to browse websites or more easily comprehend the information presented on the website. These forms of assistive technologies may include such hardware and software as screen readers, voice recognition, alternative pointing devices, adaptive keyboards, and refreshable displays (Lazar et al., 2004, Paciello, 2000). Users with disabilities are only able to utilize a website if its design is compatible with the various assistive technologies and accessibility guidelines (Lazar et al., 2004). When websites are designed to provide appropriate use of these accessibility features and support the use of assistive technologies, then the sites are called accessible websites (Slatin & Rush, 2003). In making a physical analogy, one can envision that building an accessible website is much like constructing an accessible building (Lazar et al., 2004). Just as ―an accessible building offers curb cuts, ramps, and elevators to allow a person with disabilities to enter and navigate through the building with ease, accessible website offers similar functionality‖ (Lazar et al., 2004, p. 269). Many resources provide guidance, best practices, and testing recommendations for web developers to follow, so that their websites will be accessible (Appendix C lists several of these resources). One of the major sources of good information is the Web Accessibility Initiative (W3C, 2009a) that provides guidelines, called the Web Content Accessibility Guidelines (WCAG) that help web developers make their sites accessible. The United States government is another good source of accessibility guidance, providing accessibility guidance derived from the criteria established in Section 508 (available at http://www.section508.gov). In addition to such web-based guidance, webmasters rely on web development software to help automate the process of designing and creating websites. These software products have increasingly incorporated programming that builds-in accessibility features during the creation of new web pages. These web development products (such as DreamWeaver and FrontPage) serve to assist developers with accessibility-related issues by automatically identifying areas that require accessibility accommodation. As web developers create new web pages, the software recognizes the need for an accessibility feature, prompting the web developer to provide the appropriate remedy. The software becomes a thinking extension of the webmaster‘s development process, to prevent the inadvertent overlooking of accessibility issues.

11 Another technology tool used by website developers is automated accessibility testing software. Automated testing software products are readily available to help webmasters quickly and easily find accessibility problems. These automated tools apply to the development phase of creating a new website, or apply to existing websites during the update process. Appendix B provides information on several popular automated testing products as InFocus, LIFT Machine, RAMP Ascend, WebKing, and WebXM. The indication is that the webmasters have available sufficient guidance, resources, and technological tools that would assure that websites are fully accessible. For federal website developers, the impetus to be accessible even carries the mandate of federal law (Section 508). However, most websites are not currently accessible (Lazar et al., 2004). This study attempts to explore the actual accessibility of federal health care websites, and to discover the factors that may impede attaining accessible websites. The research objectives attempt to identify the barriers to accessible websites; identify changes that would improve accessibility, and provide practical recommendations to improve website accessibility. Lazar, Dudley-Sponaugle, and Greenidge (2004) indicated that in recent studies, private and non-profit websites (Lazar, Beere, Greenidge, & Nagappa, 2003), for-profit commerce websites (Sullivan & Matson, 2000), United States state websites (Ceaparu & Shneiderman, 2002), and even U.S. Federal websites (Stowers, 2002) were found to have major accessibility problems. Lazar and Greenidge (2006) indicated that some classes of websites are actually becoming more inaccessible, as accessibility barriers have become increasingly common to those website classes. When examining the accessibility of 50 websites from 2002 to 2003, on average the websites displayed increases in the number of accessibility problems over this period. Of the 10 categories of websites examined, the study found that the biggest percentage changes in increased inaccessibility occurred in the categories of state and local governments, non-profit organizations, and software development firms. Technology affects accessibility in several areas that include:  New products that need to be accessible,  Assistive devices,  Software used to develop websites, and  Software used to test web pages for accessibility compliance.

12 Not only does technology affect accessibility from several fronts, but also its changing nature serves to make the ongoing battle to be accessible an even greater challenge.

Ethics, Policy, and Civil Rights

Democratic concepts of equality trace back to the founding of this great nation. The trail of subsequent amendments to the United States Constitution displays a progression of victories promoting equality. While these amendments resulted from long fought battles, they represent major steps forward for racial and gender equality. However, the granting of such equalities to individuals with disabilities did not begin to occur until the 1970s. Section 504 of the Rehabilitation Act of 1973 is widely recognized as being the first major disabilities rights law. The Rehabilitation Act applies to the federal government and to all entities in receipt of federal funds. The success of this legislation was marginal, as federal government efforts actually hampered its implementation (Fleischer & Zames, 2001). Subsequent legislation, as the Americans with Disabilities Act of 1990, had limited success towards establishing an awareness of accessibility issues and failed in its efforts to increase the presence of individuals with disabilities in the workplace (Jaeger, 2004). Its weakness in accomplishing its intent was due primarily to the uneven and limited enforcement efforts by the federal government (Hignite, 2000; Kruse & Hale, 2003; Kruse & Schur, 2003). The passage of the Americans with Disabilities Act (42 U.S.C.A. § 12101 et seq.) and Section 508 of the Rehabilitation Act of 1973 (29 U.S.C. § 794d) had the intent of eliminating discrimination against individuals with disabilities. However, during this time of disability- related legislation, a new technology was emerging and that technology was the Internet. The new web based products and services that the Internet made possible became increasingly common in the public and private sector. The ADA and Section 508 perhaps did not foresee the degree to which the government would exploit this technology to provide more efficient and effective services to the citizenry. Section 508 (including subsequent amendments) established web based accessibility standards and compelled federal agencies to comply with additional accessibility standards issued by the Access Board (Access Board, 2009). The Access Board is a federal agency whose role includes promoting accessible website design (Peterson, 1998). Section 508 required that

13 websites be accessible to individuals with disabilities, but its initial scope did not apply to the recipients of federal funding, nor did the law attempt to regulate directly the state or private sector. These public laws demonstrate a concern for assuring equal rights and access for those with disabilities. However, the intent of the laws, being to achieve web accessibility, has not been accomplished (Jaeger, 2006). The failure of existing legislative actions, combined with an ethical commitment to provide equal access, underscore the significance of this effort. If allowed to continue, the failure to make web accessibility a reality could result in the disenfranchising of a major segment of society. Ethical concerns and a sense of social responsibility for any under-served group compel action to assure attaining the intent of the law. ―Unequal access is also a violation of the general notions of equality and justice that are inherent in a democratic society…‖ (Jaeger, 2002, para. 19). Generally, studies on website accessibility reveal an overall poor record in attaining compliance with accessibility standards (Jaeger, 2006; West, 2002, 2004). The implication is that this situation denies a large group of individuals the full and complete access to website content. A situation of discriminatory practices that deny fair and equal access to government services is ethically unacceptable and in direct conflict with the intent of federal law (Section 508 and the ADA).

Legal Liability

Website owners are under increasing pressure to comply with the tenets of the ADA. Specifically, the ADA requires equality of services for individuals with disabilities, as prescribed by the public accommodation language within that law. There is a growing controversy in the courts over the legal applicability of the ADA concept of public accommodation. Proponents of full website accessibility propose that the U.S. Department of Justice (DOJ) intercede in this controversy by classifying websites as public accommodations that are accountable to the standards set by the ADA (Jaeger, 2004). If the DOJ indicated that the scope of application for public accommodation covered federal websites, then the websites would need to be fully and equally accessible by individuals with disabilities. To accomplish this change, the DOJ would

14 need to make a definitive interpretation asserting that the ADA applies to federal websites in the context of being public accommodations. Yet, the DOJ has not acted to make this clarification. A more clearly defined interpretation of the term public accommodation would serve to clarify the webmaster‘s responsibility regarding the need for providing website accessibility features. Such clarification would support the bringing of class actions suits against website providers that failed to comply with Section 508 criteria. Class action lawsuits are an increasingly pervasive force in today's world, and in many cases are an effective means of compelling compliance with the intent of the law (Gibson, Dunn, & Crutcher, LLP, 2009). Establishing legal liability could be a powerful force in assuring compliance with the goals of federal accessibility policy. One such example is a major class action suit filed by the National Federation of the Blind against Target Corporation, which is currently working its way through the California courts (Klein, 2007; Thatcher, 2010). The lawsuit alleges that Target failed to and refused to make its website accessible to the blind, in violation of the Americans with Disabilities Act. In the fall of 2007, a federal district court judge determined that the Target case had merit as a class-action suit and ruled Target accountable for the accessibility claim. While similar litigation has developed against other large corporations over the last decade, most of the cases resulted in private settlements (Klein, 2007). However, precedent-setting cases, such as this Target case, get headlines and quickly establish increased awareness throughout the legal and corporate communities, as well as the public sector. This increased awareness, propelled by concern over potential liability claims, would certainly help to motivate federal website managers to support and attain full website accessibility.

Importance, Benefits, and Impacts

In the United States, federal websites are required to be accessible as defined by the criteria found in Section 508 of the Rehabilitation Act of 1973. Therefore, the accessibility of federal government provided health care websites is mandated by this legislation. The expectation of the law is that a large segment of society (i.e., those individuals with disabilities) should not suffer exclusion from the benefits, services, and products offered by these websites (Access Board, 1999). The problem is that an inaccessible federal health care website would

15 effectively deny persons with disabilities the chance to use the information and services in a fair and equal manner (Jaeger, 2006). The purpose of this study is to evaluate the accessibility of federal websites that provide health care information in terms of the standards and criteria mandated by Section 508 of the Rehabilitation Act of 1973. Given that stated purpose, an expected outcome of this study is improved information accessibility for the users of those websites.

Importance

A significant value of this research lies in its uniqueness. This study explored a relative uncharted area, being the accessibility of health care websites provided by the United States federal government. The importance of this study lies primarily in its findings and recommendations that:  Addressed a lack of information within this topic area: This research focused on an unexamined class of websites (federal health care sites) and provided information as to types of accessibility problems and the relative frequency of problems.  Identified impediments to providing accessible websites: The automated testing, expert testing and side-by-side policy analysis revealed issues and barriers confronting webmaster‘s in the pursuit of accessibility.  Enhanced the conceptual framework for analysis: The research validated the elements and factors of the Technology Acceptance Model (TAM) based framework previously developed for accessibility research (Jaeger, 2006). This work uncovered new factors that improve the framework and extend its application within accessibility research. By examining this new class of websites, the study provided results that further extend and refine the conceptual framework.  Provided practical recommendations: These recommendations can improve federal policy development, agency provision of accessible websites, and the accessibility of federal health care information and services made available via websites. These findings and recommendations will help to ensure that as many people as possible are able to use government websites to obtain health related information, products, and services.

16 From a policy perspective, this research is important in its contribution to improving the policies and legislation that attempt to attain the social goals of equity and fairness for all citizens of this nation. By examining the key policies that relate to federal health care websites, this research revealed areas of conflict or inconsistencies that act as impediments. This study looked for those inconsistencies or issues by examining the key legislative documents using a classical side-by-side policy analysis approach. By searching for policy conflicts using these two approaches, it was possible to gain a richer and fuller understanding of the political environment surrounding accessibility. This type of analysis helps inform the legislative process, and indicates whether the social goal of providing an accessible federal government has resulted, as intended by existing legislation. The policy research performed by this study afforded the opportunity to raise the awareness of federal policy makers, regarding the need for compliance with statutory accessibility regulations. The side-by-side policy analysis identified policy areas that conflict and are incongruent with the intent of the existing legislation (primarily Section 508). From a website developer point of view, this research improves accessibility by discovering and disseminating best practices that will serve to raise the overall level of technical and managerial skills related to attaining accessible websites. This study revealed actual impediments faced by federal webmasters as they attempt to produce accessible websites. This research is of greatest importance and of direct value to the over 54 million Americans with disabilities, as it helps improve their ability to access the information and services provided by federal health care websites. The intent of this research effort was to facilitate increased accessibility of federal health care websites. In doing so, individuals with disabilities are better able to use, comprehend, and interact with the content, services and products offered to the American public through these federal websites.

Benefits and Impacts

Obtaining health care information via websites is increasingly common, and the Internet is used extensively by the federal government as a service to the public (Harris Interactive, 2006; Rogerson & Fairweather, 2001). This growing importance of Internet based health care information requires that these resources are usable and accessible to everyone. That importance

17 of providing information using Internet based systems serves to support the need to study this area. The intended outcome of this research is to improve accessibility levels and assure that equal access exists for all individuals as a matter of social justice. By assuring website accessibility, the expected outcome is to guarantee that individuals with disabilities will have a level of use that is equal to the use enjoyed by those individuals that do not have such disabilities. The conceptual framework used in this study provides an overview of the environmental context and provides a structure for better understanding the influences that affect the implementation of Section 508 requirements. That framework encompasses organizational factors and issues that shape the degree of value assigned to attaining accessible websites. This study confirms portions of this conceptual framework by 1) measuring the levels of accessibility for this studied class of websites, and 2) identifying the impediments that webmasters face in attaining accessible websites. This study benefits the theoretical foundations of accessibility research as it validated the existing elements and factors indicated in a conceptual framework previously developed for accessibility research (Jaeger, 2006). Additional benefits resulted from the discovery of new factors that improved the framework and extended its application within accessibility research. The research questions posed within this study guided the research by addressing the accessibility related issues that affect how agencies approach providing accessible websites. Those issues included the identification of the type of errors found on websites and the problems that face webmasters in providing accessible websites. Webmasters directly benefit from this study, as the findings and recommendations provide practical guidance that informs webmasters on actions and methods that can help them to better attain accessible websites. The benefits included recommendations that propose best practices for federal webmasters that can improve management approaches to website accessibility. The findings also identified issues and impediments that affect a webmaster‘s efforts to provide accessible websites, and provided recommended steps to mitigate those problems. The impact of these recommendations is that federal health care websites may become more compliant with the intent of current legislation and more accessible to all individuals. The findings from this study promote health care information that is more accessible, as provided by federal websites. That facilitation will arise from achieving a more complete

18 understanding of the actual accessibility of these websites and by offering pragmatic recommendations and best practices to improve the situation. The findings and subsequent conclusions derived from this study include an analysis of policy issues, recommendations for best practices, and a refinement of the conceptual framework. If these recommendations and best practices lead to websites that are more accessible, then two major stakeholder groups will benefit significantly. One such group would be the over 54 million Americans with disabilities, as it may improve their ability to access the information and services provided by federal health care websites. The second group to benefit will be the federal government that will become more accessible to the American public by having greater compliance with the accessibility standards as prescribed by existing legislation.

Overview of Research Methods and Data Collection

To obtain an indication of the accessibility of federal health care websites, the study utilizes Side-by-side Policy Analysis, Web Content Review, and two types of accessibility testing methods: Automated Testing, and Expert Testing. The following provides a brief description of each of these research methods.

Side-by-side Policy Analysis

Policy can direct or shape the decisions and actions of the government, organizations, or individuals. As policy shapes decisions and actions, it also serves to shape program impacts and outcomes. The side-by-side policy analysis for this research effort identified and discussed the key issues revealed from the literature review (chapter 2). In addition to addressing policy issues, this side-by-side policy analysis examined the key policy instruments that form the basic legal foundation that requires website accessibility for federal websites. The information policy instruments examined were the Americans with Disabilities Act of 1990 and the Rehabilitation Act of 1973, as amended by Section 508 in 1998. The primary value of this analysis was that it revealed significant differences in the way these legislative acts attempt to eliminate discrimination against individuals with disabilities. The findings of the side-by-side policy analysis refined the development of the subsequent data

19 collection efforts. Side-by-side policy analysis identified areas of weakness and deficiencies in existing policy, and revealed areas of ambiguity or contradiction. This side-by-side policy analysis was instrumental in identifying areas of further research.

Website Content Review

This study conducted a second type of policy analysis, being a review of federal health care websites to see if these sites provided an organizational policy on website accessibility. The objective is to determine whether a relationship exists between the existence of accessibility guidelines and policy statements compared to the measured accessibility. This study conducted a web content review in order to determine if a web accessibility statement exists within a website and then determine the quality of that web accessibility statement. The purpose of this content review was to assess whether federal health care websites with web accessibility policy statements have higher web accessibility than those without such statements. In the contextual framework, this analysis addressed the organizational administrative practices that may affect web accessibility.

Automated Testing

Automated testing is an unobtrusive approach to determining website accessibility. This approach employs a software program to examine a web page or an entire website to determine accessibility. This automated testing technique provides quantifiable measures (counts and types of Section 508 violations) that indicate the website‘s accessibility. This approach is appropriate for examining a web page, an entire website, a sampling of a target group of websites, or all the websites in a target population. This study applied automated testing to a purposive sample of federal websites to check for Section 508 violations, in order to provide an indication of accessibility compliance across all federal health care websites. The automated testing effort provided insight into the range of accessibility errors found within this group of websites. As such, this testing method supports the identification of higher/lower performing websites. Finding a broad range of errors allows the researcher to compare characteristics of the higher performing group (fewer errors) to the lower

20 performing group (more errors). Findings from the automated testing effort provided insights as to the error types and the relative frequency (comparative counts) of those errors.

Expert Testing

Expert testing refers to having a trained authority on website accessibility review the website for errors. The expert researcher employed a scoring checklist and structured walk- through to conduct the review of the website. The benefit of using expert testing lies in its integration of a human element. This human element (or human centric focus) supported the identification of problems, but also supported the identification and examination of the underlying causes of the problems. Expert testing is a more cost effective means of determining accessibility, when compared with user-based accessibility testing (Brajnik, 2008). A relative weakness of expert testing lies in the observance that expert evaluators have behaviors that differ from typical user behaviors, which may lead to missing some issues relating to accessibility (Doubleday, Ryan, Springett, & Sutcliffe, 1997). Expert evaluators tend to be more subjective in their evaluations, which may result in findings or conclusions that reflect personally based judgments (Law & Hvannberg, 2002). As these experts are highly trained and skilled, they are costly to employ and have limited availability (Law & Hvannberg, 2002).

Sampling and Population

The side-by-side policy analysis examined the two legislative acts that establish federal policy on website accessibility. Those two acts are the Americans with Disabilities Act of 1990 and the Rehabilitation Act of 1973, as amended by the 1998 Amendment of Section 508. This side-by-side policy analysis examined both acts, using criteria that revealed how the acts are similar, and how they differ in key areas. The sample was purposive in nature as these two acts relate most directly to the topic of website accessibility (Slatin & Rush, 2003; Waddell, 2006). For the accessibility evaluation portion of this research effort, the target population for this research was health care websites from the official health information portal provided by the U.S. Government (www.health.gov, 2009). From that portal, the criteria for selection was those websites offered by the federal government that have the intent of providing health care related

21 information to the general public, to include individuals with disabilities. The ―www.health.gov portal identifies those federal websites that relate to key multi-agency health initiatives and activities of the U.S. Department of Health and Human Services (HHS) and other Federal departments and agencies. This site is a key portal to health information from the U.S. government‖ (www.health.gov, 2009). The automated testing study used a purposive sample, examining all federal health care websites listed on the home page of that web portal. The researcher evaluated each website for accessibility by the use of automated software, testing six pages within the website for accessibility errors using the criteria set forth in Section 508. The web content review also used a purposive sample, consisting of all the health care sites listed on the federal health portal. The expert testing method examined the same sample as the automated testing. Detailed discussions on the sampling process used for each method is presented in chapter 3.

Justification of Methods

The design of an evaluation effort should align to the needs of the stakeholder groups and adapt appropriately to the circumstances of the evaluation environment (Feinstein, 2002; Patton, 1997). The multimethod design of this study primarily addressed the accessibility of federal health websites from the perspective of the individual seeking to use the information. However, the research methods used in this study also provided accessibility related findings that are from the perspective of the webmaster, who is responsible for assuring that the website is fully accessible as required by federal law. By using multiple methods for data collection, the study provided a more comprehensive depiction of the accessibility condition, when compared to the use of any single method evaluation strategy (Schutt, 2006). This combination of methods helped address the limitations of the methods individually (Jaeger, 2008; Schutt, 2006 Creswell, 2002). The methods employed to collect data included approaches that are qualitative and quantitative in nature. By including both qualitative and quantitative elements of data collection, the resultant findings were more informative and robust, providing a deeper understanding of the state of accessibility (Miles & Huberman, 1994). The policy review produced a set of qualitative assessments, comparing key characteristics of the two legislative acts. The automated testing and expert testing methods yielded quantified data (typically as counts of conditions or

22 occurrences). The web content review produced qualitative data that converted into quantified numerical counts of occurrences. The selection of methods used within this study assured the provision of data needed to address the research questions previously identified. The automated testing and expert testing provided insight into the accessibility that currently exists within federal health care websites. These two methods yielded quantified measures that represent the accessibility of the websites examined, thereby contributing directly to Research Question 1. The informational value of the question relates to supporting a comparison of the accessibility of this studied group of websites (federal health care websites) to the standards set by Section 508. This information also contributed to determining how this group of websites compares to other groups of websites that have been similarly studied and measured. Research Question 1 sought to determine the common error types and frequency of those errors. This information helped shape the recommendations made by this study. As an example, the shaping would arise from knowing the more common errors types (Section 508 violations), and then placing increased focus on providing solutions for those specific error types. The expert testing further shaped recommendations by suggesting the severity of those errors. In this way, the research provided an indication as to which errors pose the biggest deterrent to users with disabilities. By including expert testing, in addition to automated testing, the findings were more descriptive and robust from this insight into the degree to which the observed errors pose impediments to users with disabilities. Both of these factors (error types and degree of user impediment) influenced the recommendations. Research Question 2 sought to gather information about the restrictions, deterring issues, or barriers that may prevent webmasters from attaining full accessibility. The automated testing and expert testing provided insight into the technical barriers to accessibility that currently exist within federal health care websites. The side-by-side policy analysis and literature review revealed the issues that exist regarding accessibility of web sites, with the side-by-side policy analysis specifically providing insight into the policy related barriers or conflicts that could impede the efforts of webmasters in providing accessible websites. Research Question 3 sought to address the changes that could serve to improve website accessibility. These changes included additional legal awareness, training, expanded budget authority, and improved software tools that automate the features needed to assure accessible

23 websites. Knowing the issues and problems helped to shape recommendations that more directly address accessibility concerns. The side-by-side policy analysis and web content review methods addressed this question by identifying the information policy issues surrounding website accessibility. These two methods provided insight as to how these issues affect the accessibility of federal websites. The side-by-side policy analysis revealed incongruence and inconsistencies that may serve to detract from efforts to attain website accessibility, leading to proposed policy changes to improve the effectiveness of this legislation.

Conceptual Framework

Website accessibility is a new research area stemming from the advent of the Internet and the proliferation of browser technology, followed by the development of assistive devices that aid Internet interaction for those individuals with disabilities. The study of accessibility is characterized by its complexity, as it is a topic that contains aspects of policy issues and technical considerations, and must integrate issues that are user centric (Jaeger, 2006). The relative newness of this research and its inherent complexity can be the reasons that most research efforts to examine online accessibility are at a very practical level (Jaeger, 2006). The most common practical approach to the study of accessibility of health related websites typically has been an evaluation of the website by the use of automated testing tools. Such evaluation efforts have been limited to checks of the website for readily detectable problems that relate to accessibility. Such studies typically have not included any significant focus on conceptual frameworks or theoretical foundations. However, a few studies have made efforts related to developing a conceptual framework for the study of web accessibility. Due to the relative newness of this field of study, the theoretical frameworks used to address web accessibility have been adapted from other academic fields, such as education, disability research, and technology (Jacko & Hanson, 2002). The conceptual frameworks used to examine web accessibility indicate that the problem of attaining web accessibility has many components and influencing factors. Early conceptual frameworks developed from works by Leung, Owens, Lamb, Smith, Shaw, and Hauff ( 1999), Kelly, Phipps, and Swift (2004), Lazar et al. (2004) and Seale (2006). Leung et al. (1999) examined the importance of the role that assistive devices played in attaining

24 accessible websites. Work by Kelly et al. (2004) revealed that web accessibility required a more complex conceptual framework of understanding when looking at e-learning systems. Both studies indicated that web accessibility must use a holistic framework that includes such factors as usability adaptations, learning styles, and the modification of teaching approaches. Lazar et al. (2004) discovered that webmasters play an integral role in determining web accessibility, indicating that their training and current knowledge on the topic of web accessibility are additional factors that play a key role in attaining accessible websites. Seale (2006) integrated these diverse factors into an more robust conceptual framework, where attitudes and beliefs held by key stakeholders are additional influences that must be included in any conceptual representation of web accessibility. These frameworks collectively revealed that web accessibility is influenced by such factors as training, attitudes, technology, laws, organizational policies, and the unique requirements of the intended users (i.e., individuals with disabilities) (Chilson, 2002; Lazar et al., 2004; Stein, 2002, Wade & Parent, 2002). These evolving frameworks have advanced the conceptual representation used for the investigation of web accessibility by incrementally discovering facets and influences that affect web accessibility. Jaeger (2006) integrated these disparate efforts by modifying the Technology Acceptance Model (TAM) to accommodate these various factors. The TAM is a framework for understanding how users adopt and use new technologies (Davis, 1989; Davis, Bagozzi, & Warshaw, 1989). The underlying assumption of the TAM is the fundamental belief ―that perceived ease of use and usefulness can predict attitudes toward technology‖ (Lederer, Maupin, Sena, & Zhuang, 2000, p. 269). The user‘s perception of the ease of use of a given technology, combined with the perceived utility of the technology in a given context, together create a user centric attitude about the technology. The resulting attitude then influences the user‘s behavior, which subsequently affects the user‘s decision to employ or not employ that technology for a specific application. Figure 1.1 provides a visual representation of the relationships expressed by the Technology Acceptance Model. The use of TAM to the study of accessibility in the online environment is a relatively new application (Jaeger, 2006; Jacko & Hanson, 2002; Stephanidis & Savidis, 2001).

25

- Technology Acceptance Model -

Perceived ease of use User‘s behavior

User‘s Perceived utility Attitude of the technology

Figure 1.1: The Technology Acceptance Model.

Applying TAM in Accessibility Research

The conceptual framework used by this research study is a modified TAM. A modified TAM was a guiding conceptual framework in a similar application by Jaeger (2006), with origins in work by Grandon and Pearson (2004). Jaeger‘s work resulted in a modified TAM, which incorporates findings from his website accessibility study. That resultant modified TAM was the conceptual framework used for this study. Figure 1.2 presents a diagram of that modified Technology Acceptance Model used for this study. The modified TAM incorporates an environmental context, composed of the external influences that surround accessibility as defined by Section 508 and by the internal factors that are intrinsic to the organization. The external contextual environment is comprised of the federal information policies that relate to web accessibility, including influences from the voice of the public and the advocacy groups that represent accessibility related interests. The internal contextual environment involves organizational factors and influences that shape the decision-making efforts related to attaining accessible websites as prescribed by the legal mandate. The organizational acceptance of that legal mandate (i.e., Section 508 criteria) results from an assortment of influences that include the

26

- Conceptual Framework -

Modified Technology Acceptance Model

Political Climate Laws Public and Policy Pressure

Pressure to Adopt Section 508 Guidelines

Education and Training of Web Staff

Perceived Perceived Value Costs

Agency Acceptance of the Actual Mission Value of Section 508 Costs Compliance

Other Priorities Staff Time and Interests and Skills

Level of Implementation of Section 508 Criteria User Monitoring Feedback and Enforcement Amount of Types of For Whom Accessible Accessible Content is Content Content Accessible

Level of Compliance with Section 508

Figure 1.2: Conceptual framework diagram.

27 perceived value of attaining accessibility, the agency mission, competing priorities and interest, perceived costs, actual costs, and staffing resources. Together these various influences provide a complex setting that determines the organizational acceptance of the value of providing web accessibility. This modified TAM provided a conceptual framework that integrates each of the three research questions posed by this study. Research questions 1, 2, and 3 are represented in the fourth layer of framework – ―Amount of content accessible,‖ ―Types of content accessible,‖ and ―Who accessible for.‖ This is where the testing methods measure the accessibility found in the federal health care websites. Research questions 2 and 3 address the policy environment for accessibility and the online provision of health care information. These two questions relate to both the contextual and research sections of the conceptual framework by tying the findings of the research to the overarching research context. The modified TAM provided a basic conceptual framework to guide the investigation into web accessibility. In its applicability to web accessibility, the use of TAM is relatively new and still developing. As such, this framework may not yet include all the possible factors and influences that affect web accessibility. This modified TAM does reflect the key findings found so far by the prior studies on web accessibility, which the literature review covers more fully in chapter 2. This study also identified additional key factors and influences that would serve to improve and refine the conceptual framework in its application to understanding web accessibility. The section of chapter 3 (Methodology) entitled ―Relation of the methodology to the conceptual framework‖ presents a more detailed discussion of how this research supports the development of the conceptual framework. This conceptual framework depicted in Figure 1.2 indicates how the many external and internal factors influence the decision making process that surrounds the implementation of Section 508 standards. The key point is that resultant decisions can vary considerably as a function of the factors involved at any given moment, and depending upon the degree of influence (weight) that each factor may exhibit at that specific moment in time. The resultant decision as to whether and how to implement the Section 508 standards will shape the ultimate level of compliance with Section 508 on an agency website (Jaeger, 2006). ―Based on how an agency decides to implement the Section 508 standards, its website will be accessible in certain ways and accessible to individuals with certain types of disabilities‖ (Jaeger, 2006, p. 23).

28 The framework concludes by depicting how these factors directly influence the amount of accessible content, the types of content that are accessible, and the types of disabilities for which content is accessible. A federal entity (represented by the webmaster decision maker) may determine that adopting Section 508 criteria is too difficult or of insufficient value. That determination, while made in good faith, may result from faulty logic or inadequate information. As an example, the lack of training on, or awareness of, the legal requirement that mandates Section 508 compliance may result in the webmaster determining that compliance is of low value. In such a case, the levels of compliance with Section 508 standards, and thereby the levels of accessibility, will likely be low. In contrast, if the agency webmaster determines the adoption to be of great value and a high priority, then the levels of compliance with Section 508 standards, and the levels of actual accessibility, likely will be high.

How the Methods Contribute to Developing the Conceptual Framework

The purpose of this research study was to conduct an evaluation of the accessibility of federal health care websites, and to understand the factors that may impede attaining accessible websites. The evaluation of this group of websites used two methods of accessibility testing (i.e., expert testing and automated accessibility testing). These methods each contributed to the determination of how and to what extent the federal agencies have implemented Section 508 standards. The literature review and the side-by-side policy analysis supported the conceptual framework (Figure 1.2) by examining the context in which agencies implement accessibility features on their websites (Jaeger, 2006). The side-by-side policy analysis examined federal policy as an external pressure for implementing Section 508 standards. The modified TAM presented in Figure 1.2 was the conceptual framework used as the investigative framework to guide the research efforts for this study. The findings of this study informed the modification of this conceptual framework for use in web accessibility research. The newness of accessibility research and the lack of conceptual frameworks accompanying studies of web accessibility make this an area that is ripe for theoretical discussion (Jaeger, 2006). However, the prior successful use of a modified TAM in a similar application (Jaeger, 2006) suggested that using the modified TAM as a conceptual framework provided substantial contribution to this research effort. The modified TAM as a conceptual framework provided a

29 focal point (being the perspective of organizational acceptance of new technology) from which to guide this research project.

Developing the Conceptual Framework for this Study

In Jaeger‘s (2006) study of accessibility for federal electronic government websites, the modified TAM was the most relevant conceptual framework for this type of investigation. Jaeger‘s work identified four additional factors for incorporation into the framework. Jaeger‘s analysis follows:  User feedback – Based on the webmaster questionnaire, comments from users with disabilities would clearly influence compliance with Section 508 standards at many agencies.  Education and training – Many federal web developers clearly are receiving insufficient education and training about designing, testing, implementing, and monitoring for accessibility.  Monitoring and enforcement – There is a need to reconsider the enforcement of Section 508 standards. However, the current citizen-based monitoring and enforcement structure needs representation in the conceptual framework.  Political climate – Accessibility is simply not a political priority to many important government agencies, including the White House, the Department of Education, and the Government Printing Office. The widespread inaccessibility of these federal e- government websites, among others, shows that accessibility is not a priority in this political climate. Jaeger indicated these areas as significant factors to be included in the conceptual framework.

Limitations of the Conceptual Framework

The technology acceptance model (TAM) is robust and influential in explaining adoption behaviors related to information technologies (Lu, Yu, Liu, & Yao, 2003). Lu etal. (2003) provided a comprehensive overview of the development of TAM over the last two decades. That study noted that the TAM provided an appropriate basis for discovering the impact of external

30 variables on internal beliefs, attitudes, and intentions. It is important to note that the application of the modified TAM as a conceptual framework for web accessibility is a relatively new use, and has not been extensively developed or tested through empirical examination. It would be inappropriate to assume that the modified TAM used in this study is mature or complete, covering all possible factors and influences that affect the decision to adopt web accessibility. Further, the modified TAM as used in this study does not indicate the exact nature of the relationships among the factors nor does it determine the strengths or interactions among of those relationships. There may be any number of factors or issues that this modified TAM fails to recognize and explain. Many undiscovered factors or influences affecting the conceptual framework might exist in such areas as the political environment or the internal management within a federal agency. These unknown factors likely may affect how agencies do or do not adopt and accomplish web accessibility. As research efforts and theory-based examinations become more common to web accessibility, that research may discover new issues and reveal the significance each new issue. Such research also may lead to the discovery of how these issues interrelate with one another in forming the combined impact on the technology acceptance decision-making process. For this dissertation, the methods used in this research effort may reveal new factors or issues that affect the Technology Acceptance Model when modified for use as a conceptual framework for web accessibility.

Advancing a Prior Work

It is important to recognize how this study differs from the prior related work by Jaeger (2006). While this study and Jaeger‘s share similar objectives and methods, significant differences in this research effort help to build and expand upon the knowledge base that exists for website accessibility. Jaeger‘s study looked at federal electronic government (e-gov) websites, as e-gov was rapidly growing in importance at that time. This study recognizes the growing reliance that individuals place upon getting health related information via the Internet, and has selected federal health care websites for its examination. Differences exist in the methodologies employed by the studies and in the way that the two studies attempt to refine the

31 conceptual framework for use in the study of website accessibility. This section briefly addresses the differences that exist between these two studies of web accessibility, Jaeger (2006) looked broadly at the information policies related to website accessibility. The intent was to determine if compliance with Section 508 standards would result in an accessible website. This study looked more closely at the two legislative acts that promote website accessibility, being the Americans with Disabilities Act of 1990 and Section 508 of the Rehabilitation Act of 1973. By excoriating these documents in a side-by-side analysis, the intent was to reveal areas of conflict or ambiguity that may deter webmasters from attaining accessibility. This analytic approach identified strengths and weaknesses in these policies that resulted in recommendations for legislative efforts that will more effectively promote web accessibility. To identify accessibility errors, Jaeger used user based testing and expert testing, but did not apply automated testing. This study used expert testing and automated testing, but did not employ user based testing. Jaeger indicated that automated testing had numerous potential problems that made it inappropriate for a detailed examination of web accessibility. However, Jaeger notes that automated testing provides an easy way to get an indication of accessibility problems across a large number or websites, which aligns with the objectives of this research effort. Jaeger‘s use of user based testing and expert testing supported his objective of getting a detailed look at web accessibility. For that reason, this researcher considered both methods for use in this study. However, at the time of this research, there were no available facilities to conduct user based testing. As such, this researcher decided to use automated testing combined with expert testing, which afforded the opportunity to examine web accessibility in both scope and depth (detail). Jaeger used a modified Technology Acceptance Model as the conceptual framework for his research. His study found important elements that enhanced the understanding of the factors depicted within that framework. In a similar fashion, this study also examined websites in order to assess the value of the modified TAM model for the unique group under study. This study extended that examination to look more specifically at agency internal factors related to political issues and managerial/situational factors that affect accessibility. By looking more closely at these areas, this study clarified and improved the understanding of the factors in the conceptual framework identified as ―Other priorities and interests.‖

32 In summary, Jaeger‘s work provides the foundation from which this study builds and expands in the following areas:  Focused on the identification of the impediments that deter webmasters from providing accessible websites.  Used automated testing to assess the accessibility of this entire group of websites.  Examined new factors in the conceptual framework (agency political and managerial/situational issues), and  Excoriated the two leading legislative acts that attempt to promote accessibility, with the intent to provide specific policy recommendations. These unique features of this study provided a better understanding of web accessibility, improved the conceptual framework, developed best practices, and resulted in policy recommendations that may improve website accessibility for all users.

Project Timeline

This research project had four distinct phases. Phase 1 of the project addressed pre-data collection steps that included the presentation of an overview of the study (chapter 1), a literature review (chapter 2), and the presentation of the methodology of the project (chapter 3). Phase 1 or proposal preparation constituted the general preparation required to prepare a Dissertation Proposal, which included such activities as fundamental planning, problem definition, determining methodological approaches, developing project management plans, and drafting institutional research approval documents. Table 1.3 presents the timeline for Phases 2 - 4 of the study: planning and project preparation, data collection, and presentation of results. Activities in Phase 2 or pre-data collection included overall planning of the project, beginning the side-by-side policy analysis, preparing data collection instruments for each method used in the study, and pretesting the data collection instruments. Additional activities included efforts to coordinate Internal Review Board documentation, refine the design of testing procedures, resolve the practicalities of executing the methodologies, and begin execution planning. Phase 3 or data collection initiated the data collection efforts to obtain the measures of accessibility. Phase 4 or dissertation preparation included the analysis of results and the

33 dissertation preparation. The timeline provided in Table 1.3 allocates workload efforts in increments of weeks.

Table 1.3: Project timeline. Project Timeline in Weeks Project Planning, Data Collection and Presentation Activities Phase 2: Phase 3: Phase 4: Planning/Project Project Data Presentation of Results Preparation Collection Efforts and Dissertation Preparation Weeks Weeks Weeks Weeks Weeks 1-8 9-12 13-24 25-32 33-52 Side-by-side policy analysis Pre-data Collection Web Content Review Pre-data Collection Automated Testing Pre-data Collection Expert Testing Analysis of Data Collection Efforts Dissertation Preparation .

Summary

Chapter 1 addressed the details of this study‘s purpose, goal, and objectives, which shaped the research questions. The chapter presented a background on the need for research and its significance in the area of federal website accessibility. This chapter also examined the increasing importance and prevalence of health care websites and provided an overview description of the related trends in health care information seeking. The growing use of online services to provide content, products, and services adds to the importance of examining this topic area. The focus of the discussion narrowed to address how these factors relate to the target user population for this research, which is the group of users with disabilities. This chapter presented a brief overview of the study methods, with emphasis on the value derived from a multimethod approach to this investigation. By using multiple methods, the intent was to produce a richer and fuller understanding of the accessibility that exists within federal health care websites. This research provided insight into the perceived problems and issues that impede the attaining of full accessibility as required by Section 508 of the Rehabilitation Act of 1973.

34 This chapter presented a discussion of the theoretical framework used to guide the research effort. As the study evolved, the findings served to modify and improve the framework as a conceptual representation for these types of investigations. Improving the TAM was a significant contribution to the theoretical body of knowledge in this area of research, given the relative newness of accessibility research and the limited number of empirical research investigations that have examined this topic area in this rigorous manner. As an overview, Figure 1.3 provides a summary visual depiction of this research project.

35

Improving the Accessibility of Federal Health Care Websites

Purpose: The purpose of this research study was to conduct an evaluation of the accessibility of federal health care websites for individuals with disabilities, and to understand the factors that may impede attaining accessible websites. Goal: To improve the accessibility of the federal websites, which provide health care information.

Research Questions Are federal health care websites compliant with Section 508 requirements? What are the issues and barriers that affect attaining accessibility? What changes would promote websites that are more accessible?

Multimethod Approach

Side-by-side Automated Expert Testing Web Content Policy Analysis Testing Identified Review Assessed the role Supported underlying Assessed the quality of that policy plays. conclusions that technical web accessibility problems. are more general. Statements.

Analyze Data and Synthesize Findings

Recommendations

Policy Options Error Mitigation Problem Solutions Generated Developed solutions Provided measures to recommendations to that target the most resolve the formulate new and common and severe impediments that improved policy. errors. hinder accessibility. procedures and

Figure 1.3: Overview of research proposal.

36 CHAPTER 2

LITERATURE REVIEW

Understanding web accessibility as provided by federal agencies requires a broad understanding of the disability policy environment, and an awareness of the issues and factors that affect implementation of the applicable guidance. The intent of this chapter is to conduct a literature review that introduces selected issues and factors from the diverse and complex areas that affect web accessibility. The discussion identifies the issues related to web accessibility and provides an overview of how these factors impact efforts to attain website accessibility. This chapter begins with a brief introduction of the historical development of web accessibility, as a part of the Internet phenomenon and the civil rights issues that enter into the mix. The overview provides a framework for comparing web accessibility across various groups or classes of websites. This chapter also identifies the wide-ranging set of issues and challenges related to providing health care information via federal websites. These issues include the:  Accessibility of health care websites,  Policy developments and precedents,  Evaluation of web accessibility, and  Identification of federal agencies for policy and oversight. The final section of this chapter presents sources of guidance that serve to promote attaining websites that are more accessible.

Historical Overview of Web Accessibility

The Internet has had a profound impact on the world that we inhabit. Perhaps more than any development since the discovery of electricity, the Internet has the potential to provide and liberate (Ball, 2008). However, there exists a fundamental responsibility for web designers to ensure it does not prevent and restrict (Ball, 2008). Websites provide a medium for the presentation and exchange of information that is unprecedented in history. This medium has an accompanying moral responsibility in that people should not face undue discrimination in accessing content because of their disability (Sloan, 2008). Sloan identified social motivation as

37 the most important of the four core arguments to support advocating for web accessibility. The other three arguments are economic, technical, and policy issues. From the social motivation perspective, web accessibility is a desirable goal as it promotes the inclusion of individuals with disabilities in order to reach a wider audience than would otherwise be achieved. Individuals with disabilities are often a group that experiences social exclusion due to the barriers confronting their efforts to perform day-to-day tasks, where such tasks could include obtaining government services, or accessing health care information (Sloan, 2008). These barriers may be so severe or detrimental that they deny access to the information or services sought. By committing to web accessibility, the provider of a website becomes a strong proponent of social responsibility (Waddell, 1999). The economic factors that affect web accessibility arise primarily from three sources (Sloan, 2008). First, providing for accessibility is an incremental cost to the initial design and development of the website. Second, web technology and accessibility policy areas are constantly changing, evolving, and converging. This ever-transforming environment requires the periodic review of websites to assure ongoing compliance with accessibility guidelines, resulting in recurring costs. While the implementation of accessibility may incur initial and ongoing costs, these provisions will likely have a positive long term cost benefit for an organization (Sloan, 2008). Thirdly, building accessible websites assures the widest possible range of individuals can access and use the web based resources or services provided. By achieving the maximum audience, there is a clear potential economic benefit in reaching this extended customer base. While these cost expenditures may be a major deterrent to assuring accessibility, Sloan (2008) pointed out the importance of assessing the cost savings (or cost avoidance) that may result for accessibility compliance. Such financial benefits may result from:  Reduced maintenance cost, due to improved design efficiency, and  Avoiding possible legal action from unlawful discrimination lawsuits. Organizations must be aware that lawsuits may not only be financially damaging, but also may result in some very negative publicity that will be difficult and expensive to overcome. There are distinct technical benefits that closely relate to the economic benefits of providing accessible websites. Sloan (2008) identified four key technological factors that impact the decision to support website accessibility:

38 1. Search engine optimization: accessible web design helps to ensure the accurate indexing of an organizations‘ website. This is a key consideration when a business wants their website to appear at the top of the results of a web search. 2. Accessible web design makes for a more efficient display of information, improves navigation, and makes the site easier to use. Further, sites that are more accessible make more efficient use of the web server (performance enhancement), and improve the loading and response times for the end user (higher user satisfaction). 3. Designing for accessibility tends to promote the separation of content functionality from appearance issues. This separation results in improved design features that reduce maintenance and update requirements, thereby saving many hours of workload on the web management team. 4. Current accessibility design techniques promote the use of standards and technologies that improve interoperability across the existing and emerging browser platforms. Such design accommodations, once built into the initial software, minimize the work needed to adapt websites to function effectively, as the new browsing technologies gain wider acceptance. These technical benefits may significantly contribute to the cumulative cost savings that are possible by adopting accessible design at the onset of a website development effort. When these cumulative cost savings are included in the total project lifecycle cost, the net cost of providing accessible websites is often greatly reduced (Sloan, 2008). Policy factors that affect website accessibility exist in many countries and include the legislation that encourages or requires compliance with explicit criteria aimed at assuring web accessibility (Waddell, 2006). Within the United States, the two most directly applicable pieces of legislation are the Americans with Disabilities Act (ADA) and Section 508 of the Rehabilitation Act (Slatin & Rush, 2003). The ADA serves to promote the basic rights of individuals with disabilities and to protect them from unjustified discriminatory practices. Section 508 more precisely addresses website accessibility, requiring all federal entities to provide websites that meet the standards identified within that legislation. The political motivation related to requiring website accessibility stems from the concepts of civil rights, equity, and fairness that have been an integral part of the basic structure of the United States democratic republic since its formation. These concepts provide the

39 fundamental underpinnings for assuring that federal websites provide content and services that are available to everyone. The passage of the ADA and Section 508 had the intent of eliminating discrimination against individuals with disabilities, including technologies available via the Internet (Slatin & Rush, 2003). The U.S. federal government now has more than 22,000 websites with more than 33 million individual web pages (Jaeger, 2004; Bednarz, 2002). Providing government services via the web is quickly becoming "simply the way things are done" in the United States (Howard, 2001, p. 6). This means of conducting government to citizenry interactions is having a major impact on several major areas of democracy in the United States. These areas include access to political information, communication about political issues, communication with government officials and elected representatives, and the delivery of government services (Nugent, 2001). These processes (if fully web accessible) would serve to erase many of the prior barriers that individuals with disabilities faced (Jaeger, 2004). The development of the Internet and its use within the federal government presents a whole new range of opportunities for the federal government to provide information and services directly to citizens through web based services. These web based services permit the government to take advantage of this new "technology… to enhance the access to and delivery of government information and services to citizens, businesses, government employees, and other agencies at the federal and state levels" (Hernon, Reylea, Dugan, & Cheverie, 2002, p. 388). At the same time, these new web based services provide the citizenry with the opportunity to have a greater interaction with the government, affording them a greater voice in government, and to receive more efficient and effective services from the government. In this way, the public becomes "more informed about government laws, regulations, policies, and services" (Muir & Oppenheim, 2002, p. 175). From the standpoint of this nation‘s democratic ideals, these Internet based government resources represent a new form of government, but at the same time, it is an extension of what we know government to be. This new form, or medium of delivery, of government services provides a uniquely clean slate upon which the public can interact a way that is completely free from discrimination (Jaeger, 2004). Jaeger points out that ―as long as an individual can access the e-government [electronic government], it is very difficult for discrimination to occur against most groups. However, individuals with

40 certain types of disabilities, such as vision impairments, hearing impairments, learning disabilities, and mobility impairments, are still completely vulnerable to discrimination in the e-government when the websites are not fully accessible‖ (Jaeger, 2004, p. 19). If these web based federal services are not fully accessible to individuals with disabilities, then this new electronic government is not meeting its basic constitutional obligation of providing equitable and fair access for all.

Website Accessibility and Compliance with Section 508

Studies worldwide have examined websites for accessibility. Typically, these studies indicated poor levels of Section 508 compliance, and even top-level government websites had major accessibility problems (Johnston & Helms, 2008; Jaeger, 2006, 2004; West, 2004, 2002). The Disability Rights Commission (2004) examined over 1,000 government websites within the United Kingdom, with findings indicating that 81 per cent had significant accessibility problems. Marincu and McMullin (2004) discovered poor accessibility in government websites belonging to member countries throughout the European Union. These broad findings indicate that assuring the accessibility of governmental websites does not appear to be a priority issue, and is a problem with worldwide scope. Within the United States, compliance with Section 508 also has had a poor showing, but with a promising outlook. West (2002) found that the accessibility observed in government websites had increased from 11% in 2001 to 82% in 2002. When applying criteria that was more stringent in measuring accessibility, the raw results were lower but did trend upward, increasing from 14% in 2004 to 23% in 2006 (West, 2006). These figures may indicate that the amendment of Section 508 in 1998 inspired a number of state and local governments to ―look at accessibility as a genuine opportunity to bring government resources and services to an increasing number of people‖ (Patterson, 2002, p. 10). A few recent studies have more rigorously examined the accessibility of U.S. federal websites (Jaeger, 2006; West, 2006; Stowers, 2002). The studies by Stowers and West indicated poor performance by federal agencies in providing accessible websites. Jaeger (2006) examined the accessibility of federal websites that provided electronic government, finding that every

41 federal website examined had accessibility problems, with six of the ten sites having significant problems. Most early studies on website accessibility have used rather simple automated tools for determining the accessibility of a group of websites. Recent studies of accessibility are evolving in sophistication by examining websites in greater depth and breadth. It is becoming increasingly common to find stringent evaluative criteria and exacting standards used to assess accessibility. By using exacting standards, a much better sense of the true scope of the accessibility problem is possible (Jaeger, 2006).

Accessibility of Federal Websites

This section presents a brief overview of the major studies that examined the accessibility of federal government websites. Most of the studies used automated software and found relatively poor compliance with the criteria set forth in Section 508. Typically, the approach in these studies was to test a group of federal websites to calculate the number of error free sites, compared to the total number of sites. The following is a chronology of key studies performed since the enactment of Section 508 legislation that provide indications of the accessibility of federal websites. Jackson–Sanborn, Odess–Harnish, and Warren (2001) evaluated a random sample of 100 federal government websites, using an automated software-testing program. The study found that only 60% were fully accessible using Section 508 criteria. The study implies that much of the federal government‘s Internet based content is not accessible to a significant portion of the population. The study‘s recommendations included the necessity to conduct ongoing research and testing to monitor the accessibility of these websites. Another recommendation pointed to exploring the need for providing federal webmasters with additional training and building an awareness regarding the legal mandate for compliance with the existing federal accessibility policies. Stowers (2002) rated 148 federal websites for accessibility using an automated software- testing program. The study included executive, legislative, and judicial sites, finding that only 13.5% of the sites studied were fully accessible. Stowers recommended monitoring on an ongoing basis and pointed out the need to elevate the attention given to accessibility of federal

42 websites. The report urged web designers and webmasters to give greater priority to accessibility issues, including monitoring updates and making organizational policy changes to ensure that all content meets standards. Takata, Nakamur, and Seki (2004) used data collected by Ishikawa (2001) to evaluate the accessibility 10 major U.S. government websites using an automated testing software product. The testing examined for violations of the criteria specified in Section 508. Their findings indicated that only 57% of the web pages examined did not have errors. That same study tested 10 large U.S. based corporate websites, finding only 20% of the web pages had no errors. The indication is that for U.S. websites, the government agencies performed better than non- government organizations in providing accessible websites. Ellison (2004) evaluated the accessibility of United States government web pages, using an automated software-testing product. Ellison reviewed the home pages of fifty U.S. government agencies for accessibility using Section 508 guidelines. The study purposively selected 50 home pages representing well–known agencies within the Federal government. Of the 50 home pages evaluated, only 11 (22%) had no Section 508 related accessibility errors. Ellison‘s study supported his premise that the Section 508 mandate of full accessibility has not been achieved. Ellison (2004, Section: Conclusions) concluded that ―as demonstrated by this study, as well as in earlier studies, even organizations [as the federal government] that are motivated and mandated to provide 100 percent access have not achieved this goal. …if the U.S. government cannot ensure accessibility of its own web pages, it cannot credibly advocate that other organizations provide accessible information. This study establishes that the U.S. government has not met its accessibility goals.‖ King, Thatcher, Bronstad, and Easton, (2005, p. 519) reported that ―only 22 percent of the websites for United States government agencies, which have been mandated to be accessible by Section 508 for over three years, meet Section 508 requirements.‖ That report summarized prior research by West (2003). The West study presented a fourth annual update on the features that are available online through American state and federal government websites. West examined 1,663 state and federal government sites using automated testing software. Only 34 % of state and federal sites in the United States meet the W3C disability guidelines and only 24 %

43 satisfy Section 508 of the U.S. Rehabilitation Act of 1973. Federal sites (47 %) were more likely to meet accessibility standards than states (33 %). The results obtained by West (2003) were particularly significant as this was the first year that West actually tested the sites. In prior years, accessibility was determined by merely examining the website for statements that claimed accessibility. Such indications or statements could be obtained by looking at whether sites displayed TTY (Text Telephone) or TDD (Telephonic Device for the Deaf) phone numbers which allows hearing-impaired individuals to contact the agency by phone, provided text labels for graphics, or claimed that they were disability-accessible. That approach had the obvious disadvantage of not providing an actual test of accessibility as accomplished for the 2003 report by West. Jaeger (2006) conducted a more comprehensive investigation of the accessibility of federal electronic government (e-government) websites. Using a multimethod approach, Jaeger‘s study examined a broader depth and breadth of issues related to e-government website accessibility. The compliance criteria used was Section 508 based, but employed expert testing and user testing for a more user centric accessibility evaluation. Jaeger found that every federal website examined had accessibility problems, with six of the ten sites having significant problems. The studies just addressed in this section reveal low levels of web accessibility compliance among U.S. government sites. These studies found poor accessibility by testing for compliance using automated testing software. The following section presents other key accessibility studies that have delved into the possible issues and impediments that may be deterring compliance with the Section 508 criteria. Lazar, Dudley-Sponaugle, and Greenidge (2004) conducted an exploratory online survey of 175 federal web mangers to gather their opinion covering a wide range of web accessibility related topics. The study sought to assess their knowledge of web accessibility and to gather their perceptions of when and why websites should or should not be accessible. The research wanted to assess the effectiveness of existing web guidelines, to determine how information technology workers interact with those guidelines, and to ascertain their reasons for implementing accessibility. The goal of that research was to learn more about why webmasters do or do not make their websites accessible. Even though accessibility tools and guidelines are available to help in building accessible websites, and given that public policy promotes web accessibility, the

44 researchers were surprised that so many websites were inaccessible. Webmasters cited roadblocks to accessibility that included the lack of time, training, managerial support, client support, adequate software tools, and clear accessibility guidelines. Some webmasters staunchly objected to the idea that websites should be accessible, and would make websites accessible only if forced to do so. Examining a random sample of 32 U.S. government websites from 1997-2002, Hackett, Parmanto, and Zeng (2005) analyzed the effects that technology has on web accessibility. In comparison to non-government websites, the government websites maintained relatively the same in accessibility while increasing in complexity. The more general conclusion covering the broad scope of websites was that the increasing complexity, oftentimes caused by adding new technology to a web page, inadvertently contributes to increasing barriers to accessibility for persons with disabilities. An important indication from that study is that federal websites provide higher accessibility, which may result from the federal policies in place that promote accessibility (Smith, 2009). Jaeger‘s (2006) study further examined the accessibility of federal electronic government websites by looking at related laws and policies, and by surveying webmasters regarding their perceptions about accessibility. This study was perhaps the most comprehensive and revealing efforts to explore accessibility, providing an in depth look at the accessibility of a specific group of federal websites. The key findings from the Jaeger‘s (2006, p. vii) dissertation include:  Accessibility compliance varies widely between websites,  Agencies lack a standardized approach to Section 508,  Agencies‘ perceptions about the accessibility of their sites are not entirely accurate,  Compliance with Section 508 could be increased with funding and education for web developers, and  Accessible e-government (electronic government) sites remain an unfulfilled goal. That study provided ―a robust, user-centered portrait of the levels of accessibility of e- government websites, reasons for the current levels of accessibility, and perceptions about accessibility‖ (Jaeger, 2006, p. vii). In composite, these studies of issues related to accessibility explore a myriad of factors and concerns that influence the accessibility of federal websites. These studies have also provided indications of where research could further identify and explain the impediments that

45 exist to attaining website accessibility. While these studies have formed a foundation that will help shape research efforts, there remains a great deal of potential for further examination of the accessibility of federal websites. It is important to point out that Section 508 compliance criteria became public law in 1998, and these cited studies of accessibility took place from 2002 through 2006. Sufficient time had elapsed to allow the making of significant progress towards improving accessibility. However, the findings of the cited studies show poor compliance with the provisions of Section 508. It is imperative that researchers continue to conduct accessibility studies in order to remain vigilant in measuring progress towards attaining full compliance with Section 508, lest these low levels of accessibility perpetuate. Failure to attain website accessibility will have the result of excluding people with disabilities from vital resources, which are becoming increasingly important to that group (Malone et al., 2004). Another benefit of further research efforts will be to identify the underlying problems that impede the implementation of the provisions of Section 508. Only from such additional research can a body of work be built that will focus on providing clear and concise recommendations towards overcoming any impediments in the way of attaining this noble social goal.

Policy Issues Related to Web Accessibility for Federal Websites

Several studies have addressed the accessibility of federal websites, but most of these studies did not thoroughly address the policy related issues involved. This section presents an overview of the key studies of website accessibility that delved into the policy related aspects, and addressed the complexities that stem from federal policies. Presented are studies that examine important legal issues and identify policy related recommendations for increasing federal website compliance with Section 508. Jaeger‘s (2006) study of the accessibility of electronic government websites employed a number of methods to collect data: a policy review, expert testing, user testing, and an online survey of webmasters. The multiple method approach involved methods that offered unique strengths in examining the topic area and provided multiple perspectives in the attempt to understand the complex accessibility issues. The policy review portion of Jaeger‘s work explored Section 508 to determine if compliance with the accessibility standards defined within that law

46 would result in an accessible website. The thesis of the examination was that if by fully complying with Section 508 criteria for accessibility does not result in an accessible website, then the context of the accessibility of electronic government websites rests upon flawed assumptions. The approach used by Jaeger was to trace the origins of the Section 508 standards, examine their coverage and content, and then make a detailed comparison of these standards to other accessibility standards. Jaeger concluded that Section 508 standards provide the most comprehensive set of standards for assuring the accessibility of electronic government websites. That study further indicated that a website that met all of the Section 508 criteria would be ―accessible to individuals with all or almost all of the different disabilities that can be negatively affected by technological barriers to access in cyberspace‖ (Jaeger, 2006, p. 82). Jaeger‘s work revealed many examples of federal agencies resisting compliance with Section 508 stemming from a misunderstanding the goals and standards of Section 508. Some agencies seemed to view Section 508 standards as an unnecessary burden. However, Jaeger found no indication of any inherent problems with the standards provided by Section 508. ―The standards of Section 508 do not seem to be a problem that is causing lower levels of website accessibility‖ Jaeger (2006, p. 168). Jaeger‘s focus was concentrated on the standards identified within Section 508, but did not examine the entirety of the legislative act. As such, the opportunity exists to delve into an exploration of the provisions and structure of this amendment to the Rehabilitation Act of 1973. Sloan (2008) notes that Section 508 of the Rehabilitation Act requires federal agencies to procure and provide technology that meets the guidelines set forth in that legislative act. Sloan indicates that the act falls short, in that it does not require industry to produce it. The intended effect is that organizations wanting to conduct business with the potentially lucrative federal entities will act to meet those standards. Howell (2008) echoes that opinion and notes that the federal government is the largest procurer of information technology products and services in the US. Sloan suggests that in this way, Section 508 inspired many commercial web technologies to improve for accessibility. That improvement, initially mandated by the federal government, has resulted in accessible products and development tools being much more commonplace in the broader commercial market.

47 Golden and Buck (2003) examined web accessibility policy and found evidence of confusion among webmasters with regard to Section 508 and the Americans with Disabilities Act (ADA). Golden points out that Section 508 provides specific criteria for promoting website accessibility. The ADA is not so specific. The criticism of the ADA relates to the lack of specificity in its language, leading to conflicting interpretations. Some interpretations take the position that access to web based content is achievable through mechanisms other than direct web accessibility. Golden indicates that a weakness of the ADA is that is fails to directly address website accessibility or indicate the need for all websites to adopt Section 508 standards as a matter of public accommodation. In his look at state agencies, Golden puts forth two recommendations that would serve to improve web accessibility within government organizations.  Training - training will assure a consistent level of expertise and awareness of the legal requirements to comply with Section 508. Also ongoing training is needed to keep abreast of the changing technologies that impact web accessibility  Guidance - state agencies often mandate contractors to assure the accessibility of the products and services provided to the state. The verification of accessibility requires high levels of technical knowledge supported by evaluative protocols that serve to assure thorough verification of compliance. There is a need for such guidance that identifies a clearly understandable and executable process that results in a valid, reliable, and consistent assessment against Section 508 standards. In times of economic downturn, Golden warns that state and federal government agencies must share information and ideas about successful practices in web accessibility. Such sharing of resources and best practices will ensure the most effective use of public funds and serve to more efficiently attain the goal of government-wide accessibility. Supporters of efforts to revamp Section 508 have held seminars and review efforts that had resulted in key recommendations. One supporting association, AT508.com (AT508, 2010) on June 20, 2009, hosted its second ―National Town Meeting to Rescue Section 508.‖ The intent of this nation-wide meeting was to build awareness of the inherent weaknesses in the federal government‘s implementation of Section 508. The Town Meeting resulted in the following recommendations for improving Section 508:

48  Accessibility czar – Appoint an individual with the authority and resources to lead accessibility within the federal government.  Enforcement office - Create a single government entity with the authority to promote and enforce the standards identified in Section 508 across the entire federal government.  Agency report cards – independent assessments would help to assure and enforce compliance.  Accountability systems – the General Services Administration (GSA) and the Department of Justice (DOJ) are the administrators of the legal provisions stipulated in Section 508, but there is inadequate process related guidance on how these agencies will ensure accessibility. That process must also address the means to hold agencies accountable for attaining and maintaining compliance.  Education and training – contracting agents, 508 coordinators, vendors and webmasters need to be aware of the legal and ethical requirements of Section 508.  Undue burden – clarification and guidance would help to assure that an enforceable definition of this clause is available for federal entities. These recommendations indicate some of the weaknesses in the current provisions of Section 508. The legislation fails to provide an effective enforcement provision that includes a consistent and reliable set of usability, quality assurance, and certification measures. In aggregate, these recommendations allude to deficiencies that result in confusion among the federal agencies leading to lack of compliance with the Section 508 standards. Administrators and managers often blame the lack of clear guidance as to what does or does not constitute ―providing accessibility.‖ Jaeger (2004) examined policy issues related to disability law, with a focus on Section 508. That review of the legislative environment surrounding disability rights, with respect to the development of federal electronic government, resulted in a set of policy suggestions that would serve to meet more fully the expectation of equity and fairness held by individuals with disabilities. Jaeger suggests the following steps that the government should take to provide a more fully accessible government in this age of the Internet:  Public accommodation – The Department of Justice must intercede in the controversy over whether the Internet is a public accommodation, and is therefore

49 accountable to the standards identified within the ADA. If the standards of the ADA extended to cover all websites, then all websites would then have to be fully accessible to individuals with disabilities. Yet the DOJ has not acted to provide that important interpretation.  Best practices – The Access Board and the U.S. General Services Administration Center for Information Technology Accommodation (CITA) are the federal agencies responsible for the implementation of Section 508. These organizations should examine state governments to discover accessibility laws and initiatives that more effectively support or provide accessible websites.  Congressional earmarks – Congress should designate prioritized funding aimed at establishing programs that would promote compliance with Section 508. This would provide much needed funding, and give additional credibility to the government‘s commitment to accessibility. Such a boost to the commitment would help to overcome institutional resistance or inertia.  Government-industry consortium – Government must align efforts with commercial vendors to ensure a greater proliferation of accessible products and services. The government is the largest purchaser of technology products and must leverage that position to encourage vendors to create more accessible information technology. Requiring commercial vendors to build-in accessibility from the start is much more efficient that refitting products for accessibility at a later date.  Audit and enforcement - The federal government must assume a more proactive role to assure implementation of the law. Additional oversight would more efficiently address areas of confusion regarding the requirements for accessibility. Additional enforcement of compliance with the provisions of Section 508 could provide a much- needed impetus to organizational inertia, and more effectively evolve a clear and consistent set of supplementary procedures and guidelines for implementation.  Inform and educate - Education efforts by the government could inform the public as to their rights under Section 508 to have accessible websites. The public could in turn provide feedback regarding the actual accessibility of websites, informing the government as to problems that remain unaddressed.

50 Jaeger makes the point that a fully accessible federal government is only achievable through a concerted effort that elevates the priority of attaining website accessibility. The success of these federal efforts will be contingent upon obtaining and integrating the support of commercial vendors and feedback from the public. Jaeger (2004) indicated that prior to Section 508, federal laws related to disability rights had dealt with accessibility primarily in terms of physical access to structures and to intellectual access to printed materials. Section 508 took the step to ensure that technologies provided by and procured by the federal government were accessible. In his policy review, Jaeger (2006; 2004) finds that Section 508 is not the only law that addresses issues of accessibility, but it is the law with the best-defined and measurable accessibility standards. ―Section 508 is the only [law] with sufficiently detailed regulations to serve as a guide for creating accessible websites or as a measure of the accessibility of [government] websites‖ (Jaeger, 2006, p. 82). From that analysis and conclusion, this study will focus on using the standards identified in Section 508 for its determination of the accessibility of websites.

The Internet and Health Care Information

The rapid growth of the Internet presents a strong point for considering this period in time to be the advent of the Information Age. However, the speed of its arrival has left much of the medical profession ill prepared for the associated changes driven by this new technology (Coiera, 1996). The Internet affords the public an unprecedented supply of information on health care and medical related information, with varying levels of quality and relevance (Coiera, 1996). The widespread availability and the increasing use of the Internet indicates that the provision of health care information will no longer be the exclusive domain of the health care professional. Access to the Internet based health care information is already widespread in many populations. As early as 1994 saw that 46% of patients in one California clinic had access to email, with 89% of them able to access the Internet at their place of work (Fridsma, Ford, & Altman, 1994). Coicera (1996) pointed out that in some locations in the United States, proportionally more patients than doctors had access to the Internet. Patients will benefit greatly from having Internet based health care information. Web based electronic forums already allow patients to discuss their individual experiences and gather

51 a wealth of information from other individuals who may suffer from a similar malady. Coicera (1996) found that there were Internet sites offering health care related consulting and advice on a fee for service basis. Coicera also found that many organizations provide free access to such information. A growing number of Americans are using the Internet to find health information (Fox, 2005; Fox & Rainie, 2000). The Pew Internet and American Life Project (2007) indicated that within the Unites States, Internet access has developed rapidly to the point where an estimated 73% of American adults are online. Hesse, Nelson, Kreps, Croyle, Arora, Rimer, and Viswanath (2005) indicated that 63.0% of the US adult population in 2003 reported ever going online, with 63.7% of that online population having looked for health information for themselves or others at least once in the previous 12 months. Physicians remained the most highly trusted information source to patients, with 62.4% of adults expressing a lot of trust in their physicians. Accompanying this trend is the growing public endorsement of the Internet as a valuable tool in making health related decisions (Pew Internet and American Life Project, 2007). To claim that there is an ongoing worldwide revolution in health information is not an overstatement. Often health information consumers show up at their physicians‘ office armed with computer printouts, copies of news stories, and many questions (Cline & Haynes, 2001; Fox & Rainie, 2002). Fox and Fallows (2003) found that an estimated 93 million Americans had reported searching for health topics on-line, making health information searching the third most popular online activity after email and researching a product or service before buying it. For individuals searching for health care information on-line, 68% have suggested that the on-line information had at least some impact on decision making either for themselves or their loved ones (Fox & Rainie, 2002).

Cline and Haynes (2001) identified several advantages of seeking information online, including widespread access, anonymity, potential for interactivity, social support, and the ability to tailor information to one‘s needs. Further, there is an abundance of information available online, with an estimated 70,000 websites (Cline & Haynes, 2001) constituting 2% of all sites on the web (Powell & Clarke, 2002). The information gathered from online searches appears to have a notably positive influence on health behavior (Fox & Ranie, 2002; Morahan-Martin, 2004).

52 When asked where they preferred going for specific health information, 49.5% reported wanting to go to their physicians first. When asked where they actually went, 48.6% reported going online first, with only 10.9% going to their physicians first. The study found a major shift in the way in which patients acquire health and medical information. More patients are looking for information online, before talking with their physicians. From the Internet‘s early beginning to this current time, providing readily available health care information in an online manner has many obvious benefits, but also poses significant challenges for both the users and suppliers of this type of information (Sellu, 1996). The power of the Internet as an information source lies in its availability from practically any location on the globe, yet this growing medium of distribution is largely uncontrolled. That uncontrolled nature allows the offering of information with varying levels of quality. When health care is at the heart of the user‘s quest for knowledge, these widely varying levels of quality can be of great concern. Quality issues can surface in many areas. One such area relates to the timeliness of information, as assuring that the information indicates the most recent medical findings and practice guidelines. Health care is a field of rapid advancements driven in a large part by research and technological advancements. The Internet allows patients to have timely access to the latest research findings, and supports broader participation in the field- testing of treatment programs. When patients have the option to participate in their treatment programs, the Internet can provide a rich source of information on treatment progress and can indicate alternative treatment options (Coicera, 1996). The Internet can provide a wealth of good information, and its growing use for health care purposes may lead to a more informed patient base. Patients‘ knowledge may increase so that the patient requires more of the doctor‘s time to discuss treatment options and to determine mutually the best treatment course (Coicera, 1996). This increased load on the existing health services resources may cause a strain until the industry reshapes itself to meet this new level and type of demand (Coicera, 1996). The nature of the Internet is that it affords the public with the unique value of providing unparalleled rapidity in the dissemination of information. That speed of dissemination poses a significant challenge to web based health care providers. There is a mismatch in the speed with which new medical research results can effectively disseminate compared to the length of time required for careful peer review. That speed of delivery that the Internet provides is actually not

53 a liability. The reality is that this capability offers a potential solution to some of the more complex problems facing the health care system. Getting the findings of government research and scientific breakthroughs out to the broad and diverse medical practitioners has been a noted problem for some time (Coicera, 1996). As clinicians gain widespread and routine access to the Internet, they obtain the latest results and best practice guidelines in a timely and effective manner. The Internet has emerged to be a major new source of health information exhibiting the potential for improved service in the provision of health care from the patient‘s point of view (Jellinik, Lambden, & Lilley, 2000; Hardy, 1999). Internet based health care information altered the nature of medical information delivery and created a new generation of better informed service users who challenge the traditional doctor-patient relationship (Hardy, 1999; Malone et al., 2004). The biggest impact occurs at the primary care level where the patient-physician relationship is the most prevalent (Malone et al., 2004; Florin & Coulter, 2001; Guadagnoli & Ward, 1998). The impact of the Internet on the way that patients seek health care information can only increase (Malone et al., 2004). The Internet is driving a dramatic change in information access and delivery, but this change brings major implications for the delivery of health care (LaPorte, 1994). One such change is the increased need for providers of web based health care information to become more involved in the issues relating to the communication and accessibility of the information. The newness of the technology, combined with the rapidly occurring developments in this field, pose a demand for careful attention to ensure that this information is accessible and understandable by all individuals, including those individuals with disabilities.

Evolution of Disability Rights Legislation

The trail of subsequent amendments to the United States Constitution displays a progression of victories promoting equality. While these amendments resulted from long fought battles, they represent major steps forward for racial and gender equality. However, the granting of such equalities to individuals with disabilities did not begin to occur until the 1970s. Section 504 of the Rehabilitation Act of 1973 is widely recognized as being the first major disabilities rights law (Scotch, 2001; WebAim, 2010). In comparing the legislative history of equality (in

54 general) to the legislative effort made to provide disability rights (a specific group), the efforts toward progress is not such a positive depiction. The provision of legally defined rights for individuals with disabilities is a relatively new concept, less than half a century old (Jaeger, 2004). This nation has struggled in addressing the actual legal provisions that codify these human rights for those individuals with disabilities. Prior to gaining basic legal rights, individuals with disabilities suffered a range of discriminatory practices, including movements of eugenics, institutionalization, and segregation (Jaeger & Bowman, 2002; Braddock & Parish, 2001; Fleischer & Zames, 2001; Shapiro, 1993; Winzer, 1993). During the early part of America‘s history, the first legal discussions regarding the topic of ―disability‖ primarily related to limiting or denying the rights of individuals with disabilities (Jaeger, 2004). Jaeger indicates that preponderance of legal activity was primarily at the state level, serving to mandate the institutionalization of individuals with various physical, mental, and cognitive disabilities. That type of legislation continued through the 1920s with little related action at the federal legislative level. The limited federal involvement that did exist primarily related to the reviews conducted by the Supreme Court which typically supported the letter of the state laws, upholding their legality (Jaeger, 2004). As indicated, the passage of the first major disability rights law at the federal level is Section 504 of the Rehabilitation Act of 1973. This law was not without significant problems, evidenced by "a complex mixture of stereotyping and sympathy, apprehension and accommodation" (Rebell, 1986, p. 1438). Further reducing the immediate legal and social impact of Section 504 was the fact that the federal government worked to hinder its implementation for several years, eventually implementing the law only after a successive series of court orders and substantial national outcry (Fleischer & Zames, 2001; Jaeger & Bowman, 2002; Shapiro, 1993). The passage of the ADA and Section 508 had the intent of eliminating discrimination against individuals with disabilities. During this period, a new technology was emerging and that technology was the Internet. The new web based products and services that the Internet made possible were becoming increasingly common in the public and private sector. The ADA and Section 508 perhaps did not foresee the degree to which the government would exploit this technology to provide content and services more efficiently and effectively to the citizenry.

55 Relevant Policy Instruments

The Rehabilitation Act (1973) had application to the federal government and to all entities in receipt of federal funds. The success of this legislation was marginal, as federal government efforts actually hampered its implementation (Fleischer & Zames, 2001). Subsequent legislation, as the Americans with Disabilities Act of 1990, had limited success towards establishing an awareness of accessibility issues and failed in its efforts to increase the presence of individuals with disabilities in the workplace (Jaeger, 2004). Its weakness in accomplishing its intent was due primarily to the uneven and limited enforcement efforts by the federal government (Hignite, 2000; Kruse & Hale, 2003; Kruse & Schur, 2003). Underlying these important legal precedents toward attaining accessibility was a growing national ―commitment to citizens with disabilities and their right to the same level of access to the Internet and information, as all other citizens‖ (Muir & Oppenheim, 2002, p. 270). Section 504 of the Rehabilitation Act of 1973 established rather broad requirements regarding equal access for people with disabilities. In relation to the development of other efforts on web accessibility legislation, this act did serve to establish the initial rights to accessible information and communication technologies (Kanayama, 2003). The equal participation standards set by Section 504 extend to comprise the recipients of federal funds, to include state and local governments. In this way, standards of accessibility propagated throughout both federal and state governmental layers. The next major development in the history of accessibility legislation occurred in 1998, which was the amendment of Section 508 to the Rehabilitation Act of 1973. Congress initiated action on Section 508 in 1986, as an amendment to the Rehabilitation Act of 1973. The original Rehabilitation Act stands as one of the defining documents for the relationship between the U.S. government and its citizens with disabilities. The Rehabilitation Act provided protections to individuals with disabilities by establishing prohibitions against employment discrimination by the federal government, by contractors for the Federal government, and by programs and activities receiving federal financial assistance (TEITAC, 2008). During the legislative development period for Section 508, information technology became an increasingly important part of federal agencies‘ operations. Part of the intent of Section 508 was to ensure that these advances in information technology would be usable and

56 accessible to all individuals, including those individuals with disabilities. Section 508 was a landmark effort in the development of web accessibility policy. Congressional intent clearly focused on making Section 508 a targeted legislative effort that would appropriately address website accessibility. Unlike prior disability related legislative efforts, Section 508 had the full support of government (Hignite, 2000; Kruse & Hale, 2003; Kruse & Schur, 2003). The initial release of Section 508 standards for website accessibility occurred in October 1987. By April 1987, the federal General Services Administration (GSA) had published Bulletin 48 that set forth definitive requirements for agencies to provide accommodations designed to meet the needs of employees with disabilities when replacing the agencies‘ computer systems (HTCTU, 2010). In 1997, the Federal Electronic and Information Technology Accessibility Compliance Act was introduced and incorporated into the Workforce Investment Act of 1998, revising Section 508 through the Rehabilitation Act Amendments of 1998 (TEITAC, 2008). That revision to Section 508 placed strict requirements on federal agencies in the attempt to assure access to information and data for people with disabilities. This new requirement directed that all access by individuals with disabilities should be comparable to the access available to individuals without disabilities. If the agency determines that providing such access would result in an undue burden, then that agency is to (1) provide documentation as to why compliance creates an undue burden and (2) provide the information and data through an alternative means of access. The amended Section 508 enabled individuals to file complaints against federal agencies, if there is indication of noncompliance with Section 508 in procurement actions. The intent of Section 508 was to eliminate accessibility barriers, to provide equal levels of Internet access for people with disabilities, and to encourage development of assistive technologies that help achieve these goals (Section508, 2009). However, the unclear understanding of the undue burden clause is a weakness often exploited to avoid making websites accessible. Part of the undue burden clause allows for a ―separate but equal‖ approach to the dissemination of information or provision of services. That separate but equal approach does not align well with the Congressional intent of assuring fair and equal access for all individuals (Switzer, 2003). Section 508 (including subsequent amendments) established web-based accessibility standards and compelled federal agencies to comply with additional accessibility standards

57 issued by the Access Board. The Access Board is a federal agency whose role includes promoting accessible website design. Section 508 required that websites be accessible to individuals with disabilities, but its initial scope did not apply to the recipients of federal funding, nor did the law attempt to regulate directly the state or private sector. The Technology– Related Assistance for Individuals With Disabilities Act of 1988 (P.L. 100-407), and the subsequent Assistive Technology Act of 1998, referred to as the AT Act, both contain specific provisions requiring federal grant recipients to comply with Section 508. Under the AT Act (1998), recipients of federal grant funds are required to issue written assurances that commit to compliance with Section 508. These two acts are the federal government‘s attempt to disseminate policy requirements for web accessibility to the private sector, and to the state and local governments. This trail of legislative history traces the development of web accessibility policy from the nation‘s beginnings. These legislative actions evidence a compelling national concern for assuring website accessibility. This chronology of legislation indicates that it is of enduring national interest that websites comply with the accessibility provisions as mandated by Section 508.

Federal Policies Related to Website Accessibility

Several federal laws have applicability to ensuring the accessibility of information and communication technologies. Some of the federal policy efforts discussed in this section came into existence prior to the emergence of the Internet, but offer perspective on the overall intent of providing protections and assuring equity for individuals with disabilities. The following is a brief discussion of key policies that constitute the legal foundation of accessibility.

Architectural Barriers Act of 1968

For individuals with disabilities, the first federal law that addressed physical access to structures was the Architectural Barriers Act of 1968 (42 U.S.C.A. § 4151 et seq.). The intent of this law was to establish standards that assured that individuals with disabilities had equal access to all areas and services within a structure. The importance to web accessibility lies in its

58 founding of the Access Board (the Board), which initially was established to recommend structural accessibility guidelines for the federal government (Peterson, 1998). Those initial efforts led to the Access Board‘s establishment in 1973 by Section 502 of the Rehabilitation Act (29 U.S.C. 792), with the primary mission of the Board being the promoting of accessibility for individuals with disabilities (Section508.gov, 2010). The Access Board (Access Board, 2009) ensures access to federally funded facilities, and is a leading source of information on accessible design. The Board‘s responsibilities have grown, so that now it develops and maintains design criteria for the built environment, transit vehicles, telecommunications equipment, and for electronic and information technology. The Board also provides technical assistance and training on these requirements, and on accessible design and continues to enforce physical accessibility standards that cover federally funded facilities. Activities by this board have included reviews of Section 508 standards for website accessibility, and the conducting of comparative reviews of international web accessibility standards. Whenever the Access Board revises its standards, the Federal Acquisition Regulatory Council is required to revise the Federal Acquisition Requirements document. That revision mandates that each appropriate Federal agency must update its procurement policies and directives within six months to incorporate those changes. Within the overall organizational structure of the federal government, the Board functions as a coordinating body among Federal agencies and directly represents the public, particularly the interests of those individuals with disabilities. The Access Board consists of 25 members, with half of the Board membership comprised of representatives from most of the Federal departments. The other half of the Board consists of members of the public appointed by the President, a majority of whom must have a disability.

Rehabilitation Act of 1973

The Rehabilitation Act (P.L. 93-112) prohibits discrimination based on disability in the operations and the programs conducted by federal agencies, extending to apply to any programs receiving federal funding. Its applicability also includes federal employment practices and the employment practices of federal contractors.

59 Section 504 of the Rehabilitation Act of 1973 (29 U.S.C. § 70 et seq.) established rather broad standards of equal access for individuals with disabilities. These standards had application to all federal agencies and extended to those entities receiving federal funds. Section 504 states that "no qualified individual with a disability in the United States shall be excluded from, denied the benefits of, or be subjected to discrimination under" any program or activity that either receives federal funding or is conducted by any government agency. The language promotes reasonable accommodation for employees with disabilities; program accessibility; effective communication with people who have hearing or vision disabilities; and accessible new construction and alterations. Enforcement of the provisions of Section 504 falls within the individual agencies or indirectly through private lawsuits. However, for legal actions based on alleged Section 504 violations, there is no requirement to file a complaint first with a federal agency or to receive a right-to-sue letter before going to court. This more expedient path to court resolution the importance given to assuring these protections. Section 504 provisions establish the first rights for individuals with disabilities to information and communication technologies that are accessible (Kanayama, 2003). Section 504 was significant in extending equal participation to individuals with disabilities for all federal agencies‘ programs to include government provided websites.

Americans with Disabilities Act (ADA) of 1990

The Americans with Disabilities Act (ADA) (42 U.S.C.A. § 1201 et seq.) prohibits discrimination on the basis of disability in employment, state and local government, public accommodations, commercial facilities, transportation, and telecommunications. Of those six areas, the key area that has application to the discussion of website accessibility is the issue of public accommodation. In order to qualify for the protections offered by the ADA, one must have a disability or have a relationship or association with an individual with a disability. The ADA defines an individual with a disability as a person who has a physical or mental impairment that substantially limits one or more major life activities; a person who has a history or record of such impairment; or a person perceived by others as having such impairment (USDOJ, 2005). The ADA does not specifically identify all the impairments covered by the act.

60 ADA Title III: Public Accommodations

Title III of the ADA applies to businesses and nonprofit service providers that are public accommodations; privately operated entities offering certain types of courses and examinations; privately operated transportation; and commercial facilities. The U.S. Department of Justice (USDOJ, 2005) provides examples of such public accommodations as including private entities who own, lease, lease to, or operate facilities such as restaurants, retail stores, hotels, movie theaters, private schools, convention centers, doctors' offices, homeless shelters, transportation services and depots, zoos, funeral homes, day care centers, and recreation facilities including sports stadiums and fitness clubs. These examples suggest that the USDOJ interprets the ADA to apply broadly in the definition of public accommodation. It is under this umbrella concept of public accommodation that website accessibility is considered. The ADA empowers the DOJ to initiate legal action where there is a pattern or practice of discrimination in violation of Title III, or where an act of discrimination raises an issue of general public importance. The DOJ has applied the ADA broadly, allowable by its comprehensive language and the use of such non-specific terms as ―issue of general importance.‖ While the language of the ADA does not specify the term websites, the growing presence of the Internet in our everyday lives is something that would increasingly qualify as a technology of general importance to the public. It is within this context of interpretation that many supporters claim that the ADA‘s applicability should extend to website accessibility, as it falls appropriately under the concept of public accommodation.

USDOJ Policy Ruling: ADA Accessibility Requirements Apply to Internet Web Pages

In an action that supports considering websites as public accommodation and having coverage within the intent of the ADA, in September 1996 the USDOJ issued a formal Policy Ruling entitled ADA Accessibility Requirements Apply to Internet web Pages (Waddell, 1999). Waddell indicates: ―That policy ruling states that ADA Titles II and III require State and local governments and the business sector to provide effective communication whenever they communicate through the Internet. The effective communication rule applies to covered entities using

61 the Internet for communications regarding their programs, goods or services since they must be prepared to offer those communications via an accessible medium.‖ At that time (in 1996), the Internet was a new and burgeoning technology. Since that time, it has grown in capabilities and importance as a medium of communication. The Internet now offers a rich diversity in communication and information-sharing options that include text based documents, graphical content, virtual environments, audio channels, and video-streaming. As government and businesses increasingly depend on the Internet for content, programs, goods, and services, websites may increasingly fall into the concept of public accommodation.

Telecommunications Act of 1996

The Telecommunications Act of 1996 (47 U.S.C. §§ 153, 255) requires manufacturers of telecommunications equipment and providers of telecommunications services to ensure that such equipment and services are accessible to and usable by persons with disabilities, if readily achievable. The act requires telecommunications products and services to be accessible according to guidelines developed by the Access Board. The Access Board is required to review and update the guidelines periodically. The Telecommunication Act covers a broad range of products, including telephones, cellular phones, pagers, and fax machines. The Federal Communications Commission enforces requirements of the law. This act has the intent to ensure that people with disabilities will have access to a broad range of products and services such as telephones, cell phones, pagers, call waiting, and operator services that were often inaccessible to many users with disabilities.

E-Government Act of 2002

Congressional motivations for the passing of the E-Government Act of 2002 (Pub. L. 1-7-347, 44U.S.C. § 101) related to how the use of computers and the Internet was profoundly changing the inter-relationships among citizens, businesses, and government agencies. Congress also indicated that the federal government had a poor record of accomplishment in its efforts to implement advances in information technology to enhance governmental functions and services. The intent of this law was to promote the use of the

62 Internet and electronic government services, and to make the federal government more transparent and accountable, as well as to provide enhanced access to government information and services, while assuring access for persons with disabilities. The act established guidelines for the delivery of federal government information and services via the Internet. These guidelines supported such objectives as increasing opportunities for citizen participation in government; providing citizen-centered government information and services; and promoting access to high quality government information and services (P.L. 107-347, § 2). These objectives aimed to increase citizen participation in federal government, including those individuals with disabilities. Federal government websites would be accessible as this law mandated that ―all actions taken by federal departments and agencies under this act shall be in compliance with Section 508 of the Rehabilitation Act‖ (P.L. 107-347, § 202(d)).

Assistive Technology Act of 2004

The Assistive Technology Act of 2004 (P.L. 108-364) supported state efforts to improve the provision of assistive technology to individuals with disabilities of all ages through comprehensive statewide programs of technology-related assistance. This act provided federally allocated funds to states to assist with programs that promote improved utilization of assistive technology devices; loans of assistive technology devices; device demonstrations; training and technical assistance; and public awareness. States would also receive grants for programs that assisted individuals in acquiring assistive technology devices and services. At the federal level, the act provided for national activities such as the National Public Awareness Toolkit, which was a federal website that provided individuals with disabilities and the public with technical information on increased access to assistive technology. This act demonstrates the national commitment to providing assistance to individuals with disabilities and reflects the national concern for assuring that technologies are available and accessible for this group.

63 Summary

These various federal laws form the foundation of a national policy that establishes and promotes accessibility to government information and communication technologies. The concepts and principals contained within these legislative efforts provide a structure to help understand the progress made by this nation in providing disability rights, and in assuring that emerging technologies provide accessible products and services. These laws clearly assert the intent of the federal government to provide products and services that are available and accessible to the all individuals, including those persons with disabilities.

Key Efforts to Revise Section 508

When Congress enacted the Rehabilitation Act in 1973, the intent was to assure that people with disabilities had the opportunity to obtain meaningful employment with the Federal government. This legislation defined the rights of people with disabilities that would allow them to enter the federal workforce more easily. The law helped to improve employment opportunities in the federal workforce by:  Promoting a comprehensive set of services targeted to this population.  Eliminating discrimination based on handicap in programs or activities that receive federal funds. Section 508, as originally added to the Rehabilitation Act in 1986, was somewhat limited in effectiveness as it merely established non-binding requirements for technology accessibility. Subsequently, Congress amended the law in 1992 and again in 1998. In the 1992 Reauthorization of the Rehabilitation Act, a Section 508 amendment further strengthened the rights of individuals to have access to electronic and information technology. The amendment mandated that federal government electronic and information technology (e.g. websites, telecommunications, software, hardware, printers, fax machines, and copiers) be made accessible to people with disabilities. This amendment did not provide specific standards or detailed guidelines to define the criteria for achieving accessibility. In 1998, the Rehabilitation Act amendment expanded and strengthened the law significantly. Within this amendment, the law directed the formation of the Access Board, with

64 the mission to create binding and enforceable standards that will make federal electronic and information technology accessible to individuals with disabilities. The Access Board formulated well-defined standards to assist agencies meet their obligation to make information, products, and services provided by technology systems fully accessible. Incorporating this amendment into the Federal Acquisition Regulation (FAR) in 2001 resulted in a much more pronounced impact for the law. That provision made federal agencies' procurement of accessible technology subject to the same strictly defined compliance and enforcement mechanisms required by the FAR. Another benefit from the 1998 amendment was to establish a means for reporting compliance failures. If an individual with a disability finds that a federal agency has failed to comply with Section 508, then the individual can file a formal administrative complaint or pursue a civil lawsuit. Such complaints and lawsuits only apply against products procured. In July 2006, the United States Access Board formed a committee with the mission to update the accessibility standards for electronic and information technology covered under Section 508 (United States Access Board, 2008). One objective was to assure closer alignment of the updated standards with existing international accessibility standards. The Access Board (the Board) established the Telecommunications and Electronic and Information Technology Advisory Committee (TEITAC) comprised of 41 organizations representing the various accessibility related stakeholders. To assure a comprehensive and harmonized set of new standards, the Access Board consulted with stakeholders that included federal agencies; telecommunications, electronics, and information technology industries; public and nonprofit organizations representing individuals with disabilities; accessibility consultants; academic experts; and several international standards organizations. On April 3, 2008, the TEITAC presented its report to the United States Access Board recommending revisions to the Board‘s standards for electronic and information technology covered by Section 508 of the Rehabilitation Act of 1973. The report also addressed updates to guidelines for telecommunications products issued by the Board under Section 255 of the Telecommunication Act. These two laws form the primary legal framework for accessibility in the American information and communications technology environment. ―The principal content of this report is a set of recommended updates to the Section 508 Standard and the Section 255 Guidelines that commercial vendors and Federal

65 departments and agencies use to develop, procure, maintain, or use E&IT [electronic and information technologies] and telecommunications products and services (Section 6). This report proposes standards needed so that people with disabilities might achieve access and use of information comparable to that of people without disabilities‖ (United States Access Board, 2008). The report recommends provisions that explicitly address alternative technical approaches to assuring information, products, and services are consistently accessible, using technologies and practices that are both feasible and effective. The committee‘s report details recommended changes to both the substance and the structure of the standards and guidelines. The recommendations, adopted by consensus under a process common to Federal advisory committees, specify updates that are responsive to market trends and technological innovations that continuously reshape the range of products covered. These updates cover technologies used for communication, computing, storage, duplication, and production, among others. The report addresses all types of disabilities, including those that are sensory, physical, speech-related, or cognitive in nature. The committee‘s recommendations included a revised set of performance criteria that describes access capabilities for products more generally. The method of categorization was by features or capabilities. This differs dramatically from the approach taken for the existing Section 508 standards, as the organizing approach for those existing standards was by discreet product type. This move to more general terminology supports a broader application of accessibility. The recommendations within the report include performance and technical provisions; revised standards; updated and redefined terms; and procedures that will ensure greater accessibility in the areas of documentation, support, and maintenance. The report is also forward looking by making recommendations for future updates, while emphasizing the need for ongoing compliance testing, and further research. Once the TEITAC turned the report over to the Access Board, the Board conducted an initial review of the report‘s recommendations. That review by the Board led to the Board making proposed updates to Section 508 standards and telecommunications guidelines. A Draft Rule document now incorporates those changes that will update the Section 508 standards and the Telecommunications Act guidelines. In May 2010, that draft rule became available for public review until June 2010.

66 In November, 2011, the Access Board submitted the proposed changes (the Final Rule) to the Office of Management and Budget (OMB), for publishing in the Federal Register as a final opportunity for public comment. This public comment period will continue through fall 2012, when the Board will publish the Notice of Proposal Rulemaking in the Federal Register. The Final Rule will be published in Federal Register sometime after the notice of proposed rulemaking estimated sometime late in 2013 at the earliest. Once the Access Board issues the Final Rule, there will be a grace period before the Final Rule becomes reality, in order to provide time to update other related federal documents (e.g. the Federal Acquisition Regulations).

Evaluation of Website Accessibility

Evaluation is a critical element towards assuring accessible websites. A comprehensive evaluation plan should include both new web design efforts and verification of the compliance of existing websites with current guidance. While the intent of an accessibility evaluation may seem to be simple - to identify any barrier that may impede or prevent an individual with disabilities from using the website for its intended purpose – the actual process of evaluation may be very complex and challenging (Sloan, 2008). Sloan indicates that these challenges stem from the diversity of user needs, uniqueness of each browser used, and the nature of the functionality that the website attempts to provide. Such complexity of evaluation may require the employment of a mix of assessment tools and techniques to identify website accessibility problems. The goal of the evaluation effort is to uncover all true instances where an individual with a disability may have difficulty in using the website for its intended purpose (Sloan, 2008). This concern for determining all true problems necessitates the identification of all barriers that adversely affect accessibility. To find the entirety of such problems requires the evaluator to have a more holistic approach, versus the more traditional approach of conducting a page-by- page search for accessibility problems. Sloan purports that this type of holistic approach requires a focus on the tasks or outcomes that the user chooses to complete. This approach is a process- oriented analysis from a user‘s perspective, where such tasks will flow through a series of actions across multiple pages, rather than visiting a single page in isolation (Kelly, Phillips, & Swift, 2004).

67 Website evaluation for accessibility should be an integral element of the initial design, development, and testing within a website project. The evaluative effort does not end when the website goes from development stage to final production. Developers must conduct evaluations periodically as the website is modified and as accessibility requirements continue to unfold. During either of these scenarios, the most important aspects of an accessibility evaluation is the need for the accurate reporting of problems, and for a prioritization of barriers in terms of where solutions are most needed (Sloan, 2008).

Evaluation Techniques

Web accessibility evaluation has garnered quite a bit of attention as a practical challenge and as an academic research topic (Sloan, 2008). This interest has lead to significant activity towards developing methodologies and techniques for assessing website accessibility, to include the creation of software tools to support this type of evaluation. Sloan (2008) categorized the evaluative techniques into three groups:  Automated software testing;  Manual accessibility inspection (expert testing); and  User evaluations (user based testing) Each of these techniques will be discussed as to their strengths and weaknesses, and in terms of number of accessibility problems found; types of problems revealed; reliability of the results; expertise or technical skills required to use or interpret the findings; and resources required (as time, effort, cost) to conduct the evaluation.

Automated Software Testing

Automated software products are now available that provide a quick and inexpensive indication as to the accessibility of websites. Software testing for accessibility is an acceptable ―first step‖ towards determining if problems exist, and as a means to indicate the scope of these problems. Given those parameters (quick indication, plus easy execution and understandability), the approach of using an automated tool is appropriate (Moss, 2007).

68 Automated testing software performs an unobtrusive check of the specified website for errors that might cause accessibility problems. However, in many instances the software can only determine if a situation exists that may be an accessibility problem, requiring human intervention to make a valid determination if the problem is truly accessibility related. When the concern is to discover all accessibility issues for a given website, it is important to recognize that: ―automated testing programs can misidentify elements as accessible or inaccessible, do not take into account different disabilities or variations of abilities among people with similar disabilities, do not address issues of usability or functionality, do not address issues of compatibility with assistive technologies, and miss many accessibility problems that a person can identify, among other limitations‖ (Jaeger, 2006, p. 8). Automated testing should be just one part a comprehensive testing program. The over reliance on automated testing tools to indicate accessibility may create a false impression that a good rating equates to a highly accessible website (Witt & McDermott, 2004). As an accessibility testing method, the use of automated software products offers several advantages that include (Thatcher, 2006; Sloan, Gregor, Booth, & Gibson, 2002):  Fast reporting of certain accessibility problems with minimal human involvement,  Rapid analysis of software code with possibly greater accuracy than human inspection,  Relatively low resource and cost requirements, and  Minimal expertise is required to interpret the very basic accessibility findings. Additionally, this type of software testing can quickly generate benchmark data that can be used to compare a website against accepted best practices; or for the comparison between websites; or to compare accessibility of a single website when measured over a given time period. The use of automated tools has definite drawbacks for accessibility testing (Thatcher, 2010; Jaeger, 2006; Sloan, 2008). Perhaps the most significant drawback is that these tools require a great amount of human intervention to determine true accessibility barriers. The simple fact is that these tools cannot capture all the accessibility errors. In many cases, the most potentially significant accessibility barriers will go undetected (Sloan, 2008). Sloan identified other drawbacks of this type of testing tool as:  Inability to determine the actual impact of the problems identified;  Failure to accurately identify all accessibility problems that exist on a web page; and

69  Reported findings tend to be overly complex and excessively technical. These weaknesses bring into question the appropriateness of their use in performing accessibility evaluations (Jaeger, 2006). While these tools are relatively easy to use and may produce simplistic reports, their findings may mislead as being definitive studies of accessibility. Solely relying on this type of testing could result in a site deemed ―accessible‖ even though many problems would prevent use by individuals with disabilities.

Expert Testing

Expert accessibility testing of a website involves a skilled investigator who examines the website for potential barriers to its use. This manual approach by a competent researcher tends to yield a high quality set of findings, and provides for an efficient process of remedying the problems (Sloan, 2008). The scope of this type of research method extends beyond a simple checklist of common accessibility problem areas, expanding to include testing using various browsers and under various browser conditions (as with images turned off or with text enlarged). The drawbacks of this approach include the rarity of this type of skilled expert, and the associated problems in acquiring, scheduling, and remunerating his services. The process of examining a large website or a group of several websites can be extensively time consuming. These two factors combined (rare skill and timeliness) result in a high cost for this type of investigation.

User Based Testing

User testing involves having actual Internet users with disabilities interact with the website. The users would typically employ checklists, structured walk-throughs, or unstructured sessions to examine, review, and probe the website. The benefits of this type of testing is that the narrative rich sessions provide more accurate and objective results, revealing problems that are most likely to affect the intended website audience (Law & Hvannberg, 2002; Doubleday et al., 1997). As the users have a more disability-oriented perspective, the findings typically reveal true accessibility problems and better assess the implications of those problems (Killam & Holland, 2001).

70 Major drawbacks of this process are that user testing is more time consuming and expensive compared to the other accessibility testing methods (Doubleday et al., 1997). User testing tends to highlight the most relevant problems, but is not as strong an identification of the underlying cause of the problem when compared to expert testing. This method only identifies problems actually encountered during the scripted tasks, and it is not exhaustive in revealing the entirety of the problem picture (Doubleday et al., 1997). Other concerns relate to the generality of the findings, which tends to be limited, depending upon the unique type and extent of the disability of the user testers (Gray & Salsman, 1998). Locating and recruiting test participants may be very difficult, as well as the scheduling and logistical factors involved (LaPlant, Laskowski, & Stimson, 2001; Gray & Salsman, 1998). These factors also make this method both time consuming and very expensive (Lang, 2003). The literature indicates that no one method is sufficient to evaluate the accessibility of a website (Lang 2003). However, these three methods (automated, expert, and user accessibility testing) used in combination will provide the best possible picture of accessibility (Lang, 2003). Such a fully integrated approach is the best means for determining the greatest number of accessibility problems, the associated impact of those problems, and the underlying nature of the problems. When resources and funding constraints pose a serious concern, a program that includes these methods in a blended testing approach is arguably the most appropriate means to evaluate accessibility (Lang, 2003). Performing an evaluation of a website or group of websites is one way of raising awareness of accessibility and providing a richer understanding of the factors that help ensure that all users will have better access to the information and services provided via the Internet (Sloan, 2008).

Application of these Methods

Most accessibility studies of the websites of the United States government have employed only the automated testing method (West, 2004; Jackson-Sanborn, Odess-Hamis, & Warren, 2003; Stowers, 2002). Within that larger sphere of federal website accessibility studies, those studies that limited examination to federal electronic government websites have generally used solely the automated testing approach (Ellison, 2004; Michael, 2004; Lilly & Van Fleet; 2000; Ritchie & Blanck, 2003). In a more comprehensive examination, Jaeger (2006) researched

71 federal electronic government websites using expert testing and user based testing to evaluate the accessibility of those sites. Each of these studies looked at websites in the federal sector and determined the accessibility of these federal websites to be generally poor. Studies examining the accessibility of health care websites are limited in number, and employed only automated testing software. McCord, Frederiksen, and Campbell (2002) used automated software testing to examine selected web-based health information resources and databases. That study found each of the eight sites to have accessibility problems. Zeng and Parmanto (2003) looked at a selection of health care websites using automated testing and found poor accessibility. This literature review found no evidence of accessibility studies that specifically covered federal websites that provide health care information.

Website Content Review

This study conducts a website content review (or web content review) in order to determine if a web accessibility statement exists within a website and then determine the quality of that web accessibility statement. The purpose of this content review is to assess whether federal health care websites that have web accessibility policy statements have higher web accessibility than those without such statements. The literature review found no prior application of this method to federal websites or to health care websites. Stein (2002) and Chilson (2002) indicated the value of such a relationship, as it provides an indication of the awareness of accessibility within the website provider‘s organization.

Sources of Guidance on Web Accessibility

Section 508 presents guidelines that have the goal of providing a minimum level of accessibility using assistive technologies currently available to those with disabilities (Cardinali & Gordon, 2002). However, Section 508 is not the only widely used set of guidelines for website accessibility. Xiong (2008) indicated that there are two predominant sets of guidance related to web accessibility, with one set being Section 508 standards and the other set being the Web Content Accessibility Guidelines (WCAG) developed by the World Wide Web Consortium (W3C). The W3C has taken the standards as developed in Section 508 and extended them to

72 reach a new level of accessibility, usability, and quality in website development efforts (Caldwell, 2006). The World Wide Web Consortium (W3C, 2010a) is an international organization devoted to developing web standards, with a mission of developing protocols and guidelines that ensure long-term growth for the web. W3C's efforts serve to develop standards that define key parts of what makes the World Wide Web work for all individuals (W3C, 2010a). The W3C is universally peer accepted by leading governments for providing guidance that helps to ensure web accessibility (Smith, 2009). While these WCAG guidelines provide additional guidance that is beyond the accessibility criteria covered within Section 508, the scope of this study is limited to examining websites for compliance with the criteria stated within Section 508. Several studies have detailed the similarities and differences between the WCAG and Section 508 (Hudson, 2002; Thatcher, 2010; Access Board, 2010).

Federal Agency for Policy and Oversight

Section 508 established web-based accessibility standards and compelled federal agencies to comply with additional accessibility standards issued by the Access Board (the Board). The Access Board is an independent federal agency whose role includes promoting accessible website design. The Board is organizationally structured to function as a ―coordinating body among Federal agencies and to directly represent the public, particularly people with disabilities‖ (Access Board, 2010). The Access Board acts to promote and encourage accessibility by providing informative websites that offer extensive tools and resources to help a web designer build websites that are more accessible. Created in 1973, this board is now a leading source of information on accessible design. The Access Board develops and maintains design criteria for information technology, including the accessibility standards that are part of Section 508. The Access Board has the authority to review web accessibility standards and propose recommendations, including modification of existing legislation (e.g., the criteria for website accessibility found in Section 508).

73 Summary

This literature review briefly discussed several key topics related to this study‘s purpose, importance, objectives, research questions, and methods of analysis. The major topics included federal policy, federal agency organizational management, the evaluation of accessibility, ethics and social influences, and legal liability. Table 2.1 provides a summary of the major issues associated with each of these topic areas.

Table 2.1: Summary of key issues. Summary of Key Issues Topic Area Issues Do existing Have the Are there gaps or Do existing laws policies address the compliance deficiencies in the Federal provide standards new and emerging standards in current federal that ensure full technologies to Section 508 policy policies on accessibility? assure become obsolete? accessibility? accessibility? Are webmasters Do federal Do federal What are the actual aware of the legal webmasters have webmasters receive impediments that Federal agency requirements and adequate guidelines adequate training deter webmasters potential exposure organizational and procedures to on web from providing from not providing management assure accessible accessibility fully accessible fully accessible websites? requirements? websites? websites? Does automated What constitutes an What are the best Would better testing provide an efficient and practices in evaluative tools and Evaluating appropriate effective program creating, testing, methods more accessibility assessment of of testing to assure and evaluating effectively assess website accessible websites accessibility? accessibility? websites? accessibility? What impact does the occurrence of What is the impact What caused some What factors have inaccessible Ethics and of inaccessible groups of websites caused so many websites have on websites on to decline in social websites to be not the national goals individuals with accessibility over influences fully accessible? of civil rights and disabilities? time? equitable treatment of all individuals? What would be the Can legal action Do current laws legal impacts if the What should be the become a highly fail to provide Department of legal definition of effective means of Legal adequate Justice determines the undue burden building awareness enforcement, liability that websites have clause in Section and motivating oversight, and legal become public 508? compliance with remedies? accommodations? accessibility laws?

74 This literature review (chapter 2) has identified the following key themes and factors that emphasize the diverse and complex areas related to this research effort:  The increasing use of the Internet as a means to deliver health care information and services;  A complex federal policy environment that surrounds the current federal requirement to provide accessible websites;  The various strengths and weaknesses of the methodologies used to study website accessibility; and  The prior findings from relevant research studies on federal web accessibility. These key elements provide relevant background information as a context for this research effort, while also clarifying the benefits and drawbacks considered in selecting the approach taken by this study. The following chapter presents a more detailed look at the specific methods used by this study; indicates the project timeline; and presents the tasks and activities necessary to support the data collection and data analysis used within this research effort. That next chapter concludes by addressing the concerns for assuring valid and reliable data, justifying the methods used, and presenting the steps taken to mitigate any assumptions and limitations inherent to this research project.

75 CHAPTER 3

METHODOLOGY

Methods Overview

The purpose of this research study was to conduct an evaluation of the accessibility of federal health care websites for individuals with disabilities, and to understand the factors that may impede attaining accessible websites. Key components of the methodology include:  Side-by-side policy analysis – performed an assessment of the issues identified from a side-by-side comparison of the information policy instruments that most closely address website accessibility. The documents examined are the Americans with Disabilities Act of 1990 and the Rehabilitation Act of 1973, as amended by Section 508 in 1998.  Web content review - conducted a review of federal health care websites to examine for content related to accessibility. Provided quantitative measures as to whether a web policy statement exists and qualitatively assessed the quality of that web statement.  Automated testing – performed an unobtrusive examination of website accessibility by using a software program that examined a website for compliance with the standards identified in Section 508. Automated testing provided quantitative measures of instances of non-compliance using the accessibility standards in Section 508.  Expert testing – used a trained authority to review the website for accessibility errors. Expert testing provided quantitative and qualitative measures regarding the number of instances of non-compliance with the accessibility standards identified in Section 508. Results of this study provided indicators as to the accessibility of the federal websites that provide health care information. These methods provided insight into and indications of the problems, factors, and issues that influence the providing of accessible websites. The research design of this study included four phases: pre-data collection activities, data collection planning and preparation, data collection and analysis, and the presentation of

76 findings. Table 3.1 identifies the tasks related to Phase 1: Pre-data collection. Table 3.2 presents the timeline for Phases 2-4 of this study: project planning and preparation, data collection and analysis, and presentation of results. Activities in Phase 2 (planning and project preparation) included planning of the project, initiating the side-by-side policy analysis, preparing the data collection instruments, and pre-testing the respective data collection instruments. Phase 3 (project data collection efforts) initiated data collection methods to obtain measures of accessibility and initiated the analysis of data. Phase 4 (presentation of results and dissertation preparation) completed the analysis of the data collected and included the preparation of the dissertation document. To complete tasks identified in Table 3.2, the schedule indicates 12 months of effort.

Table 3.1: Tasks and schedule of pre-data collection steps. Phase 1: Pre-data Collection Steps Step Task 1. Present an overview of the study (Chapter 1). 2. Conduct a literature review (Chapter 2). 3. Develop the overall methodology for this project (Chapter 3):  Identify web accessibility issues related to policy environment (Chapter 2);  Present and justify data collection methods;  Identify units of analysis for each method used;  Identify measures for each method, as applicable; and  Develop data collection instruments. 4. Successfully present/defend dissertation prospectus

Table 3.2: Timeline of data collection and presentation activities. Project Timeline in Weeks Project Planning, Data Collection and Presentation Activities Phase 2: Phase 3: Phase 4: Planning/Project Project Data Presentation of Results Preparation Collection Efforts and Dissertation Preparation Weeks Weeks Weeks Weeks Weeks 1-8 9-12 13-24 25-32 33-52 Side-by-side policy analysis Pre-data Collection Web Content Review Pre-data Collection Automated Testing Pre-data Collection Expert Testing Analysis of Data Collection Efforts Dissertation Preparation

77 Table 3.3 indicates the steps taken to conduct the activities of data collection planning and preparation, data collection, data analysis, and the presentation of the findings. Each activity has an associated duration that corresponds to the timeline presented in the project schedule (Table 3.2). In Table 3.3, steps 1-4 identify each of the methods used in this research effort. The side-by-side policy analysis (step 1) required no pre-data collection effort and began in week 1. The remaining methods required pre-data collection efforts that occurred during weeks 1-8, with data collection starting in week 9.

Table 3.3: Research design scheduled tasks. Schedule of Tasks Phases 2 - 4 Step Task Schedule 1. Conduct side-by-side policy analysis Weeks 1-12 2. Web Content Review - planning and preparation Weeks 1-8  Identify areas of focus  Determine measures of accessibility policy  Develop protocol for data collection efforts  Develop data collection instrument  Pre-test web content review protocol 3. Automated testing - planning and preparation Weeks 1-8  Research, evaluate and select automated testing software  Develop protocol for data collection efforts  Develop data collection instrument  Pre-test use and data collection using automated software 4. Expert testing - planning and preparation Weeks 1-8  Identify measures of accessibility  Develop protocol for data collection efforts  Develop data collection instrument  Pre-test use and data collection using automated software 5. Conduct web content review, automated testing, and expert testing Weeks 9-24 6. Begin analysis of data from all methods. Weeks 9-32 7. Complete the analysis of data collection efforts Weeks 25-32  Prepare findings and develop conclusions  Propose recommendations from conclusions  Identify best practices  Suggest future research based on conclusions 8. Dissertation preparation and presentation Weeks 25-52

Table 3.4 indicates the sequence order for the four research methods used within this research study. This sequencing in executing the research methods supported the ability to make comparisons of findings that allowed refining or shaping of subsequent methods. For example,

78 as the automated testing looks at the entire population of websites, the automated testing started earlier in the schedule.

Table 3.4: Sequence order of the research methods. Sequence Order of the Research Methods Method Ordering Reasoning Value Added from Order Provided a better Conducted prior Revealed policy incongruities or Policy understanding of problems to all other procedures that are impediments to Analysis identified in subsequent methods. webmasters. methods. Knowing the agency‘s policy on Allowed grouping of websites Web Conducted prior accessibility allowed a better by ―with policy‖ versus Content to automated understanding of the type of errors or ―without policy.‖ Supported Review testing. amount of errors encountered during comparison of measures for the automated testing. two groups. Automated testing revealed a range of Allowed the comparison of the Conducted prior accessibility. From that range, high measures for the higher Automated to expert and low accessibility sites helped accessibility sites in relation to Testing reviews. determine the candidates for expert the lower accessibility sites. testing. Automated testing provided Allowed the comparison of the information that determined the measures for the group of Expert selection of the sample for expert higher accessibility sites to the Conducted last. Testing testing, by knowing the range of group of lower accessibility accessibility found in the population sites. sample.

The sequencing called for conducting the automated testing prior to the selection of websites for the expert review. This approach accommodated the possibility that the automated testing findings would reveal a range of accessibility across the websites examined. That range would allow the sorting of websites into categories of higher or lower accessibility. As that proved to be true, this approach supported making a purposive selection of websites for the expert review, so that the expert review sample had representation from both groups (higher and lower accessibility). This approach supported the comparing of responses from the two groups to gain additional insight into possible differences or similarities. Had the automated testing not revealed a significant range of accessibility, then the sampling for the expert testing would have been a random sample from the health care web links found on the government health care portal. The Units of Analysis section of this chapter provides additional information on sample size and sampling procedures, as does each of the chapter sections that address each of the methodologies used in this study.

79 Study Purpose, Goal, Objectives, and Research Questions

The purpose of this research study was to conduct an evaluation of the accessibility of federal health care websites for individuals with disabilities, and to understand the factors that may impede attaining accessible websites. The goal of this study was to improve the accessibility of the federal websites that provide health care information. Objectives (OB) to meet this goal included: OB1. Evaluate compliance of federal health care websites using Section 508 criteria. OB2. Identify the issues and barriers that affect attaining compliance. OB3. Identify practical recommendations to promote websites that are more accessible. These objectives served to guide the research conducted within this study. Data collection methods developed for this study addressed each of the following research questions (RQ) in terms of evaluating the accessibility of federal websites that provide health care information: RQ1. Are federal health care websites compliant with Section 508 requirements? RQ2. What are the issues and barriers to attaining accessibility? RQ3. What changes would promote websites that are more accessible? These research questions guided the development of the methods used in this study and shaped the analysis of the results of the data collection efforts.

Relationship of Research Questions to Methods

Table 3.5 presents the relationship between the research questions of this study and the proposed research methods. Each of the methods supports one or more of the research questions (RQ) as follows:  RQ1: The automated testing and expert testing methods addressed research question 1 by providing quantitative data related to the types of errors and counts (where errors are instances of non-compliance with the criteria established in Section 508).  RQ2: The automated testing, expert testing, and side-by-side policy analysis examined the underlying problems (as technical or programming problems and issues (as policy conflicts) that may impede webmasters from providing websites that are accessible.

80  RQ3: All four methods yielded findings that indicated changes that could promote website accessibility. The web content review examined each website for content related to accessibility policy statements. The findings included qualitative information that indicated the commitment of the organization to providing accessible websites. A comparison was made of the presence of such content to the relative occurrence of errors, to determine if a relationship exists between ―statement presence‖ and accessibility ―error counts.‖ The side-by-side policy analysis produced qualitative information that addresses issues and impediments that stem from the key documents related to website accessibility.

Table 3.5: Relationship of research questions to methods. Relationship of Research Questions to Research Methods Research Methods Research Side-by-side Website Questions Policy Content Automated Expert Analysis Review Testing Testing RQ1. Are federal health care websites compliant with Yes Yes Section 508 requirements? RQ2. What are the issues and barriers to attaining Yes Yes Yes accessibility? RQ3. What changes would promote websites that are more Yes Yes Yes Yes accessible?

To a degree, each of the methods yielded results that influenced policy related recommendations. Each method revealed information that contributed to providing recommendations in the areas of federal policy, organizational policy, and webmaster practices that could serve to improve the accessibility of federal health care websites.

Relationship of Study Objectives to Methods

Table 3.6 presents the relationship of the research methods to the objectives of the study. Each of the methods used in this study provided findings or additional insights that led to identifying practical recommendations towards improving website accessibility.

81 Table 3.6: Relationship of methods to study objectives. Relationship of Methods to Study Objectives Method Objective OB2. Identify the issues and barriers that affect attaining compliance. Side-by-side OB3. Identify practical recommendations to promote websites that are Policy Analysis more accessible. Web OB3. Identify practical recommendations to promote websites that are Content more accessible Review OB1. Evaluate the compliance of federal health care websites using Section 508 criteria. Automated OB2. Identify the issues and barriers that affect attaining compliance. Testing OB3. Identify practical recommendations to promote websites that are more accessible. OB1. Evaluate the compliance of federal health care websites using Section 508 criteria. Expert OB2. Identify the issues and barriers that affect attaining compliance. Testing OB3. Identify practical recommendations to promote websites that are more accessible.

Relationship of the Methodology to the Conceptual Framework

The purpose of this study was to conduct an evaluation of the accessibility of federal health care websites for individuals with disabilities, and to understand the factors that may impede attaining accessible websites. The methodology encompassed four research methods to address this purpose. The study conducted a review of related literature, gathered qualitative and quantitative information from federal health care websites, and examined key policy instruments. The following briefly identifies how each of the methods used in this study explored accessibility, and identifies how each method adds value and understanding to the conceptual framework used for this study (refer to Figure 1.2 the conceptual framework diagram): 1. Literature review – Presented the complex nature surrounding accessibility and identified key issues that affect attaining accessible websites. Identified the conceptual frameworks used to study the topic of website accessibility. 2. Side-by-side policy analysis – Performed an assessment of the issues identified within the literature review by conducting a side-by-side comparison of the information policy instruments that most closely address website accessibility. The documents examined are the Americans with Disabilities Act of 1990 and the Rehabilitation Act of 1973, as amended by Section 508 in 1998. This analysis provided information that

82 supports an improved understanding of the Federal Laws and Policies section of the modified TAM. 3. Web content review - Conducted a content review of federal health care websites to see if they provide information related to accessibility, as web accessibility policy statements. This method provided measures of the existence and quality of the web accessibility statement. This content review provided information that supports an improved understanding of the agencies‘ Perceived Value and Other Priorities sections within the conceptual framework. 4. Automated testing - An unobtrusive approach to determining website accessibility by using a software program that examined the website for compliance with the standards identified in Section 508. For this study, the product selected was the HiSoftware CynthiaSays online tool (HiSoftware, 2010). This method provided quantitative measures of the number of instances of non-compliance with the accessibility standards identified in Section 508. This method addressed the sections of the conceptual framework entitled Amount of Accessible Content, Types of Accessible Content, and For Whom It is Accessible. 5. Expert testing – This method involved having a trained authority on website accessibility review the website for errors. This method provided quantitative and qualitative measures relative to the number of instances of non-compliance with the accessibility standards identified in Section 508. This method addressed the following sections of the modified TAM: Amount of Accessible Content, Types of Accessible Content, and For Whom It is Accessible. Results of the research study informed the development of recommendations towards improving the accessibility of websites at the federal level. The recommendations addressed the areas of web development, technical tools, organizational policy, federal legislative policy, and training, as discussed in the literature review (chapter 2).

Units of Analysis

Schutt (2006) defines units of analysis as the level of social life on which the research question is focused. The units of analysis for this research study was federal health care websites.

83 Knowing the units of analysis is fundamental to understanding how the measures achieved represent the population under investigation. Operationalization as defined by Babbie (2001, p. 121) is ―the process where researchers specify empirical observations that can be taken as indicators of the attributes contained within a given concept.‖ For this study, the research design included the evaluation of website accessibility in terms of accessibility errors. The term accessibility error (or errors) will refer to situations where the website fails to meet the accessibility criteria as provided in Section 508. Attributes empirically assessed include error types and counts (number of occurrences). Table 3.7 provides summary information of the sample population and sample size for each of the methods used by this research study. The population of this study is the health care portal site for the federal government (www.health.gov, 2008), to include the federal health care websites linked from that portal site. The Office of the Secretary, U.S. Department of Health and Human Services oversees this official government health care portal site. From that authoritative source, the study included the portal site and all health care links listed on that site. In examining the web portal, the population (the specific units of analysis) constituted a purposive selection of this portal site (a convenience choice based upon the importance, type, and ownership of the information provided). The following section details the sampling process used for each method in this study.

Table 3.7: Populations for each methodology. Sample Populations Used for Each Method Method Analytic Sample Sample Approach Size Population The side-by-side policy analysis examined Comparing texts to find two documents: the Americans with instances of Side-by-side Disabilities Act of 1990 and the inconsistency, = 2 Policy Analysis Rehabilitation Act of 1973 as amended by ambiguity, and/or Section 508. This analysis also examined misalignment subsequent efforts to revise these policies. Web Reviewing website The web portal provided at Content statements relating to n = 20 http:www.health.gov, to include the links to Review accessibility policy websites available from that portal site. Examining for Section The web portal provided at Automated 508 errors by type and n = 6 http:www.health.gov, to include the links to Testing count websites available from that portal site. Examining for Section A convenience sample of 6 selected from the Expert 508 errors by type and n = 6 organizations that have active links from the Testing count www.health.gov portal. .

84 Explanation of Research Methods and Data Collection

This study explored the accessibility of federal websites that provide health care information. The study examined accessibility issues using multiple methods that include side- by-side policy analysis, web content review, automated testing, and expert review. This section presents the steps taken to execute each method used within this study. Also presented are the data collection instruments specific to each method. In executing the data collection, the researcher was the sole data gatherer, processor, and analyst for all methods. This served to mitigate errors that could arise from multiple data gatherers applying guidelines in an inconsistent or incorrect manner. All research activities used a single dedicated personal computer, thereby minimizing data collection and processing errors, and increasing the reliability of the collection effort.

Side-by-side Policy Analysis

Policy analysis is "the process through which we identify and evaluate alternative policies or programs that are intended to lessen or resolve social, economic, or physical problems" (Patton, 2010, p. 1). Policy analysis is a broad term that encompasses a range of different types or methods of analysis. McClure, Moen and Bertot (1999) discuss several types that are particularly useful in the examination of information policy. Examples include historical perspective, review of the key policy instrument, descriptive modeling, side-by-side analysis, review of related policy instruments, literature review, and key issues. The work by McClure, Moen and Bertot (1999) demonstrated the usefulness of performing a descriptive assessment of an information policy initiative by using side-by-side analysis. This technique for information policy analysis provides a powerful means to assess policy initiatives and serves to inform the methodology and data collection techniques within an empirical policy research study. A side-by-side policy analysis provides a descriptive assessment of the content of information policies, to include a detailed comparison between specific sections or criteria that pertain to two or more policy instruments. In its application within this study, this approach compared the content of one law to the content of another law, detailing similarities and differences in policy across specific topic areas. McClure, Moen and Bertot (1999) stated

85 that this side-by-side method of analysis offers an excellent shorthand and summary technique to descriptively assess and compare policy initiatives. In examining policy issues, this study conducted a side-by-side analysis of the two legislative acts that establish federal policy on website accessibility. Those two acts are the Americans with Disabilities Act of 1990 and the Rehabilitation Act of 1973. In analyzing the Rehabilitation Act there was a focus on the web accessibility provisions established by the 1998 Amendment of Section 508. This side-by-side policy analysis examined both acts, using criteria that revealed how the acts are similar, and how they differ in areas related to web accessibility. This side-by-side policy analysis included a look at policy related themes and addressed issues identified in the study‘s literature review, and discussed how each policy attempts to assure website accessibility. The side-by-side policy analysis examined these key areas:  Legislative intent;  Means used to accomplish or promote the intent;  Underlying objectives of the legislation;  Scope of application; and  Determination of financial burden. The benefits from examining these factors and from conducting this type of analysis were:  Identified alignment across acts that strengthen the federal policy stakeholder interests;  Determined incongruence that serves to weaken policy;  Discovered other strengths and weaknesses found in these acts; and  Revealed the efficiency and effectiveness of existing legislation. Achieving these benefits supported the purpose of this side-by-side policy analysis, which was to provide a better understanding of the way the two pieces of legislation promote website accessibility, and to identify areas of incongruence that serve to deter achieving the overarching intent of attaining website accessibility.

Web Content Review

As a method of research, the content review of documentation can provide useful description of the information within those documents (Babbie, 2001). It is a valuable tool for

86 discovering trends and patterns in documents (Schutt, 2006). This study used a content review to examine federal health care websites for content typically known as a website accessibility statement. The intent was to see if a relationship exists between the presence of a website accessibility statement and the number of accessibility errors found on a website. Federal health care websites that have website accessibility statements may have higher levels of web accessibility than those without such statements. To accomplish this, the study compared the findings of this method (web content review) to the findings provided by the automated testing method, and to the expert testing method used within this study. Many websites now offer accessibility related information through a website accessibility statement. In their research on accessibility statements, Sloan, Dickinson, McIlroy, and Gibson (2006) defined a website accessibility statement as a specific page (or designated page section) on the website that is devoted to accessibility content. That study found that in many instances the web accessibility statement may have the purpose of presenting a general policy towards avoiding unlawful discrimination against disabled people, or addresses the technologies used in the development and testing of the website to assure accessibility. Even Grounds (an accessibility consulting company) indicated that a website accessibility statement is a document whose purpose may include one or more of the following (EvenGrounds, 2010): affirm the website‘s compliance to accessibility standards and legislation; provide instructions to users of assistive technology; and address any inaccessible parts and features of the website.

Steps for Website Data Collection

1. Develop approach to operationalize the web content review. 2. Define accessibility statements and identify data elements to be collected. 3. Identify goals and measures. 4. Select study sample (all health care related websites linked from the www.health.gov website). 5. Design web content review protocol and the data collection method 6. Develop data collection instrument. 7. Test data collection method and instrument.

87 a. Develop web content review protocol for pretesting of process. b. Conduct pretest of data collection method and instruments. c. Review results of pretests. d. Refine data collection method and instruments based on the review, if needed. 8. Administer web content review.

Operationalizing the Measures

For the purposes of this research effort, the study used the following indicators to identify an accessibility statement within the websites examined. To qualify as an accessibility statement, the content must be a specific page (or designated page section) on the website that is devoted to accessibility content, and contains at least one of these three elements: 1. Affirms the website‘s compliance to accessibility standards and legislation; 2. Provides instructions to users of assistive technology; or 3. Addresses any inaccessible parts and features of the website. Lemon (2005) indicated the qualities of an effective accessibility statement and identified the following four features as being important to providing good accessibility statements: 1. Indicates a commitment to accessibility or to providing an accessible website (e.g., "We are working to make the website as accessible as possible for use by all individuals‖ or ― Changes have been made to this website to help with navigation, reading text, color schemes, and to allow the use of assistive technologies, as screen readers."), 2. Provides a means of contacting the webmaster to resolve problems, 3. Describes accessibility features or explains how to use the features provided within the website. Lemon (2005) provides these examples:  Link phrases make sense when read out of context.

 Tabular data is marked up correctly.  Form controls are explicitly associated with their prompts.  Presentation format employs style sheets.  Font-sizes are specified using relative units of measurement.

88 4. Provides a statement as to intent to conform to specific accessibility guidelines (e.g., Section 508, W3C/WAI). This study used these four criteria to determine if the federal health care websites are providing quality accessibility statements. The following presents the criteria used to assign the quality assessment indicators. Statements meeting all four requirements are HIGH quality. Statements that meet 2-3 criteria are MEDIUM quality, and statements meeting only one criterion are LOW quality.

Web Content Review Goals and Measures

Table 3.8 presents an overview of goals, measures of access, and definitions of the measures by types and kinds of accessibility statements found on agency websites. The goal of the web content review was primarily to determine whether the website provided an accessibility statement, and then to assess the quality of that statement. A data collection worksheet, provided as Table 3.9, identifies the criteria used in making the determination, and depicts the process (i.e., logical approach) used to determine the existence of an accessibility statement and to then assess the quality of that statement. The researcher performed an exploratory thematic content analysis in order to obtain an understanding of the content themes (types of information) presented in the web accessibility statements. The choice of thematic content analysis was appropriate due to the exploratory nature of this investigation (Aronson, 1994). This thematic content analysis was not part of the dissertation prospectus. During the review of the web accessibility statements, the researcher noted a wide variation in the content and format of presentation. The researcher decided to add this thematic content analysis, in order to inform the findings provided by the other analytic methods within the web content review. This thematic content analysis was an informal effort to understand more completely the issues that derived from the findings of the other analytic methods used in this web content review.

89 Table 3.8: Web content review goals and measures. Web Content Review Goals and Measures What is Web Content Measures Definition of Measure Measured? Review Goal Agency Identify if The existence of a web To qualify as an accessibility website agency website accessibility statement statement, the content must be a accessibility has a an (YES/NO indication). specific page (or designated page statement accessibility section) on the website that is statement devoted to accessibility content, AND contains at least one of these three elements:  affirms the website‘s compliance to accessibility standards and legislation;  provides instructions to users of assistive technology; or  addresses any inaccessible parts and features of the website. Agency Determine the Four criteria will 1. Indicates a commitment to website quality of the determine if the federal accessibility or to providing an accessibility website health care websites are accessible website. statement accessibility providing quality 2. Provides a means of contacting statement accessibility statements. the webmaster to resolve Accessibility statements problems. that meet all four criteria 3. Describes accessibility features will be considered to be or explains how to use the of HIGH quality. features provided within the Statements that meet 2-3 website. criteria will be considered 4. Provides a statement as to intent to be of MEDIUM to conform to specific quality and statements accessibility guidelines (e.g. meeting only one Section 508, W3C/WAI). criterion will be considered to be of LOW quality.

Agency Identify content Identify theme by A theme is indicated when similar website themes or types category or content type, types or categories of content accessibility of information accumulating a count of appear on multiple web site statement presented within the number of accessibility statements. each web occurrences of that accessibility theme. page for that website.

90 Table 3.9: Data collection worksheet – Web content review. Data Collection Worksheet - Web Content Review Agency Identification (URL): Is the stated element Source present? Document Information Element Yes No Determination Website Specific page or section of the website Indicate YES in this column if the that is devoted to accessibility content. statement EXISTS, and contains at (Statement EXISTS?) least one of the three elements. If YES, Element 1. Content Otherwise, indicate NO. then affirms compliance to determine if accessibility standards Please circle the appropriate each and legislation; indicator: element is Element 2. Provides present. instructions to users of YES NO assistive technology Element 3. Addresses any inaccessible parts and features of the website Accessibility Indicates a commitment to accessibility Assign rating based on number of Statement or to providing an accessible website. elements found: Provides a means of contacting the 4 elements =HIGH webmaster to resolve problems. 2-3 elements = MEDIUM Describes accessibility features or 1-0 elements = LOW explains how to use the features provided within the website. Please circle the appropriate rating: Provides a statement as to intent to conform to specific accessibility LOW MEDIUM HIGH guidelines

Data Collection Instrument Development

Based upon findings from the literature review, accessibility policy may exist in two locations (Stein, 2002). First, accessibility policy may exist on the agency website, often from a link at the bottom of the home page of the website. Second, policy on accessibility may be part of the agency‘s organizational policies and procedures related to website development and testing. This study examines the website for an accessibility statement, and attempts to assess the quality of the statement provided. Website policy statements on accessibility may contain many differing forms of information related to accessibility (Stein, 2002). Stein indicated four types of policy statements that may provide accessibility related information:

91 1. Web development policy 2. Accessibility policy 3. Section 508 compliance 4. Links to accessibility related information Stein indicated that it is important to search for web-based statements as these types of statements may contain information related to web accessibility. Each document review (website or web page) may contain multiple kinds of policy statements per document. Table 3.9 provides an example of the log used for data collection, to include the criteria used in determining the existence of an accessibility statement, and in assessing the quality of that statement.

Web Content Review Protocol

Websites may provide accessibility related information through an accessibility statement, which may be a specific page or section of a page on the site devoted to accessibility. The literature review (chapter 2) presented several studies that addressed the topic of accessibility for a variety of website categories. However, this research effort could only identify one study that attempted to provide a protocol for conducting an expert review of web accessibility statements. The reason for so little research in this area may stem from the fact that Section 508 does not specifically require the provision of an accessibility statement in order to ensure a website‘s accessibility. Sloan, Dickinson, McIlroy, and Gibson (2006) conducted an expert review of 24 websites in the United Kingdom that provided accessibility statements. That purposive sample consisted of governmental, educational, retail, and financial entities. The findings of that study guided the protocol used in this study‘s search for accessibility statements. The study found:  21 of the 24 sites provided a link to the accessibility statement from the home page.  For the other three sites, the researcher used a web search facility (search box) or site map to find the accessibility statement.  Many sites that did provide a link to the accessibility statement on the website home page positioned the link at the very bottom of the page, while some sites provided the link visually towards the top of the page.

92  20 of the 24 sites used the word ―accessibility‖ for their link to the accessibility statement. This study examined websites located in the United Kingdom, where there are no governmental set of guidelines for providing web accessibility statements. That is also true in the United States. The searching procedures evolved from expert expectations, as no prior related research on search protocols existed for locating accessibility statements on websites.

Web Content Review Process

The researcher conducted a web content review of all websites in the sample data set, where the sample consisted of all active links from the www.health.gov portal to federal agency websites that provide health care information. The data collection process consisted of the following steps: 1. Access the www.health.gov portal. 2. Sequentially access each active link from the sample data set 3. The link will take the researcher to a web page that will be examined for designated search terms by performing a web page scan protocol (details of a web page scan are provided in Appendix D) 4. Scan the page to assure it is the homepage of the website, or that it proceeds/links to the homepage. If the website indicates no other home page, then the researcher will assume this page to be the website homepage. 5. Examine this homepage for the following terms and phrases that would indicate information or active linking to accessibility related content or policy statements. If these terms are active links, then the researcher follows the links to search for relevant information. The designated search terms are: a. Accessibility or accessibility policy; b. Website (or web) policies; c. Section 508; d. Web (or website) notices; and e. Disability (Disabilities). 6. Complete the data collection worksheet.

93 7. Proceed to next website until the entire sample is complete. In order to derive the search terms identified in Item 5, the researcher completed a randomized search of United States Government websites until 25 sites provided statements that meet the criteria used in this study for ―accessibility statement.‖ The five terms identified in Item 5 are the most common terms for links to the website‘s accessibility statement.

Pretesting the Data Collection Method and Instrument

In an effort to improve the reliability and validity of the information gathered in this method, prior to the initiation of this method, three Web accessibility experts reviewed the data collection instruments and data collection processes. The experts selected for this review were internationally renowned experts with extensive professional experience and academic publications on the topic of Web accessibility. Each of the expert reviewers deemed the instruments and processes appropriate for this study. The data collection instrument identified in Table 3.9 was pretested to improve reliability and validity, and to measure inter-rater reliability. The researcher recruited and trained four external coders to execute the data collection process using the worksheet identified in Table 3.9. These four individuals, each with prior professional social science research experience, pretested the data collection method and instrument. The researcher recruited participants from the research staff at the Information Institute, at Florida State University. The selection of researchers for the pretest was a purposive selection from those Institute staff members who have experience in conducting academic research and in the testing of data collection instruments. The selection was also availability based, subject to recruiting willing volunteers. The researcher, who has knowledge of the skills and abilities of many of the Institute staff, determined the selection of candidates. The pretest participants received instructional training, including actual practice sessions, to familiarize themselves with the pretesting data collection process, and the use of the data collection instrument. The pretest examined four federal agency websites to search for website accessibility statements and assess the quality of the statement. The pretest coders gathered data from four randomly selected federal health care websites. The websites used for this test were federal health care websites, but were not websites from the sample population (i.e., web links from the

94 www.health.gov portal). This pretesting and refinement process supports reliability and face validity by confirming that the process and the data collection instruments measure what they are purported to measure. The researcher compared coded responses across coders to ensure inter- rater reliability. Acceptable Krippendorff‘s Alpha values resulted, indicating high inter-rater reliability (Bernard & Ryan, 2010). . Pretesting efforts involved having the researcher code the same data as the external coders. In this approach, the coded responses by the researcher and the external coders were compared across all coders to ensure inter-rater reliability. Acceptable Krippendorff‘s Alpha values resulted, indicating high inter-rater reliability. This latter testing ensures that the researcher‘s responses were acceptably similar to those obtained by the external coders. This type of testing provides additional support as to the reliability and validity of the methods used for data collection. The researcher coded three federal health care websites to conduct measurement of intra- rater reliability. The researcher conducted this coding in two sessions, allowing one month passing between coding sessions, in order to measure intra-rater reliability over time. An acceptable Cohen‘s Kappa value resulted, confirming high reliability (Choudhury, 2010). For the actual data collection associated with this method, the researcher was the sole collector of data. These steps support the validity and reliability of this research methodology. If these tests indicated any instruments or processes to be not reliable, then the researcher would have revised that process and/or instrument, and then repeated the inter-rater and intra-rater reliability testing. The researcher was the sole executioner of the web content reviews in order to support a high level of data collection consistency. The researcher has significant background and experience in the topic area of web accessibility (Brobst, 2009). The researcher utilized a designated personal computer system for each web content review, supporting improved reliability and validity. The uniform conducting of the data collection and assessments by a single researcher, using a single method for collecting data across a number of websites, strengthens internal validity. Any generalizations from the findings are limited to the parameters of the data collection for the sample population, thereby supporting external validity. Generalization of findings did not extend to other agencies or entities, but the findings could indicate ideas and strategies that would be of use to other health care websites. These steps support reliability and validity, and

95 were subjected to re-evaluation throughout the course of the overall study effort, and at the end of the study‘s data collection efforts. Chapter 5 presents a detailed discussion of findings from this method. As the nature of the research for thematic content was exploratory in nature, the data collection techniques used for this method were not subjected to pretesting. The researcher served as the expert reviewer in analyzing the accessibility statement to determine the categories of information contained within the statement. The conducting of pretesting efforts, and the associated measured reliability and validity statistics as Krippendorf‘s Alpha or Cohen‘s Kappa, would have possibly provided greater support to the reliability and validity of this effort. In conducting this data collection, the research served as sole data collector, using a dedicated personal computer.

Description of the Sample

This method examined all of the federal health care websites identified from the www.health.gov portal. The sample used for this method is the same sample as the automated testing method, examining all 36 web links on the www.health.gov portal. Table 3.10 lists these 36 websites. The examination of the 36 websites revealed that some web links were duplicates. That is, multiple web links on the www.health.gov portal referred to a single website. The determination was that the 36 Web links indicated 20 unique websites, as identified in Table 3.11. The sample size for this study was 20 unique websites. The data collection effort took place during the period of June 23 - 24, 2010.

Thematic Content Analysis

As part of the web content review, the researcher conducted a thematic content analysis of the accessibility statement. The method for identifying the statement was the same as previously identified. The sample determination was also the same. Acting as an expert reviewer, the researcher examined each of the web accessibility statements to identify themes or categories of content. The researcher read the entire web accessibility statement annotating the topics discovered during that review.

96 Table 3.10 Websites linked to the WWW.HEALTH.GOV web portal. Web Links at WWW.HEALTH.GOV Federal Agency Name URL 1 www.health.gov is a portal to the web http://www.health.gov/default.asp U.S. Department of Health and Human http://www.hhs.gov/ 2 Services (HHS) New Physical Activity Guidelines http://www.healthfinder.gov/ 3 released 4 Improved healthfinder.gov Makes Health http://www.hhs.gov/ophs/news/20080922.html 5 National Health Information Center http://www.health.gov/nhic/ National Women's Health Information http://www.womenshealth.gov/ 6 Center 7 MedlinePlus® http://www.nlm.nih.gov/medlineplus/ 8 2008 National Health Observances http://www.healthfinder.gov/nho/default.aspx 9 2008 Federal Health Information Centers http://www.health.gov/nhic/pubs/2008clearinghouses/clearinghouses.htm 2008 Toll-Free Numbers for Health http://www.health.gov/nhic/pubs/2008tollfreenumbers/tollfreenumbers1.htm 10 Information 11 HealthierUS.gov http://www.healthierus.gov/ 12 Healthy People 2010 http://www.healthypeople.gov/ 13 Dietary Guidelines for Americans 2005 http://www.healthierus.gov/dietaryguidelines/ 14 Physical Activity Guidelines http://www.health.gov/paguidelines/ Obesity Just as Risky for Teens as Heavy http://www.healthfinder.gov/news/newsstory.aspx?docID=624441 15 Smoking 16 Using a Balloon to Repair a Broken Back http://www.healthfinder.gov/news/newsstory.aspx?docID=624407 17 Work May Not Be Best for Young Kids http://www.healthfinder.gov/news/newsstory.aspx?docID=624426 18 More Health News >> http://www.healthfinder.gov/news/default.aspx 19 Get Health News by E-mail >> http://www.healthfinder.gov/news/subscribe.aspx 20 Noticias en Español >> http://www.healthfinder.gov/news/esdefault.asp 21 HHS Agencies http://www.hhs.gov/about/index.html#agencies 22 Office of Public Health and Science http://www.osophs.dhhs.gov/ophs 23 Office of the Surgeon General http://www.surgeongeneral.gov/ 24 Consumer Product Safety Commission http://www.cpsc.gov/ 25 Department of Agriculture http://www.usda.gov/ 26. Department of Defense – The Army http://www.armymedicine.army.mil 27. Department of Defense – The Air Force https://www.afms.mil/sg/index.htm 28. Department of Defense – The Navy http://navymedicine.med.navy.mil 29. Department of Defense – TriCaree http://www.tricare.osd.mil/ 30. Department of Veterans Affairs (VA) http://www.va.gov/ 31 Environmental Protection Agency (EPA) http://www.epa.gov/ Occupational Safety and Health http://www.osha.gov/ 32 Administration (OSHA) Office of Personnel Management – http://www.opm.gov/ehs/Emphlth.asp 33. Employee health programs Office of Personnel Management – http://www.opm.gov/insure/health/index.asp 34. Federal Employee Health Program 35 Social Security Administration (SSA) http://www.ssa.gov/ 36 healthfinder® directory of organizations http://www.healthfinder.gov/organizations/

97 Table 3.11: Data Collection Worksheet – presence of an accessibility statement. Findings: Does the website have an accessibility statement? Condition 1. Condition 2. Is any one of these three criteria met? Does this website Criteria 1: Criteria 2: Criteria 3: Is there a web Are both qualify as having an Name of the Does it confirm Provides Addressed any site section conditions accessibility web site compliance with instructions on inaccessible devoted to met? statement? existing the use of parts of the accessibility standards or assistive website content? legislation? technology? addressed? Office of the Surgeon General Social Security Agency OSHA OPM OPHS NLM Navy Medicine HHS Healthy People Health Finder EPA Consumer Product Safety Army Medicine Department of Agriculture Air Force Surgeon General Health.gov Women‘s health Veterans Admin. TriCare Healthier Us

Automated Testing

The automated testing method identified the types and counts of instances of non- compliance using the accessibility criteria established by Section 508. Automated testing provided a relatively quick and easy method for testing a large number of sites to get an indication of the website‘s accessibility. The use of this method allowed the researcher to test a larger number of websites when compared to user-based testing or expert testing. Testing more websites provides a better basis for making inferences about the population, but does not provide for a generalizable conclusion about all federal health care websites. The goal of the testing is to identify potential accessibility barriers, as defined within Section 508.

98 Steps for Data Collection

1. Identify automated testing goal, measures of accessibility, and definitions of measures. 2. Select study sample from the population. 3. Research and select automated testing software product. 4. Develop data collection instrument. 5. Develop method to implement data collection. 6. Test data collection method and instruments. 7. Modify data collection instrument and method based on review, if needed. 8. Administer automated testing.

Automated Testing Goals and Measures

Table 3.12 presents an overview of goals, measures of accessibility, and definition of the measures. The software product collected error counts and indicated the types of errors, based on the criteria established in Section 508. There are a number of free automated software products suitable for conducting accessibility testing of websites. Such a software product can quickly examine a website, methodically exploring each web page within the site. The ease of use and the relatively fast performance allows the researcher to examine a great number of sites, without concern for human participant issues as fatigue, scheduling, and the costs associated with participant participation. This depth and scope of probing in a quick, convenient, and economical manner is perhaps its strongest feature.

Table 3.12: Automated testing goals, measures, and definition of measures. Automated Testing Goals, Measures, and Definition of Measures What is Measured Automated Testing Measure of Definition of Review Goal Accessibility Measure Types of Section 508 Identify and collect Error counts by type The software product errors counts of types of probes the website to potential find violations of accessibility barriers Section 508 criteria.

99 Automated software testing products have been used widely in studies of website accessibility (Bray, Flowers, & Gibson, 2003; Chilson, 2002; Diaper & Worman, 2002; Stowers, 2002; Schmetzke, 2001). As a research methodology, automated testing has essentially become a de facto standard in conducting website accessibility studies (Chilson, 2002; Mates, 2001). Automated software testing products typically allow the user to select the specific criteria of evaluation, from several available options (Fagan & Fagan, 2006). This study used automated testing to examine each of the websites made available on the www.health.gov portal, using Section 508 criteria. The results provided by an automated test consist of error levels (priorities) and error counts. Some automated testing products even identify the specific Section 508 violation. Appendix E provides an example of the output received from an automated testing product. The software discovers errors by electronically reviewing the website programming code and identifying a violation of the standards provided by Section 508 guidelines. The software may classify these errors as Priority 1, 2, or 3. ―Priority 1‖ errors are accessibility errors that violate Section 508 standards (Wells & Barron, 2006). The testing report also lists results categorized as ―Priority 2‖ and ―Priority 3‖ errors. These errors are possible errors or warnings (Fagan & Fagan, 2006). These warnings are much more subjective in nature and may only indicate possible conditions that could produce a Section 508 violation. Such warnings usually require manual human intervention to conduct further research as to the exact nature or description of the error, followed by a human inspection of the condition to determine if it does actually exist.

Selection of the Automated Testing Sample

The sample for this testing was an inclusive sample taken from the population of websites identified in the www.health.gov portal. The www.health.gov portal site and all federal health care websites linked to that portal will undergo automated testing (6 websites). Table 3.13 identifies all the active links on the www.health.gov portal. These 30 links are pathways to other government websites that provide access to health related information. However, the links are not all separate websites. Closer examination reveals that these links stem from the six individual government websites. Table 3.14 identifies these six sites, and includes a website entitled Get Health News by E-mail as being a website link from the www.health.com portal. However, this

100 Table 3.13: Identification of the URLs.

Listing of Active Links on the www.health.gov Portal Website

Name of the website or active link URL text link www.health.gov (Homepage) http://www.health.gov/default.asp Quick Guide to Healthy Living http://www.healthfinder.gov/prevention/default.aspx Nutrition and Fitness http://www.healthfinder.gov/prevention/category.aspx?catId=1 Everyday Health and Wellness http://www.healthfinder.gov/prevention/category.aspx?catId=9 Pregnancy http://www.healthfinder.gov/prevention/category.aspx?catId=3 For Parents http://www.healthfinder.gov/prevention/category.aspx?catId=4 Heart Health http://www.healthfinder.gov/prevention/category.aspx?catId=5 Public Health and Safety http://www.healthfinder.gov/prevention/category.aspx?catId=6 HIV and STDs http://www.healthfinder.gov/prevention/category.aspx?catId=7 Cancer Screening and http://www.healthfinder.gov/prevention/category.aspx?catId=8 Prevention Important Screening Tests http://www.healthfinder.gov/prevention/category.aspx?catId=2 Diabetes http://www.healthfinder.gov/prevention/category.aspx?catId=10 For Women http://www.healthfinder.gov/prevention/category.aspx?catId=11 For Older Adults http://www.healthfinder.gov/prevention/category.aspx?catId=12 Health Communication, Health http://www.health.gov/communication/ Literacy, and e-Health Healthy People http://www.healthypeople.gov/2020/default.aspx Dietary Guidelines http://www.health.gov/dietaryguidelines/ Physical Activity Guidelines http://www.health.gov/paguidelines/ Healthcare.gov http://www.healthcare.gov/ National Health Information http://www.health.gov/nhic/ Center Womenshealth.gov http://www.womenshealth.gov/ MedlinePlus® http://www.nlm.nih.gov/medlineplus/ 2011 National Health http://www.healthfinder.gov/nho/ Observances 2011 Federal Health http://www.health.gov/nhic/pubs/2011clearinghouses/clearinghouses1.htm Information Centers 2011 Toll-Free Numbers for http://www.health.gov/nhic/pubs/2011tollfreenumbers/tollfreenumbers1.htm Health Travelers Bringing Measles http://www.healthfinder.gov/news/newsstory.aspx?docID=651709 Back to U.S., CDC Says Study Finds Strong Smoking- http://www.healthfinder.gov/news/newsstory.aspx?docID=651641 Asthma Link Certain Diabetes Drugs Better http://www.healthfinder.gov/news/newsstory.aspx?docID=651695 for Heart Health, Study Finds More Health News http://www.healthfinder.gov/news/default.aspx Get Health News by E-mail https://public.govdelivery.com/accounts/USOPHSODPHPHF/subscriber/new? Noticias en Español http://www.healthfinder.gov/news/esdefault.asp

101 Table 3.14: Identification of the website tested.

Listing of Websites on the www.health.gov Portal Website

Website identification Website information elements 1 Website Name www.health.gov URL http://www.health.gov/ Site owner Office of Disease Prevention and Health Promotion 2 Website Name Healthfinder.gov URL http://www.healthfinder.gov/ Site owner National Health Information Center 3 Website Name Healthy People URL http://www.healthypeople.gov/ Site owner U.S. Department of Health and Human Services 4 Website Name Healthcare.gov URL http://www.healthcare.gov/ Site owner U.S. Department of Health and Human Services 5 Website Name Womenshealth.gov URL http://www.womenshealth.gov/ Site owner Office on Women's Health 6 Website Name MedlinePlus® URL http://www.nlm.nih.gov/ Site owner U.S. National Library of Medicine, 7 Website Name Get Health News by E-mail URL https://public.govdelivery.com/ Site owner Not a government website

website was not included in this sample as it is not a government provided website, and as such fell outside the parameters of this investigation. For each website, the purposive sample included the home page and five additional web pages within that site. Because the home page of a website typically serves as the initial gateway to all subordinate content within the website, the testing of the home page provided a meaningful representation of the accessibility for the site. The five additional web pages will be a purposive selection of web pages taken from the active links and navigational menus provided on the website‘s home page. In this way, the researcher attempted to capture a sample of those pages that are most used, or that provide content that the website provider considers to be of greatest importance to the intended audience. Table 3.15 identifies the specific web pages examined within this study. The study period for the automated testing was April 2011.

102 Table 3.15: Automated Testing - Identification of the web pages tested. Automated Testing Identification of web Pages Tested Within Each website

Name of the Unique web Page URL website

1 www.health.gov http://health.gov http://health.gov/communication/Default.asp http://health.gov/paguidelines/ http://health.gov/dietaryguidelines/ http://health.gov/nhic http://health.gov/paguidelines/toolkit.aspx 2 Healthfinder.gov http://www.healthfinder.gov/ http://www.healthfinder.gov/news/ http://www.healthfinder.gov/HealthTools/ http://www.healthfinder.gov/prevention/ http://www.healthfinder.gov/findservices/default.aspx http://www.healthfinder.gov/HealthAtoZ/ 3 Healthy People http://www.healthypeople.gov/2020/default.aspx http://www.healthypeople.gov/2020/implementing/mobilize.aspx http://www.healthypeople.gov/2020/connect/default.aspx http://www.healthypeople.gov/2020/consortium/default.aspx http://www.healthypeople.gov/2020/topicsobjectives2020/default.aspx http://www.healthypeople.gov/2020/about/default.aspx 4 Healthcare.gov http://www.healthcare.gov/ http://www.healthcare.gov/news/blog/disabilities04142011.html http://www.healthcare.gov/news/blog/improvingcare04122011.html http://www.healthcare.gov/law/introduction/index.html http://www.healthcare.gov/foryou/employers/index.html http://www.healthcare.gov/foryou/disabilities/index.html 5 Womenshealth.gov http://www.womenshealth.gov/ http://www.womenshealth.gov/about-us/funding-opportunities/ http://www.womenshealth.gov/orgs.cfm http://www.womenshealth.gov/tools/ http://www.womenshealth.gov/news/english/651844.htm http://www.womenshealth.gov/about-us 6 MedlinePlus® http://www.nlm.nih.gov/ http://www.nlm.nih.gov/training.html http://www.nlm.nih.gov/biomedical.html http://www.nlm.nih.gov/index.html http://www.nlm.nih.gov/healthit.html http://www.nlm.nih.gov/hsrph.html

103 The researcher‘s choice of using six total pages per website stemmed from pragmatic concerns regarding the time involved in conducting the testing. Additionally, the determination to include six web pages from each website developed from studies by Guest, Bunce, and Johnson (2006) for determining non-probabilistic sample sizes. That work indicated that basic elements for themes present in as early as six reviewing efforts. This study was consistent with other findings that indicate 80% of meta-themes identify within the first five reviews (Brinck, Gergle, & Wood, 2002; Virzi, 1992). With such purposive sampling, it is important to avoid any pretense that such an approach is probabilistic, and hence representative. Such a sample will only provide indications, and cannot support generalizations of the population. While the findings may suggest broader indications for all federal health care websites, the study cannot purport wider generalization beyond this sample population.

Results from Automated Testing Method

The automated testing method contributed to this research study by providing information that addressed the accessibility of federal health care websites and identified the accessibility problems that individuals with disabilities may confront in their attempts to use these websites. The automated accessibility testing accumulated the number of violations of Section 508 criteria (referred to as errors) and identified the type of error. This information allowed the researcher to address several areas related to accessibility. As to the indication of accessibility, the testing provided the corresponding number of errors for that agency‘s website. Table 3.16 indicates a simple example tabulation of these results, while the actual data collection worksheet included a means to capture the errors counts by type of error. Analysis of this data revealed whether there was a large or small range of error

Table 3.16: Worksheet – Automated testing data collection. Worksheet – Automated Testing Data Collection Agency Name Number of errors discovered 1. Agency One 2. Agency Two 3. etc Total = Column Summation of errors Calculated Average = (Column Summation) / (Number of agencies .

104 distribution (high to low number of errors). That error distribution indicated whether accessibility problems are common or rare. The distribution information indicated the severity of the problem by the number errors found. Comparing error counts for individual sites to the overall average number of errors indicated relative performance within the group. Note that the number of errors was only an indicator of the severity of the accessibility problems. The determination of true severity would need to examine the affect that the errors have on the usability of the site to those individuals that have disabilities. As to the types of accessibility errors, automated testing provided the corresponding number of errors for that agency‘s website, and the type of each error. Error type refers to the Section 508 standard violated. Analysis of this information revealed the range of error types observed and indicated the frequency with which the error type occurred. This information indicated which type disability is most commonly impacted by the inaccessibility measured by this testing. As an example, if errors relating to text size or contrast color were more common, then the inaccessibility of the website would most severely affect those individuals with disabilities related to sight. If the analysis indicates that agencies have a broad range in accessibility, then this finding provided valuable information for the expert testing method. In other words, each website would have a corresponding number of accessibility errors. Looking at all the results in a ranked listing may indicate a significant difference in the average error count for the upper group compared to the average error count for the lower group. Given that a broad range in the number of accessibility errors existed, it was advantageous to use that information in the selection of agencies‘ websites for expert testing. By selecting websites for expert testing purposively from those agencies having higher and lower accessibility, and then comparing the results between these two groups did reveal distinct differences or patterns in the error types. Such differences influenced the discovery of policies or procedures that promote or inhibit the attaining of accessible websites. The agencies having less errors (greater accessibility) may offer insightful understanding of error propagation or provide indications of best practices for attaining full accessibility. This approach of grouping of websites into higher and lower accessibility was a simple means to attempt to discover significant differences between the two groups. This simple approach does ignore the situation where a website only has a single error, but that one error is a

105 major problem when compared to another website having multiple trivial errors. However, such an analysis would require determining an evaluative schema that would require the following informative elements:  Relative importance (weight) of each error type compared to other error types;  Relative importance of that error as to its impact on individuals possessing each type of disability, and  Disability related demographics for the set of users of that website. The literature review conducted for this study did not indicate the availability of these three pieces of information for the studied group of websites or for websites in general. Determining such information would require extensive research that is beyond the scope of this investigation. The simpler approach of grouping by error counts did yield useful and insightful information on the accessibility of these websites.

Selecting an Automated Testing Software Product

This researcher previously reviewed several automated testing products on two occasions (January, 2008 and February, 2009) in conducting studies on website accessibility. On the first occasion, the researcher used the WebXact software product, but that product is no longer available as a free service (Brobst, 2009). On the latter research effort, this researcher used the CynthiaSays (HiSoftware, 2009) automated testing software, which remains available as a free service for conducting accessibility testing. However, it was appropriate to conduct a review to assure the selection of a product that meets all the necessary criteria for this research effort. The product selected for this research effort needed to meet the following five criteria:  Low or no cost to use,  Identify errors based on Section 508 criteria,  Describe the errors (type of error, identifying the Section 508 criterion),  Provide the ability to accumulate (count) the number of errors found by type, and  Preferably allows the testing of the entire website in an easy manner. These criteria supported assuring the collection of appropriate data, while doing so within the resource limitations of this research effort. Table 3.17 indicates the worksheet format used in the selection of the automated testing product.

106 In selecting the automated testing tool for use within this study, note that several software tools exist that support automated testing for web accessibility. The World Wide Web Consortium (W3C, 2010b) maintains a composite listing of these tools. That list provided information on over 140 tools for testing accessibility. Of those 140+ tools, 49 products were capable of checking accessibility against Section 508 criteria. Using the worksheet indicated in Table 3.17, the researcher evaluated each product against the five major criteria for selection. Only one product met all the criteria specified for this study. That product was the HiSoftware Cynthia tool (HiSoftware, 2010). The researcher used that product to test the accessibility of the websites that have active links from the www.health.gov portal.

Table 3.17: Worksheet to select an automated testing product. Worksheet to Select an Automated Testing Product C1. C2. C3. C4. C5. Is product Low or no Uses Identifies Ability to Allows acceptable? cost? (/N) Section 508 error count entire If all five Product criteria? type? errors by website criteria are met URL name (Y/N) (Y/N) type? testing in an (all=Y) then (Y/N) easy indicate manner? “Acceptable” (Y/N)

Data Collection Instrument Development

The data collection instrument depicted in Table 3.18 allowed the researcher to consolidate the information provided from the automated software testing reports. In automated testing, the assessment effort uses an unobtrusive approach to determine website accessibility. In conducting this type of testing, the remote user simply enters the web address (URL: Uniform Resource Locator) of a selected homepage, allowing the software to examine that website for accessibility problems (violations of Section 508 criteria). The software conducts its assessment without any impact on the characteristics of the website. The resultant report identifies the type of occurrence, a brief description of the problem, and allows the researcher to count the number

107 of occurrences by type. Appendix E provides an example of the output received from the CynthiaSays automated testing portal.

Table 3.18: Data collection instrument – Automated testing. Data Collection Instrument – Automated Testing Site URL: Line Counts of Identifier Code for Description of Number of Types of Violations Type of Section 508 Violation Occurrences Violation 1 2 3 4 5 … TOTAL COUNT OF VIOLATIONS:

Method to Implement Data Collection

The researcher administered the automated testing prior to the selection of websites for the expert testing. The findings of the automated testing indicated the degree to which the sampled websites had a broad range in accessibility. As that was the case, the selection of websites for expert testing could be purposive to ensure that sampled websites included those agencies that fall into the higher and lower bands of accessibility. `The purpose of the automated testing was to identify types and counts of accessibility problems from the websites linked to the www.health.gov web portal. Steps to conduct the data collection included:

1. Identify all URLs for the websites linked to the www.health.gov web portal. 2. Access the automated software. 3. Enter the URL for each site to be tested. 4. Obtain output analysis and record data onto data collection instrument. 5. Conduct analysis.

108 Had the automated software failed to produce a report, then the researcher would have made additional attempts at a later time/day. That was not the case, as all automated testing was successful.

Test Data Collection Method and Instrument

The automated software system produced a great deal of the collected data and performed substantial website analysis. The researcher transferred the data provided from the output reports to the summary worksheets. Concerns for inter-coder reliability in preparing the summary worksheets are negligible and did not warrant testing. The researcher conducted all automated testing to assure consistency of the data collection efforts. The researcher used a single computer system to conduct the automated testing to support reliability and validity.

Expert Testing

Within this study, expert testing refers to an evaluation conducted by persons knowledgeable about website development and web accessibility. The researcher will be the expert tester for this method. The researcher has significant background in web accessibility (Brobst, 2009), and has personally developed three commercial websites and two personal websites. The researcher has programming knowledge of HTML coding, and has experience using web development tools that incorporate accessibility features. The expert testing method will examine each of the sample websites (n=6) for compliance with Section 508 standards. Expert testing is particularly valuable as a research method, as it is very unlikely that the evaluation of a website by user testing could be as thorough. To achieve the thoroughness achievable by expert testing, user testing would need to design tests that represented all people who possess the wide range of different types and levels of disabilities that must be accounted for, when designing for accessibility. Expert testing allows for fewer test cases by selectively identifying a representative array of potential issues confronting people with different disabilities. In a study similar to this research effort, Jaeger (2006) indicated that expert testing did identify the major accessibility issues that exist on a given website.

109 Steps for Data Collection

1. Identify assessment goals, measures of accessibility, and definitions of measures. 2. Select sample of federal agency health care websites for evaluation of accessibility. 3. Develop data collection instruments. 4. Develop method to implement data collection. 5. Pre-test data collection method and instruments. 6. Execute testing of websites.

Expert Testing Goals and Measures

The goal of expert testing is to determine the degree to which the selected federal health care websites meet Section 508 accessibility standards. This testing method provides a more complete evaluation of the accessibility of these sites when compared to automated testing. Automated testing can identify actual violations of Section 508 criteria, but many times provides only indications of possible errors. With automated testing, human intervention is required to determine if the possible error was an actual detriment to accessibility. Expert testing can proceed to make that determination immediately upon detecting the error situation, and can better assess the severity or effect that the error will have on the user of the website. Expert testing provides a more user-centered look at the overall accessibility than automated testing provides, as automated testing focuses on the technical coding used to develop the site versus a user-centric focus on the actual website usability and functionality. Expert testing does not claim to find all the accessibility issues for all possible users, but it can indicate the depth and breadth of accessibility problems (Lazar & Greenidge, 2006). Table 3.19 summarizes the goal, measures, and definition of measures used for the expert review. This method uses a previously tested data collection instrument to assess compliance with the criteria identified in Section 508 by identifying potential barriers to accessibility for users. The instrument used in this method had been developed and refined over several years and has been used in similar applications (Bertot, Snead, & Jaeger, 2005; Jaeger, 2006). The instrument will be described in greater detailed in a subsequent section of this chapter, but certain highlights are relevant to this discussion and will be covered here.

110 Table 3.19: Expert testing goals, measures, and definition of measures. Expert Testing – Goals and Measures What is Measured Expert testing Measures Definition of Review Goal measures How well the site Identify potential 10 criteria are used Each of the 10 complies with accessibility barriers which encompass all criteria is assigned a Section 508 criteria 16 standards from rating of 0,1,2, Section 508. indicating how well the site addresses the criteria being assessed

The data collection instrument examines specific design points or functionality within the website, in relation to the criteria in Section 508. This instrument encompasses all 16 key standards of Section 508, arranged into 10 primary questions, with some questions having sub- questions. These 16 standards are contained in Section 508 (see Appendix F, § 1194.22). Each question receives a numerical score that represents the researcher‘s determination of a ―grade‖ for each question. A question may be answered as ‗No‖, ―Somewhat,‖ or ―Yes,‖ receiving a respective score of 0, 1, or 2. This type of measurement (by use of a scoring system) supports the comparison between sites and between the types of evaluation methods (Bertot, McClure, Thompson, Jaeger, & Langa, 2003). While this data collection instrument was successful in evaluating federal electronic government websites, the instrument was also pre-tested and refined to assure its applicability for evaluating the federal health care websites, which is the group being examined by this study.

Selection of the Expert Review Sample

Studies by Guest, Bunce, and Johnson, (2006) for determining non-probabilistic sample sizes indicated that basic elements for themes presented in as early as six samples. That study is consistent with other findings on web usability research that indicate 80% of meta-themes identify within the first five sets of observations (Brinck, Gergle, & Wood, 2002; Virzi, 1992). The selection of websites for expert testing will be a purposive sample taken from the population

111 of websites identified in the www.health.gov portal. However, this approach does not fully support generalizations or representativeness, and only provides indications (Patton, 1990). With one exception, the sample of websites used for this expert review was the same sample as used in the automated testing. One of the web pages selected for the Medline site was no longer available at the time of the expert review. For this expert review, the web page replaced the web page. Table 3.20 indicates the websites and specific web pages examined for this testing effort. The expert review method examined each of the sample websites (n=6) for compliance with Section 508 standards. The expert testing looked at each sampled website using the two Internet browsers most commonly used. These two browsers are Internet Explorer (IE) and Mozilla FireFox. As of May 2010, these two browsers account for 84% of all Internet browsing, with Explorer at 63% and Firefox at 21% (NetApplications, 2010). In the testing for compatibility with assistive devices, the researcher only used the IE browser. The reason for this was that the assistive device product could only be used with the IE browser. The researcher‘s choice of using six total pages per website stems from pragmatic concerns regarding the time involved in conducting the testing and on studies by Guest, Bunce, and Johnson (2006) as stated above. With such purposive sampling, it is important to avoid any pretense that such an approach is probabilistic, and hence representative. Such a sample will only provided indications, and cannot support generalizations of the population.

Data Collection Instrument

This section details the data collection instrument used for this expert review. The data collection instrument used in this study developed from use in a number of research projects over a three-year period (Bertot, Snead, Jaeger, & McClure, 2005). In an application similar to this research effort, that instrument evaluated electronic government (e-gov) websites in terms of the standards of Section 508 (Jaeger, 2006).

112 Table 3.20: Identification of the web pages tested in expert review. Expert Review Identification of Web Pages Tested within each Website

Name of the Unique Web Page URL Website 1 www.health.gov http://health.gov http://health.gov/communication/Default.asp http://health.gov/paguidelines/ http://health.gov/dietaryguidelines/ http://health.gov/nhic http://health.gov/paguidelines/toolkit.aspx 2 Healthfinder.gov http://www.healthfinder.gov/ http://www.healthfinder.gov/news/ http://www.healthfinder.gov/HealthTools/ http://www.healthfinder.gov/prevention/ http://www.healthfinder.gov/findservices/default.aspx http://www.healthfinder.gov/HealthAtoZ/ 3 Healthy People http://www.healthypeople.gov/2020/default.aspx http://www.healthypeople.gov/2020/implementing/mobilize.aspx http://www.healthypeople.gov/2020/connect/default.aspx http://www.healthypeople.gov/2020/consortium/default.aspx http://www.healthypeople.gov/2020/topicsobjectives2020/default.aspx http://www.healthypeople.gov/2020/about/default.aspx 4 Healthcare.gov http://www.healthcare.gov/ http://www.healthcare.gov/news/blog/disabilities04142011.html http://www.healthcare.gov/news/blog/improvingcare04122011.html http://www.healthcare.gov/law/introduction/index.html http://www.healthcare.gov/foryou/employers/index.html http://www.healthcare.gov/foryou/disabilities/index.html 5 Womenshealth.gov http://www.womenshealth.gov/ http://www.womenshealth.gov/about-us/funding-opportunities/ http://www.womenshealth.gov/orgs.cfm http://www.womenshealth.gov/tools/ http://www.womenshealth.gov/news/english/651844.htm http://www.womenshealth.gov/about-us 6 MedlinePlus® http://www.nlm.nih.gov/ http://www.nlm.nih.gov/training.html http://www.nlm.nih.gov/biomedical.html http://www.nlm.nih.gov/grants.html http://www.nlm.nih.gov/healthit.html http://www.nlm.nih.gov/hsrph.html

The Information Use Management and Policy Institute (Bertot, Snead, Jaeger, & McClure, 2005) originally developed the data collection instrument used in this study. Table 3.21 identifies the data collection worksheet used in this study. The design of the instrument supported the evaluation of electronic government websites in terms of the criteria identified

113 Table 3.21: Expert testing data collection instrument. Data Collection Instrument - Expert Testing Site URL: Score Category (0 = No, Criteria Grouping Number 1 = Somewhat, 2 = Yes) 1 Provides an equivalent for elements related to content and 0 1 2 services?  Provides equivalent alternatives to auditory content?  Provides equivalent alternatives to visual content?  Alternative formats of elements of multimedia presentations synchronize to the appropriate parts of the presentation? 2 Does not rely on color alone? 0 1 2  All information conveyed through color also conveyed without color? 3 Content clear and organized so as to be readable to any 0 1 2 user? 4 Provides context and orientation information at all times? 0 1 2 5 Provides clear navigation mechanisms? 0 1 2 6 Has tables and other formatting that transform gracefully? 0 1 2  Identifies row and column headers on tables?  Title frames with text identifies frame and facilitates navigation? 7 Does not rely on moving pictures or flash to convey 0 1 2 content?  Designs pages to avoid flicker rates above 2 Hz or below 55 Hz? 8 Works comprehensively with assistive technologies? 0 1 2  All electronic forms allow users with assistive technologies to access the information, field elements, and functionality required for completion and submission of the forms, including directions and cues?  Text-only equivalent page available for every page that cannot otherwise be made completely compliant with all other requirements? 9 Ensures user control of time-sensitive content changes? 0 1 2  Users not timed out of applications?  Users given an alert message and the option to indicate that more time is necessary? 10 Ensures direct accessibility of embedded user interfaces? 0 1 2 TOTAL SCORE:

114 within Section 508 (Jaeger, 2006). Jaeger‘s (2006) study indicated the value of that instrument, in the rigorous examination of electronic government websites. The work by Jaeger (2006) provided a more detailed presentation of the development of this data collection instrument. As this study is looking at a different group or category of websites, the researcher subjected that data collection instrument to intra-coder and inter-coder pretesting to support its reliability and validity for this application. Previous web accessibility studies employed this instrument, finding that it provided a more comprehensive assessment of the accessibility of the selected federal websites than was possible through the singular application of the methods of automated testing or user based testing (Jaeger, 2006). This research study will adapt that instrument for use within this expert review, anticipating that it will provide similar results for this unique class of websites. The researcher solely conducted the expert reviews of the 12 websites, as the researcher has experience performing accessibility evaluations of websites (Brobst, 2009).

Data Collection Pretesting

To improve the reliability and validity of the information gathered in this method, the researcher arranged for an external review of the data collection instrument and the data collection process by accessibility experts. The experts selected for this review were internationally renowned experts with extensive professional experience and academic publications on the topic of web accessibility. Each of the experts deemed the instrument and processes appropriate for this study. Another effort to support reliability and validity of the findings involved the pretesting of the data collection instrument and the data collection process. This pretesting included calculating the statistical measure for inter-rater reliability (Krippendorff‘s Alpha). The researcher recruited and trained four external coders to execute the data collection process using the instrument identified in Table 3.21. The coders used the study‘s data collection instrument to gather data for four purposively selected federal health care websites. The researcher compared the set of coded responses across the external coders to ensure inter-rater reliability. Acceptable Krippendorff‘s Alpha values resulted, indicating high inter-rater reliability (Bernard & Ryan, 2010).

115 Pretesting efforts involved the researcher coding the same data as the external coders. In this approach, a comparison of the coded material from the researcher and the external coders determined the correlative inter-rater reliability. Acceptable Krippendorff‘s Alpha values resulted, indicating high inter-rater reliability. This latter testing ensures that the researcher‘s responses were acceptably similar to those obtained by the external coders. This type of pretesting, supported by acceptable Krippendorff‘s Alpha values, provides additional support as to the reliability and validity of the methods used for data collection. The researcher coded three federal health care websites to conduct a measurement of intra-rater reliability. The researcher conducted this coding in two sessions, allowing one month passing between coding sessions, in order to measure intra-rater reliability over time. An acceptable Cohen‘s Kappa value resulted, confirming high reliability (Choudhury, 2010). For the actual data collection associated with this study, the researcher was the sole collector of data. These steps help support the validity and reliability of this research methodology. If these tests indicated any instruments or processes to be not reliable, then the researcher would have revised that process and/or instrument, and then repeated the inter-rater and intra-rater reliability testing. It is important to clarify the exact nature of this expert review. That is, this review attempted to replicate the review process performed by Jaeger (2006). The reasons for this replication are two-fold. First, this replication is intentional, so that this study will (to a degree) produce findings that allow comparisons to that earlier study. Second, this method of assessing accessibility may prove to be a valuable tool for use in future accessibility studies, and may become a recommended evaluative approach (i.e., a best practice). The clarification made here is that this expert review is not a systematic and comprehensive examination of all the accessibility issues that may exist in these web pages. Further, this review is not a code based examination of the accessibility requirements identified as WAI or W3C checkpoints that may relate to the specific questions posed in the data collection instrument, or associated with the various standards within Section 508. It is acknowledged that these more exhaustive measures, that scour the code for accessibility errors and use human intellect to interpret and understand the nature and implications of the accessibility issues, are very effective methods for improving the accessibility of any website.

116 The data collection worksheet incorporates all 16 of the technical standards that are provided within Section 508 (§1194.22 [a] – [p]). Table 3.22 indicates the key relationships between the questions identified in the data collection worksheet and those 16 technical standards. The expert review used the data collection worksheet as a guide in conducting an evaluation of these six websites selected for the sample. The expert review also identified other significant usability and accessibility issues that may affect how well individuals with disabilities would be able to use these websites.

Table 3.22: Relationship of Section 508 criteria to the data collection worksheet.

Relationship of the 16 Technical Standards in Section 508 to the 10 Categories on the Data Collection Worksheet

Category Section 508 Topics Number Location 1 Provides an equivalent for elements related to content and §1194.22 (a,e,i,,l) services? 2 Does not rely on color alone? §1194.22 (c) 3 Content is clear and organized so as to be readable to any §1194.22 (b) user? 4 Provides content and orientation information at all times? §1194.22 (b,d) 5 Provides clear navigation mechanisms? §1194.22 () 6 Do tables and other formatting presentations transform §1194.22 (g,) gracefully? 7 Does not rely on moving pictures or flash to convey §1194.22 (b,) content? 8 Works well with common assistive technologies? §1194.22 (d,e,l,m,n) 9 Ensures user control of time sensitive content? §1194.22 (n,p) 10 Ensures direct accessibility of embedded user interfaces? §1194.22 (f,l,m)

The instrument design allowed the expert review to collect information on multiple levels through a single evaluation (Jaeger, 2006). First, some questions are open-ended which allows the researcher to assess the positive and negative aspects of the website. Some questions have sub-elements that indicate key considerations that relate to the larger accessibility issues posed by the main question. These sub-elements may provide important details about usability or design features that provide a more robust understanding of the site‘s accessibility.

117 Second, the instrument presented a scoring schema, where the researcher was able to assess a rough ―grade‖ for each question. The researcher assigned scores in the following manner. In scoring a question, the researcher examined the website for every instance of the condition identified in the question. If that condition is never satisfied or consistently missing from a site, then the researcher assigned a score of ―0.‖ A score of ―1‖ is awarded if the answer to the question is ―Somewhat.‖ In this situation, if any given element appeared in only some places but not others, then the researcher assigned a score of ―1.‖ If the researcher examined the website for every instance of the condition identified in the question, and if that condition is always satisfied, in every instance, then the researcher assigned a score of ―2.‖ By summing the individual scores for each question the, researcher is able to assign a total point score that represents the accessibility score for the entire website. Given that the review consisted of 10 question groupings, the maximum score for a site is 20, while the minimum possible score is 0. This type of scoring approach relates to the Functional Capability Requirement Index, as developed to assess the overall functionality of websites by assigning scores to important elements (Jaeger, 2006; Texas Center for Digital Knowledge, 2002). This type of scoring system is helpful for comparison purposes between sites and between types of evaluation (Bertot, McClure, Thompson, Jaeger, & Langa, 2003; Jaeger, 2006). By using open-ended questions, the researcher is able to obtain and describe details that are important for understanding the accessibility of each site. By using the scoring approach, the researcher is able to provide a comparative perspective from the relevant open-ended data.

Method to Implement Data Collection

The purpose of the expert review was to identify accessibility problems as indicated by the standards put forth by Section 508. Steps to identify the problems per agency website included: 1. Perform the expert review. 2. Complete the data collection instrument. 3. Analyze/compare data.

118 The researcher will cross compare the findings of all methods, to look for indications of alignments or inconsistencies.

Evaluating the Web Pages

The following discussion provides details as to the process used to evaluate the web pages. The information presented for each question includes:  How the question relates to providing accessibility,  An indication of the impact which could result from a failure to provide the related accessibility features, and  An overview of the criteria used to make the assessment. The intent is to provide a better understanding as to how each question relates to assuring the accessibility of a website, to include providing example of how this accessibility feature or function may be of value to individuals with impairments or disabilities. Question 1 determined if text equivalents exist for every element related to content. Websites should provide content that conveys essentially the same information, function, or purpose as auditory or visual content. While some individuals cannot use images, movies, or sounds directly, they may be able to gain value from a web page that includes equivalent information for the visual or auditory content. That equivalent information must provide the same purpose as the visual or auditory content. As an example, given an image of a button that links to another part of the web page that provides help, the text equivalent could be ―Go to the help section of this page.‖ The benefit of text equivalents lies in their capacity to provide presentations that are accessible to people from various disability groups using a variety of assistive technologies. Typically, text based content can be readily output to speech synthesizers and Braille displays. Text based presentations are also very scalable allowing the user to adjust the size to compensate for a visual impairment. When displayed visually, the equivalent text benefits users who are deaf as well as the majority of web users. In other situations, providing non-text equivalents (e.g., pictures, videos, and pre-recorded audio) of text is also beneficial to some users, especially nonreaders or people who have difficulty reading. In movies or visual presentations, visual action such as body language or other visual cues may include enough audio information to convey the same

119 information. Unless the website provides verbal descriptions of this visual information, people who cannot see the visual content will not be able to perceive it. In this evaluation, the researcher examined each sampled web page to locate all audio, visual or multimedia presentations to assure that equivalent text is available to the user. The examination included examining each image for the use of an ALT TAG description. If the image does not use an ALT TAG, then the researcher scanned the text area of that web page to located descriptive text that explained the image content. Question 2 provided an examination to assure that the web page does not convey information by solely relying on the use of color. That is, for all information conveyed though the use of color, the web page must also convey that information without using color. If the website uses color alone to convey information, individuals who cannot differentiate between certain colors will not be able to understand the information. In addition, this type of testing addresses the situation where users must employ accessibility devices that have non-color or non-visual displays. The concern is that individuals with visual problems, as poor contrast recognition or color blindness, are able to fully understand all information presented on the web page. The evaluation of this question examined the page from two perspectives. First, the researcher viewed the web page to assure that the color schema provided appropriate levels of contrast. When foreground and background colors are too close to the same hue, there may not be sufficient contrast when viewed using monochrome displays or by people with certain types of color deficits. The researcher then viewed the pages to ensure that content was not conveyed solely by the use of color. As part of the accessibility analysis, but not part of the scoring of the website, the researcher viewed the page in gray scale to assure that all content conveys equally when compared to the colorized versions. This is a common type of color related visual imparity called achromatopia. Additionally, testing examined each web page to determine if the use of colors compensated for the most common forms of color blindness (protanopia, tritanopia, deutanopia). The researcher used a free online testing service to test the web pages for the four type of color blindness mentioned above (Colorfilter.wickline.org, n.d.) Question 3 investigated whether the web page presented content that is clearly understandable and well organized, so that it is readable by any user. The factors used to evaluate

120 the page included determining if the page presents a consistent page layout, easily recognizable graphics, and uses language that is readily understandable. These simple factors assure that the web page conveys information in a way that benefits individuals who may have a variety of disabilities. These features help those individuals with cognitive disabilities or who have difficulty reading. The basic premise is that the use of clear and simple language promotes effective communication. Access to complex written information can be difficult for people who have cognitive or learning disabilities. Using clear and simple language also benefits people whose first language differs from that of the web page, including those people who communicate primarily using sign language. For scoring the website, the researcher reviewed each sampled web page within the website, comparing the six pages to determine if there was a consistent page layout used. Next, a review of each page indicated if the graphic images were readily recognizable and understandable. Not used in scoring but part of the accessibility review, the researcher used the Flesch- Kincaid Index as an indication of the readability and understandability of the content (Accessibility Institute, 2011). If the website included a significant amount of text, then the researcher copied that text section into to calculate the index, which represents the educational grade level required for the user the comprehend the text. Question 4 examined the website for features that assure that context and orientation information exists at all times. By providing context and orientation information, the web page helps users understand complex pages or elements. Grouping elements by use of headings, physical proximity, or by use of sections within a web page are techniques that successfully convey relationships or associations to the reader. Providing contextual information about these relationships between elements can greatly improve the understanding of the information presented. Complex relationships between parts of a page may be difficult for people with cognitive disabilities and people with visual disabilities to interpret. Grouping elements, and providing context and orientation information about the relationships between the elements, can aid the accessibility and usability of the website. In scoring the website, the researcher examined each web page for the appropriate use of section headings, and the division of large blocks of information into more manageable groups

121 where natural and appropriate. Additionally, the researcher examined each web page for the use of tab navigation, and observed the general organization of data presented. Question 5 indicated the importance of providing clear navigational mechanisms. The website should provide access to content through a variety of navigation mechanisms, including sequential navigation, direct navigation, searches, and structured navigation. By providing a variety of navigation and search mechanisms the users with disabilities (and all users) have a more efficient means of access to the web page content. Direct navigation is important to users with some physical disabilities (as limited manual dexterity) and to users with visual impairments. Direct navigation may be possible with the pointing device (e.g., mouse) or the keyboard (e.g., keyboard shortcuts). Providing clear and consistent navigation mechanisms also benefits individuals with cognitive disabilities or blindness. Site maps and search boxes increase the likelihood that a person will find what they are looking for within the website. The features may be much more convenient and expeditious for those users that have limited dexterity. These more direct browsing features would be of value to those individuals who are unable to search from page to page due to limited manual dexterity. To accomplish this review and to score the website, the researcher scanned the web pages to determine if navigation mechanisms are present, to include navigation bars, breadcrumbs, and drop down or pull out menus. The researcher assessed additional factors that were not part of the scoring of the website. The additional factors included the provision of search blocks, site maps, and the use of a table of contents for those web pages that provided extensive content. The researcher examined for the consistent placement of these features across the web pages that make up the website, which promotes a more understandable, locatable and consistent method of navigation. Question 6 tested for accessibility by investigating whether tables transform gracefully. For this study, the term transform gracefully refers to the enlargement of text within the data table. When the user attempts to increase the size of table, each element within the tables should be visible and readable. Each row heading and column heading should increase proportionally in step with the data element within those rows and columns. Some accessibility assistive devices allow users to navigate among table cells and access header and other table cell information.

122 Unless the web developer properly codes the table, these tables will not provide the user with the appropriate information. In addition to proportionality, a key accessibility feature for data tables is the provision of row and column headings. When a user employs a screen reader to view a web page that has a data table, the presence of row and column headings is essential to assure the user can understand the information that the table represents. Another common problem related to data table presentation occurs when data tables present as images, as opposed to actual tabularized data elements. In many case these images are not accessible in a way that would allow the user to access the data elements within the columns and rows. To be accessible, tables presented as images should have the content made available in an alternative text based manner. To evaluate this question and score the website, the researcher examined each web page to determine if tables are present. Upon finding a table, the researcher first determined if the table was an image, or presented (coded) as a data table. If the table is a true data table, and not and image, then the researcher determined if row and column headings were provided. The next step was to further test the data table to assure that the data elements, and the row and column headers, would enlarge in way so that all the elements and headings were readable. Question 7 required the reviewer to examine the web page for the presence of moving pictures, text, or flashing elements used to convey content. Examples of these items on a website could include animated GIF images; flashing images; scrolling text; or blinking text. In general, any moving content can be distracting, to the point where the rest of the page is unreadable to visitors with cognitive disabilities. Accessibility guidelines indicate that web designers should avoid the use of animated GIFs (W3C, 2010a). Gratuitous use of animation can be distracting to some visitors, making it difficult to focus on the content. Individuals with cognitive impairments may find the use of these moving images to be severely distracting. Screen flickering or intermittent light stimulation in animated GIFs can cause seizures for web users with photosensitive epilepsy. Rapid changes in images from dark to light, such as a strobe light effect, can also trigger seizures. Some assistive technologies, such as screen readers, are unable to cope with scrolling text. Visitors with poor eyesight or cognitive disabilities may be unable to read the content at the pace it scrolls. The motion can be distracting, and may prevent the reading of important

123 information on the rest of the page. Individuals with cognitive or visual disabilities may be unable to read moving text quickly enough or at all. Movement can also cause such a distraction that the rest of the page becomes unreadable for people with cognitive disabilities. People with physical disabilities might not be able to move assistive devices quickly or accurately enough to interact with moving objects or flashing images. Some assistive technologies, such as screen readers, are unable to cope with moving objects. For the vision impaired user, blinking text can be very problematic when using screen readers. Screen readers frequently get stuck on the text, and read it repeatedly, and could cause the system to crash. Blinking text can also distract users from other parts of the page. Features as text or image movement are unnecessary to catch the attention of visitors to the web page. The simple fact that they are on the web page indicates their interest in what it has to offer. In evaluating the website for this question, the reviewer scanned each web page to determine if any of the following features where used: animated GIFs; flashing images; scrolling text; or blinking text. Question 8 required the researcher to test the website to ensure the pages work well with assistive technologies. Many hardware and software products are available to assist individuals with disabilities. For the purposes of this study, the researcher tested the websites using a screen reader and a screen magnifier. The screen reader and text magnifier used for this study was the Zoom Text 9.1 product (Ai Squared, 2011). This product is one of the most frequently used assistive software products (Illinois, 2008). For this test, the researcher used the Internet Explorer (IE) web browser software. This assistive device (Zoom text 9.1) only works with IE. IE is the most commonly used browser in the United States, with about 48 per cent of the market (StatCounter, 2011). The decision to use only one browser in this effort was supported by the findings by Jaeger (2006) indicating that ―…changes in browsers had little impact on levels of accessibility, and sites that did not work with a particular assistive technology were not likely to work with a similar assistive technology. (Jaeger, 2006, p. 86)‖ The use of electronic forms on websites may be a problem area for users with disabilities, if the form design does not accommodate assistive devices. For scoring the website, the researcher scanned the web pages to determine if automated forms are in use. If forms are used, then the researcher examined the source code related to that form to ensure that the

124 appropriate use of ALT TAGS and LONGDESC modifiers are in place to fully describe the form‘s elements, when used with an assistive device. Question 9 examined whether the website supported accessibility by providing user control of functions that may lead to being timed-out of applications. Individuals with visual or cognitive impairments may have a great deal of difficulty in reading, comprehending, and completing online forms. These individuals may need additional time to complete the form, and need to be able to arrange for additional time. Section 508 does not address what is an appropriate amount of time to allot to complete an online form. In the context of accessibility standards, sufficient time means a period of time that allows even the slowest users to complete a task. A rule of thumb for accommodating the slowest users is to allow up to 10 times the amount of time it would take an average user to complete the task (Georgia Tech Research Institute, 2007). In assigning a score for this test, the researcher tested for this condition by scanning each page for the existence of a possible time sensitive input function (e.g., text entry field or an online form). If a possible time sensitive input area existed, then the researcher initiated an effort to complete the form. The researcher would cease the completion of the form before finishing, in order to see if the form would time out. The researcher monitored the web page to see if the form timed-out, or if the website provided a prompt in order to have the time limit extended. The researcher would wait for 30 minutes and note if the form timed out or if a prompt appeared. If a form appeared to be so complex that additional waiting time was appropriate, then the researcher test completed the form and multiplied that time by 15 to determine an appropriate waiting period to test for a time-out message to appear. Question 10 seeks to assure that embedded user interfaces are directly accessible to the website user. Embedded user interfaces are sections of code (e.g., scripting) within a web page that interacts with the web user to produce content or provide functionality. Scripting is the most common use of a web programming extension language that allows control of one or more applications. ―Scripts" are distinct from the core code of the web page, as they are usually written in a different programming language. Early script languages were often called batch languages or job control languages. The set of instructions that make up a script often contain a set of commands that interact with the website user to provide content of control functions within the web page.

125 Not all users can access scripting when using assistive devices. In most cases, screen readers will not render scripting. Users with cognitive impairments may choose to deactivate scripting to minimize distractions. Therefore, for the web page to be accessible the web developer should provide a text alternative for any content that is contained within scripting, including all text and descriptions for any images. Users accessing the web page with a screen reader or with scripting disabled will receive this alternative text. To assess whether a web page provides direct accessibility to embedded user interfaces requires an extensive knowledge of web page programming, and an associated knowledge of scripting languages. The requirements for accessible embedded user interfaces stresses the need for ensuring that anything embedded on a web page is just as accessible as anything else on the page, whatever technology was used to create the embedded feature. Checking for embedded user interfaces is primarily a code-based examination, followed by the testing of the functionality to assure that the accessibility issue is addressed. For this study and for scoring purposes, to determine the accessibility of the embedded user interfaces, the testing approach followed the guidance put forth by the WebAIM organization, which is one of the leading providers of web accessibility expertise internationally (WebAim, 2011). Section 508 of the Rehabilitation Act (in the United States) requires that the functionality and content of embedded user interfaces scripts be accessible to assistive technologies such as screen readers. The researcher examined the source code of the web pages to determine the use of embedded user interfaces by searching for the use of SCRIPT tags. If scripting code appeared in a web page, the researcher loaded that page into the IE browser with scripting disabled, testing for equal functionality or the presence of an alternative accessible option. The evaluation processes or scoring schemes presented for each question do not attempt to capture all possible accessibility problems. These processes present the means by which the study evaluated the questions in order to support a scoring system that allows a comparison of accessibility performance from website to website. In addition to the above accessibility criteria, the researcher made note of key accessibility issues discovered during the review.

126 Data Analysis

Analysis of the data collected yielded findings related to the accessibility of federal websites that provide health care information. Methods used to collect the data involved four major analytic techniques: side-by-side policy analysis, web content analysis, expert review, and automated accessibility analysis. In combination, these analytic techniques address such areas as:  How current policy supports or inhibits website accessibility;  Compliance with Section 508 criteria, and  The problems that impede webmasters in attaining accessible websites. The following section briefly describes how each of the analytic techniques supports the integration of the data collected from the multiple methods used in this research study.

Side-by-side Policy Analysis

The side-by-side policy analysis produced qualitative data used to identify and assess the consistencies and incongruities that may affect attaining accessible websites. The policy review included indentifying key issues from the accessibility policy environment, and assessing the impact of those issues with respect to how federal agencies act to implement the criteria provided in Section 508. The results of the side-by-side policy analysis guided the development of study recommendations that attempt to improve the existing policies suggest new policies that may serve to promote accessible websites.

Web Content Review

The web content review identified and evaluated the use of web accessibility statements. By examining websites for statements relating to the commitment to provide accessible content, the analysis examined whether websites with web accessibility statements may have higher levels of accessibility than those sites without such statements. By comparing these findings to the automated testing and expert testing findings, could indicate that the existence of such accessibility policy statements correspond to a strong commitment by agency webmasters toward providing accessible websites. The web content analysis indicated the importance that agencies

127 attach to website accessibility. This analysis could indicate that those federal agencies that focus on issues of disability and have an accessibility-testing program in place are more likely to possess accessible websites.

Expert Review

The expert testing method examined the sample websites for compliance with Section 508 standards. Expert testing provided a more thorough evaluation of a website‘s accessibility compared to automated testing. By employing expert testing, the analysis provided an improved overall understanding of whether the selected federal health care websites meet Section 508 accessibility standards. This type of data analysis provided a more complete assessment of the accessibility of the websites, and better supported the understanding of the context of the accessibility problem. Expert testing does not typically find all the accessibility issues, but this type of data analysis can indicate the depth and breadth of accessibility problems (Lazar & Greenidge, 2006).

Automated Testing

The automated testing method identified the types and counts of instances of non- compliance with the accessibility criteria established by Section 508. Automated testing provided a relatively quick and easy method for testing the sampled sites to get an indication of compliance. This type of data analysis allows the testing a greater number of websites, providing a more substantial basis for making inferences about the sample population, but does not provide for a generalizable conclusion about all federal health care websites. When expert testing combines with automated testing, a fuller picture develops as to the true state of accessibility for this class of websites. Expert testing indicates the types, context, and frequency of accessibility problems, while the automated testing can indicate how widespread these problems may be throughout the class of examined websites. Together these methods provided an indication as to the amount of variation in accessibility observed from website to website.

128 In summary, data integration will include an overall analysis of all data to obtain the most comprehensive picture possible of the accessibility of federal health care websites. Data integration will extend to include the review and update of the conceptual framework.

Ensuring Valid and Reliable Data

This research study implemented several steps to assure a high level of data reliability and validity. For those methods that collected similar data elements, a comparison of the findings from each method supported consistency. Where pretesting has occurred, the researcher compared the results of pretest data to the subsequent data collection efforts, to assure consistency and to support internal validity. As websites are dynamic and undergo frequent updates, the researcher conducted all testing within as reasonably small period as possible to minimize any problems with changes that occur over time. To promote validity, experts on disability issues, web development, and website accessibility examined the data collection process and instrument design for appropriateness prior to conducting the data collection. Pretesting helped to assure that the data collection instruments recorded findings in a stable and consistent manner, and ensured strong inter-rater reliability. In addition, the researcher was the sole data collector, using a single dedicated personal computer. The use of multiple methods to examine website accessibility also supports reliability. This research effort used automated testing and expert testing as two methods that provide measures of accessibility. Both methods identified the type and count of instances where the criteria established within Section 508 was not achieved. This use of multiple methods that address the same question helped to confirm the reliability of the data collected. While the research does not support generalization across all federal health care websites, certain steps support the strength of the indications obtained. Steps taken to address issues of validity and reliability included:  Construct validity – A consistent approach to defining and measuring of accessibility results by using the standards defined in Section 508.  Content validity – The expert testing comprehensively accounted for each primary component of the Section 508 standards for accessible websites.

129  Criterion related validity – Data collected from each method was cross-compared where applicable to assure collective agreement and to support complimentary assessments.  Face validity – Study data collection instruments and protocols were pretested and reviewed by expert researchers. To promote validity and reliability, the researcher re-evaluated these criteria throughout the course of the research study and again at the conclusion of the data collection effort.

Justification of Methodology

The study design employed a multimethod approach using the following methods: side- by-side policy analysis, web content review, automated testing, and expert testing. The method selection was contingent upon the method‘s ability to support the purpose of the study and for its potential to generate findings that support the following study objectives: RO1: Evaluate compliance of federal health care web sites using Section 508 criteria. RO2: Identify the issues and barriers that affect attaining compliance. RO3: Identify practical recommendations to promote websites that are more accessible. This multimethod research approach supported the purpose of this study, which was to evaluate the accessibility of federal health care websites for individuals with disabilities, and to understand the factors that may impede attaining accessible websites.

Assumptions and Limitations of the Methods

This section identifies the major assumptions and limitations within this study. Also discussed are the steps to mitigate those issues. Key assumptions included: 1. Using multiple methods of data collection can provide a composite picture that provides a more comprehensive understanding than any single evaluation strategy (Babbie, 2001; Miles & Huberman, 1994). 2. Minimal changes to websites occur during the study period. To address this concern, the researcher executes each method in a timely manner to minimize the impact of such changes.

130 3. The automated software testing correctly identifies Section 508 violations and is a valid and reliable product. To address this concern, the researcher will select an automated testing product that has a proven history of successful application in the testing of websites for violations of Section 508 criteria. The researcher reevaluated the assumptions during the course of the study and at the conclusion of the data collection efforts. Chapter 8 includes a detailed assessment of the study‘s assumptions and limitations. A significant limitation is the inability to conduct user-based accessibility testing for this research effort. Studies indicated that user based accessibility testing involving persons with disabilities is the best way to determine whether websites are accessible (Jaeger, 2006, 2003; Kanayama, 2003; Mueller, 2003; Nielsen, 2000; Slatin & Rush, 2003; Witt & McDermott, 2004). However, no facilities existed locally that would support the conducting of this type of testing. Further, the costs and resources needed to conduct such user based accessibility testing exceeded what was available for this research effort. The methodological discussion must address two remaining limitations. First, within the web content review, federal agency policies found during the web content review may not reflect the agency‘s actual actions to attain compliance with accessibility requirements. Secondly, the sampling approach used in these methods does not support generalizability. The methodology section of this chapter identified the several efforts taken to strengthen the indications that may result from this research effort.

Summary of Methodology

The methodology supported the study purpose, which was to conduct an evaluation of the accessibility of federal health care websites for individuals with disabilities, and to understand the factors that may impede attaining accessible websites. This examination of accessibility issues employed multiple methods that included side-by-side policy analysis, web content review, automated testing, and expert review. The use of multiple methods provided a more comprehensive understanding of the accessibility of federal health care websites than any single method could accomplish.

131 The results of the data collection and data analysis efforts of this study provided an overview of the accessibility found within federal agency websites. The methodology also provided for the identification of problems that confront agency web mangers and impede their efforts to attain accessible websites. Findings from this research effort have the potential to improve website accessibility by revealing practical steps to address the problems and to identify ―best practices‖ that can be successfully adopted by current federal webmasters. Dissemination of such pragmatic solutions and best practices will advance the management of federal websites. This study attempts to make direct contributions to improving web accessibility by making recommendations in the areas of policy, management, web development and the conceptual framework used to guide this type of research. These contributions should result in making websites more accessible to all individuals, including those individuals with disabilities.

132 CHAPTER 4

SIDE-BY-SIDE POLICY ANALYSIS

This chapter presents a policy analysis of the current United States federal policy on website accessibility. The method of evaluation was a side-by-side analysis of the two legislative acts that most closely relate to federal web accessibility policy. The information policies examined are the Americans with Disabilities Act of 1990 (ADA) and the Rehabilitation Act of 1973, as amended by Section 508 in 1998. This side-by-side analysis of the two federal acts revealed significant differences in the ways these acts attempt to eliminate discrimination against individuals with disabilities. The focus of this analysis is from the perspective of ensuring website accessibility. The chapter begins by presenting a brief discussion of the existing policy environment surrounding website accessibility. Next is a presentation of the key issues, including the positions of the key stakeholders. Findings from the side-by-side comparison include recommendations that could address or mitigate problems identified in the analysis. The chapter concludes by presented a methodology for determining which policy recommendations might be both feasible and effective options for improving current web accessibility policy. The selection of these two legislative acts lies in their relevance to website accessibility. The Americans with Disabilities Act of 1990 and Section 508 of the Rehabilitation Act of 1973 together have the intent of establishing a legal mandate for equality regarding accessibility. The two acts are the most comprehensive federal civil rights statutes that address the rights of people with disability within the context of online services. The provisions provided by these legislative acts apply to federal websites and other Internet based services.

Policy Problems Surrounding the ADA and Section 508

Certain ambiguities, contradictions, and potential problems become evident when comparing the ADA and the Rehabilitation Act. Yukins (2004) identifies several areas of concern, but the following five issues from Yukins‘ work relate most directly to the topic of web accessibility policy: the terminology of accessibility versus accommodation; the application of

133 the ADA to information technology; the impediments to attaining web accessibility (as a failed policy effort); the cost concern (an unfunded mandate); and the lack of clarification regarding the phrase ―unfunded burden‖.

Accessibility versus Accommodation

Yukins‘ (2004) research indicates that Section 508 grew out of a ―sister‖ statute, Section 504 of the Rehabilitation Act of 1973. Section 504, much like the Americans with Disabilities Act (ADA), calls for making accommodations for those individuals with disabilities. Section 508, in contrast, calls for accommodating all persons with disabilities. The dichotomy here becomes the difference between a universal design approach versus and case-by-case accommodation effort. Universal design strives to provide products, services, and environments that are usable and effective for everyone, not just people with disabilities. The underlying goal in Section 508 is to ensure that every piece of information technology bought or maintained by the federal government is accessible, across a very wide range of potential disabilities. However, the ADA never reached this far. The ADA only calls for ―reasonable accommodations‖ for individuals with disabilities. This difference in perspective remains a major inconsistency in how these two acts view the provision of accessibility.

Applying the ADA to Information Technology

As a practical and legal matter, the courts have not interpreted the ADA to indicate the requirement for all information technology (IT) to be accessible. In one case, the Department of Justice (DOJ) has interpreted the ADA to mean that certain public accommodations must be made in order to make communications accessible (Jaeger, 2002). In another situation, the Department of Education has interpreted the ADA to mandate that universities be proactive in ensuring equal access to electronic communications (Jaeger, 2002). Meanwhile, the federal courts have been reluctant to apply the ADA broadly to information technology. Until the DOJ issues clarification as to exactly how the ADA relates to web accessibility, the current version of Section 508 marks the leading edge of federal policy that acts to ensure accessibility in information technology within the United States (Yukins, 2004).

134 Accessibility as a Failed Policy

The Rehabilitation Act banned discrimination, and supported by numerous court rulings effectively demanded reasonable accommodation for those with disabilities (Yukins, 2004). Yukins noted that while Section 508 was added to the Rehabilitation Act in 1986, the original language only established general guidelines for electronic accessibility, as compared to subsequent versions of Section 508 that provided clear and concise criteria and standards. Later refinement of Section 508 in 1992 strengthened the language but failed to mandate accessibility for all federal information technology, instead relying on individual agency guidelines to encourage accessibility (Yukins, 2004). These efforts to encourage accessibility in the federal agencies, by and large, failed because they lacked a real means of enforcement (Jaeger, 2006; Yukins, 2004).

An Unfunded Mandate

The intent of Congress in revising Section 508 in 1998 was to convey that the particular purpose of the proposed amendments to Section 508 was to require each federal agency to procure, maintain, and use electronic and information technology that allows individuals with disabilities the same access to information technology as individuals without disabilities (Yukins, 2004). Yukins discovered that the early legislative reports failed to understand the scope and cost of the Section 508 remediation effort. During the initial Congressional deliberations on Section 508, its policy development discussions lacked an appropriate understanding of the resources and expenses that would be associated with this piece of legislation (Yukins, 2004). There was never a concern that the effort was in dire need of subsidized funding to make implementation of this law possible. The unfunded mandate aspect of this law was a potential problem that would severely hamper its implementation, and consequently the accomplishing of its intent.

135 “Undue Burden” Controversy

As provided within the Section 508 statute, agencies are not required to modify or purchase accessible technology (as websites) if doing so causes an undue burden. Section 508‘s statutory language offered little clarification on what would constitute an undue burden. The undue burden issue directly relates to how agencies are to pay for accessibility. The undue burden clause assumes that an agency will be able to reallocate funding from existing sources to pay for accessibility. Only if the agency is unable to internally reallocate funding may an agency then claim an undue burden (Access Board, 2009). Yukins (2004) maintains that the wording of Section 508 was inadequate to convey clearly the Congressional intent behind the undue burden provision, and thus left unanswered the hardest question posed by accessibility: who is to pay for it?

Key Issues and Weaknesses of Section 508

Section 508 has been in existence since 1998, but recent studies show a poor level of accessibility for federal websites (Jaeger, 2006; West, 2004, 2002). As an information policy with the intent to attain website accessibility, this policy has fallen short (Jaeger, 2006). The most important factors that contribute to the poor success of this policy are failure to enforce, lack of central oversight, government inaction, and the need for new standards and definitions (Paciello, 2007; McKean, 2007).

Failure to Enforce

Paciello (2007) indicates that one of the key weaknesses of Section 508 is the lack of enforcement or commitment to Section 508 standards by the federal agencies. This viewpoint is shared by others, who have also noted that Section 508 lacked an enforcement mechanism, which resulted in few agencies complying (McKean, 2007; Jaeger, 2006). McKean indicates that a later amendment of Section 508 partially addressed the enforcement issue, but what resulted was a change that mostly affected enforcement from an industry point of view (the provision of accessible products). Since that change of Section 508 in June 2001, the primary

136 enforcement focus has been on private industry, with almost no oversight to monitor compliance by federal agencies.

Lack of Central Oversight

When it comes to federal agency compliance, some experts claim that the Section 508 amendment has some enforcement provisions (McKean, 2007). If a federal agency is non- compliant, individuals may take individual or group legal action to sue. However, the true nature of the problem lies in that Section 508 has not established or defined any special governmental entity to oversee enforcement (McKean, 2007). Such an oversight agency could be more proactive in promoting compliance and in examining agencies websites in an effort to both establish accessibility and to deter embarrassing and potentially costly lawsuits. McKean (2007) indicated that this lack of a central enforcement agency is a known problem and openly acknowledged within the federal government. Terry Weaver, director of the Center for Information Technology Accommodation at GSA has stated that ―Section 508 doesn‘ put anyone as the central authority‖ and the only real monitoring of Section 508 compliance occurs when lawsuits are filed (McKean, 2007, p. 1).

Government Inaction

Paciello (2007) indicated that the thinking of the federal agency culture was that accessibility compliance was to be achieved by ensuring that industry designed, developed, and delivered accessible electronic and information technology. Section 508 includes wording that provides for the acquisition of only those products that are accessible. However, for the most part, agencies have neglected to include Section 508 requirements within federal contracts. Without the making of Section 508 requirements explicit in federal contracting and procurement efforts, industry has no motivation to invest money and resources required to enhance their products for accessibility. Without such explicit documentation placed in all federal contracts, industry has little, if any, motivation to comply. The implication to web accessibility is that when federal agencies contract for web development efforts, the need for accessible websites may not be contractually addressed/required due to poor federal contracting procedures.

137 The impact of this failure to motivate and require compliance affects federal employees and the broader citizenry. OMB cited that ―When properly implemented, Section 508 improves the accessibility of government information and data, and ensures government [information technology] is accessible to Federal employees and citizens with disabilities. Building an accessible infrastructure creates an environment for hiring persons with disabilities‖ (Paciello, 2007, p. 1).

Outdated Standards

Experts levy another major criticism of Section 508, claiming that the Section 508 provisions are outdated, which serves to further complicate attaining compliance (Gunderson, 2009; McKean, 2007). Gunderson (2009) identified significant weaknesses in the way that Section 508 criteria have addressed making web pages more accessible. These shortcomings relate to issues of navigation (as in the use of headers and lists), and to image requirements (in matching images to stylize text and to image maps). Further, Section 508 does not appropriately address accessibility related to language choices (specifically towards retaining and providing accessible features when the user requests language changes). As technology continues to advance, existing Section 508 criteria require ongoing update to ensure continued web accessibility. When Congress originally developed the web accessibility standards for Section 508, there was a very different ―technological landscape.‖ Web accessibility was primarily for an Internet that consisted of static websites, accessed from personal computers. The current technology landscape includes complex and dynamic web applications, accessible by tablets, cellular telephones, and a wide array of mobile devices. When Section 508 was written, Congress could not have envisioned the emergent technologies that are now commonplace. The U.S. Access Board has attempted to address those shortcomings by appointing an advisory committee in 2006 to review and rewrite the accessibility criteria within Section 508. Indications are that the Access Board had underestimated the scope of that tasking, as the committee has discovered interoperability problems with new technologies and other shortfalls in the provision‘s standards (McKean, 2007). That effort remains incomplete, as the Access

138 Board has not presented Congress with any new proposals for legislation. This effort needs revitalization, if there is to be any hope for improving Section 508 criteria (McKean, 2007).

Need for New Standards

The prior section indicated that the existing Section 508 criteria needs to be updated, while this section presents the case for taking the next step, which is to develop future standards that will help ensure accessibility for newly developed technologies and to address emerging technologies and convergent technologies (McKean, 2007). The challenge to rewriting Section 508 resides in the need to create guidelines that not only work today, but will also work with upcoming technologies. The new technological devices of today have made many of the old Section 508 criteria obsolete, indicating that the focus of updating Section 508 needs to be on the technologies of tomorrow (McKean, 2007). McKean indicated that while it is now possible to accommodate people with a much wider range of disabilities than ever before, many of the old- solution strategies currently identified in Section 508 do not work with the new and emerging technologies. In their current form, the Section 508 standards barely acknowledge even well established web technologies such as Bluetooth, streaming video, and Flash animations (McKean, 2007). The growing field of web applications poses another major hurdle that requires addressing in the next rewrite of Section 508. While federal agencies have made some progress keeping their websites up to date, they now face the fast-growing use of web applications that have created new accessibility problems. McKean (2007) stated that the Access Board committee for the rewrite of Section 508 may be comfortable with website accessibility issues and software application standards, but they face a far greater difficulty in defining Section 508 standards for web applications.

Need for New Definitions

Efforts to rewrite the Section 508 standards must also address the problem of defining the term disabled (McKean, 2007). Section 508 requires expanded coverage that addresses the disabled and the disadvantaged. These two groups are legally quite distinct. From that legal

139 perspective, Section 508 as currently written does not extend its civil rights protections to those individuals that fall into the classification as being disadvantaged. As an example, people with weakness in their hands or fingers need keys that are easier to strike than standard keys. However, a keyboard meeting that requirement might not be appropriate for people with tremors, who might inadvertently hit multiple keys if they had to use such a keyboard.

Underlying Assumptions

The ADA and the Rehabilitation Act are the first and most significant pieces of legislation towards establishing the rights of individuals with disabilities (Jaeger, 2004). These major legislative efforts have established a solid legal framework for requiring website accessibility (Section508, 2009). However, the mere act of passing these laws has not made accessibility a reality. Actually, several broad based studies indicate that overall efforts to attain website accessibility have fallen dramatically short (Jaeger, 2006; West, 2004, 2002; Kruse & Hale, 2003; Kruse & Schur, 2003; Hignite, 2000). These public laws demonstrate a concern for assuring equal rights and access for those with disabilities. However, the intent of the laws, being to achieve accessibility, has not been accomplished (Jaeger 2006). The following presents a discussion of what may have been significant weaknesses in the underlying assumptions that existed during the development of these policies. Each of these assumptions are briefly discussed as to the role that they could have played in deterring these legislative acts (ADA and Section 508) from fulfilling their intended goals.

Assumption: The mere passing of a law will make the intent a reality.

It is perhaps a sad but true situation that despite the importance of a law, its implementation may not occur. Any number of reasons could exist for that situation. These reasons could be examined as separate lesser assumptions that together combine to indicate that the legislation could have been much stronger, had they been addressed.  Unfunded mandate: Laws regarding civil rights tend not to have funding allocated for their implementation (U.S. Commission on Civil Rights, 2002). However, the costs of redesigning websites to be accessible can be significant. Concerns for return-

140 on-investment and the need to allocate funding to projects with a perceived higher priority may trump issues of equity and fairness. The assumption was that entities would willingly reallocate the necessary funding to make their websites accessible, which has proved not to be the case.  Culture change (training and skills): Accessibility has not had the coverage or exposure that other web issues have enjoyed. As web technology is a relatively new area of science, experiencing rapid technological growth, web designers and web managers have not had significant training or exposure to accessibility requirements (Tang, 2001). Section 508 served to make a dramatic impact on the web technology field, causing a change in the way website testing and development was conducted. The need to retrofit or integrate for accessibility may have caught the web managers without the necessary training or skills required for managing this new task. The assumption was that Section 508 would have little impact on the federal agencies‘ web management culture. For those managers that attempted to integrate accessibility and comply with Section 508, they initially found themselves scrambling to acquire the skills and support needed to achieve compliance.  Culture Change (Awareness.) The legislation failed to provide a set of programs or activities that would serve to educate the web mangers about the need for accessibility and to explain the legal requirements that now exist. Without some proactive set of events and activities, the web managers would have little opportunity to become aware of this major change. An awareness program would likely have served to influence the attitudes and behaviors of the web managers, resulting in deliberate actions to make websites more accessible. The error in assumption here was that change (culture change) would occur without any sort of awareness activities to encourage and facilitate this change to the way websites are developed and tested.  Enforcement: These two legislative acts make little provision for the oversight by any federal agency (Yukins, 2004). Notably missing from Section 508 is any significant oversight entity to monitor and enforce the requirements put in place by this act. The act contains no terms that address the need for active and ongoing oversight, auditing, enforcement, and imposing penalties for non-compliance. The

141 assumption was that federal entities would rise to the call of the legislation and would make appropriate efforts to comply fully with the intent of the law.

Assumption: Definitions as understood by the drafters of the legislation are understandable in the same way by those responsible for implementation.

A good example of this situation relates to the use of the terms ―undue burden‖ within Section 508. Section 508 has a clause that allows entities to defer from making websites accessible, if that cost would be an undue burden. Many entities have chosen to consider any cost, regardless of how small, to be an undue burden (McKean, 2007). This is a major weakness of Section 508, as the legislation failed to define the phrase undue burden, or to provide explicit guidance that would allow an entity to determine the actual size of the undue burden. The faulty assumption in this case was that no clarification or guidance was necessary, assuming that entities would somehow arrive at a collectively correct understanding of the term and would make appropriate decisions in regarding an undue burden. That undue burden provision, depending on its interpretation, could allow any website owner to not comply with the intent of the intent of Section 508, based on their own unique interpretation of what constitutes an undue burden. The lack of a specific definition or supporting guidance for the term undue burden provided a much too easy way for web managers to get around the intent of this law. These assumptions may have been mere oversights by the drafters of this policy. It is also possible that the unclear language was intentional, to require specific interpretations of the law to occur through the judicial system, on a case-by-case basis.

Key Stakeholder Groups

Stakeholder analysis is the identification of a project's stakeholders, and includes an identification of their concerns, interests, or issues. Stakeholders are persons, groups, or institutions with interests in a policy issue. Key stakeholders are those who can significantly influence, or are affected by, the success of the proposed policy options that may arise from a policy analysis effort. An examination of the stakeholder‘s concerns and interests can serve to

142 identify possible conflicts of interests between stakeholders that make a particular policy option less desirable. Table 4.1 identifies a few of the major stakeholder groups that have varying interest levels in federal policy developments on website accessibility. The definition of these groups is not singular in nature, in that members of any one group may also be members of other group. The table indicates that the website owners would be very interested in new web accessibility policy, as they are legally required to comply with any new guidelines. They also have a stake from a financial perspective, as modifying and testing existing websites to comply with new accessibility regulations can be quite costly. The second group of stakeholders, the federal government, has the responsibility to ensure the addressing of concerns for fairness and equitable access for all federal website users. The third group, being federal website users with disabilities, wish to receive fair and equal access to websites as a basic civil right. Each group has stakes in assuring website accessibility and maintains an important level of influence in shaping political decisions in this area.

ADA Amendments Act of 2008

During the years since the early 1990‘s, Luongo (2008) stated that Congress grew displeased by several prominent Supreme Court decisions handed down in ADA related cases. Congress felt that the court‘s interpretations were too narrow and not in line with the original intent of the ADA. Congress reacted by passing the ADA Amendments Act of 2008 (Public Law 110-325, ADAAA), effectively expanding the scope of the original law. Congress found that the Court has ―narrowed the broad scope of protection intended to be afforded by the ADA, thus eliminating protection for many individuals whom Congress intended to protect (Luongo, 2008).‖ The scope of ―disability‖ broadened to the maximum extent permitted by the terms of the ADA. This provision was included in the ADAAA in order to reinstate the broad scope of protection afforded by the ADA that, in the view of the Congress, the Supreme Court has improperly narrowed. The ADAAA also expanded and clarified prior ADA wording to provide an expanded coverage of the definition of ―disability‖ and more carefully described the degree of disability that would effectively qualify an individual for protection under this law

143 Table 4.1: Key Stakeholder Groups. Key Issues by Stakeholder Group Stakeholder Group Concerns or Issues  Ensure accessibility so they reach their entire customer base.  To comply with legislative mandates. Agency Website Providers  To conform to recent court rulings. (Owners)  To qualify for government funding and contracts.  An ethical responsibility towards assuring equity, fairness, and access for the disabled as a fundamental responsibility of good governance.  Ensure equity, fairness, and access for the disabled as a fundamental responsibility of good governance.  To accomplish this, the government would have an interest in the outcomes and impacts of relevant legislation. Federal  By measuring the levels of accessibility, the government would have an Government indicator that either the existing policies have been effective, or that new legislation may be required.  An analysis of the accessibility error by type and frequency would further serve to indicate what new legislation might look like or need to address.  To receive fair and equal accessibility to websites as a basic civil right as defined by the democratic foundation of this republic.  Testing would serve to identify impediments that impede attaining Website users with accessible website. disabilities  Knowing the types and frequencies of errors could lead to interventions that overcome these obstacles and result in improved websites.  Testing may also identify accessibility problems that are not currently addressed by current legislation.

Pending or Recent Policy Instruments

The Access Board is an independent federal agency devoted to accessibility for people with disabilities (Access Board, 2009). Created in 1973, the Board is now a leading source of information on accessible design. The Board develops and maintains design criteria for information technology, including the accessibility standards that are part of Section 508. In 2006, the Access Board (the Board) initiated a review and update of the web accessibility standards contained in Section 508. This review effort by the Access Board is not a formal policy review covering the entirety of Section 508, but only an examination of the existing criteria with the intent to validate and update the standards. As part of the review process, the Board consulted with various stakeholders that included: federal agencies, telecommunications and information technology companies, public

144 and nonprofit agencies and other organizations representing individuals with disabilities, accessibility consultants, academics, and international standards organizations. The goal of this review effort is identical to the intent originally held by Congress in developing Section 508. That intent is to develop appropriate web accessibility standards so that people with disabilities might achieve a level of access and use of information that is comparable to that of people without disabilities. In 2008, the Board issued its report with recommendations for new guidelines to be included into Section 508. The report recommended changes to both the substance and the structure of the standards and guidelines with Section 508. The report proposes to expand the scope/coverage of Section 508 by providing new language that addresses a range of issues that include: new and convergent technologies, market forces, compliance concerns, and international harmonization of web accessibility standards. The report recognizes the importance of providing standardization across governments worldwide, and to that end the committee coordinated its work with other standard-setting bodies in the U.S. and abroad (Access Board, 2009). As of this date, those recommendations have not yet been enacted into law.

Side-by-side Policy Analysis

Comparing the ADA to the Rehabilitation Act

The Rehabilitation Act of 1973 (as amended in 1992 and again in 1998 with Section 508), and the Americans with Disabilities Act of 1990 are civil rights laws that have the intent to protect individuals with disabilities from discrimination (Leuchovius & Parker, 2008). The Rehabilitation Act also goes beyond ensuring legal protections by providing for direct services to people with disabilities, which help them to become qualified for employment. An examination of the two laws in a side-by-side analysis shows how these laws serve to support one another (or work against one another), providing an indication of how separately and together they serve to establish information policy in the area of website accessibility. This section presents a side-by-side policy analysis of the Americans with Disabilities Act of 1990 and the Rehabilitation Act of 1973, with a focus on the provisions established by the 1998 Amendment of Section 508. This side-by-side analysis examines both acts, using criteria

145 that will reveal how the acts are similar, and how they differ in key areas. The criteria of evaluation selected for the side-by side evaluation are:  Legislative intent,  Means used to accomplish the intent,  Objectives identified within the respective acts,  Application scope,  Language style employed,  Costs of financial burden placed on the implementing entities (website owners),  Definition of ―disabled,‖ and  Action stance taken. These criteria derive from a thematic analysis of the factors and issues developed in the preceding discussion of the policy environment and from evaluative factors commonly employed to analyze legislative acts (Huberman, & Miles, 2002). Table 4.2 presents a summary set of the key findings revealed by the side-by-side policy analysis.

Intent

Findings. The intent of the ADA was to prevent discrimination against individuals with disabilities and did so with broad coverage. By comparison, the Rehabilitation Act had much narrower protections and subsequent court interpretations were drawn narrowly. Issues. The earlier Rehabilitation Act significantly influenced the development of the ADA in several important ways. First, Leuchovius & Parker (2008) indicated that the predominant sentiment among the ADA supporters was that the Rehabilitation Act alone was inadequate to end the widespread discrimination against people with disabilities that then existed. Secondly, the courts‘ interpretations of the antidiscrimination provisions within the Rehabilitation Act were very narrow in application. Congress felt that these narrow interpretations were inconsistent with the original intent of the law (Yukins, 2004). Congress reacted to these narrow interpretations by providing much broader protections in the ADA. The two acts are in conflict. The conflict does not stem from intent, but from the protections afforded and the scope of those protections.

146 Table 4.2: Key findings: Side-by-side analysis. Key Findings Side-by-side Policy Analysis Findings Criteria ADA Rehabilitation Act and Section 508 Designed to outlaw discrimination Rehabilitation Act, in general had Intent against disabled individuals; narrower interpretations. provided broader coverage. Uses the principles of A case-by-case application. Means to the Intent ―mainstreaming‖ and universal design. To support the integration of people To support the integration of people with Objectives with disabilities into the mainstream disabilities into the mainstream of of society. society. Applies the same requirements found Applies to federal agencies and to Application in the Rehabilitation Act to state and federally funded programs. local government entities. Language Broadly based, imprecise, with Precise standards and criteria. style undefined terms. Costs No limits to achieve accessibility. Limited, up to an ―undue burden.‖ Original Definition Individual with a disability. Individual with a handicap. of “disabled” Action Stance Reactive and adaptive. Proactive.

Recommendations. Looking comprehensively at both Acts, the narrow interpretations typically associated with the Rehabilitation Act could be corrected by combining the two Acts, while retaining the broader language and coverage found in the ADA. The two Acts closely align in intent and objectives, and by combining these two acts, the resultant legislation would benefit from a broader interpretation. Looking more narrowly at the two Acts, with respect to website accessibility, the formation of a new legislative act would allow the provisions of Section 508 to be removed from the Rehabilitation Act, and embedded into a document that uses the more encompassing, as the language used in the ADA.. This would allow for broader interpretations of the requirements to make website accessibility a mandate that applies to a greater scope of website providers (as opposed to only having federal applicability, as is now the case). This formation of a new policy would provide the opportunity to clarify (or align) the intent and scope of coverage for website accessibility.

147 Means to the Intent

Findings. The original Rehabilitation Act generally indicated that provisions to accommodate individuals with disabilities were to occur when deemed necessary by the responsible party. This indicates a case-by-case-approach to assuring equity and accessibility. The ADA employed the ―mainstreaming‖ approach, which was the integration of individuals with disabilities as fully as possible into society, regardless of the costs involved. When Section 508 amended the Rehabilitation Act, it adopted this ―mainstreaming‖ approach, supported by universal design concepts that would serve to ensure the accessibility of websites from the onset. The means to intent has another aspect, relating to the enforcement of the provisions within the acts. Neither act provides for an independent central authority with strong legislative and financial powers to appropriately study, audit, and enforce these laws. Issue. The ADA approach was more broadly based, in that it built-in accessibility to fit all of society, and was not a retrofit application. The ADA approach helped in structuring the language used in the Section 508 amendment to ensure a broader application than originally provided, and also attempted to have accessibly built into all products (as opposed to the prior case by case manner). A major deficiency within both acts is the lack of a central authority with the appropriate resources and mission to ensure compliance. Studies on website accessibility have found poor compliance (Jaeger, 2006; West, 2004, 2002), indicating that such a central enforcement agency is necessary. Recommendation. When Section 508 became part of the Rehabilitation Act, it used the approach of ―mainstreaming‖ by requiring that web managers employ universal design when developing websites. This alignment of language use strengthens the acts in their ability to accomplish their intent to ensure equal access to websites. To enhance the effectiveness of these two legislative acts, a strong recommendation is for the establishment of a federal entity that would serve as a central authority for the administration of web accessibility policy. This federal entity would need to possess full authority to administer and enforce the statutory requirements for accessible. Other government agencies, with similar structures for enforcing policy compliance, have proven to be very successful at attaining high levels of compliance with federal policies (GAO, 2009).

148 Objectives

Findings. The Rehabilitation Act and the ADA share a close relationship in their objectives. Both acts indicate that the objective is to support the integration of individuals with disabilities into the mainstream of society. Issue. The Rehabilitation Act and the ADA have several shared principles and objectives that are critically important to the cause of accessibility. The underlying theme within both acts is the tenet that people with disabilities - including individuals with the most severe disabilities - have the right to work (Leuchovius & Parker, 2008). This perspective is a major change in federal accessibility policy, as prior to the ADA government agencies assumed that most people with severe disabilities were unemployable (Leuchovius & Parker, 2008). The advent of the ADA required federal agencies to acknowledge that individuals with even the most severe difficulties have the ability and the right to work. Equity for all was the underlying concept. Recommendation. The researcher makes no recommendation for change, as both Acts align closely in their stated objectives. Both laws have the fundamental approach that it is necessary to support the integration of people with disabilities into the mainstream of society (Leuchovius & Parker, 2008). With these acts, it becomes unacceptable to provide separate (as in ―separate but equal‖) settings or programs as a way to establish equity in the workplace, unless such accommodations are actually necessary to ensure equal benefit.

Application

Finding. The Rehabilitation Act applies to federal agencies and to federally funded programs. The ADA has the same application scope as the Rehabilitation Act, but extends the accessibility requirements to state and local governmental entities. Issue. The Rehabilitation Act protects the civil rights of persons with disabilities. It prohibits discrimination based on disability by the federal government, federal contractors, and by recipients of federal financial assistance (Leuchovius & Parker, 2008). Recipients of federal funding are required to make their programs accessible to individuals with disabilities. While these protections are limited to apply only to programs or businesses that receive federal funds, it served as an important model for the ADA.

149 The ADA extends that requirement for accessibility to apply to state and local government entities. Therefore, the biggest difference in ―application coverage‖ is that the Rehabilitation Act applied to federally funded programs and that the ADA extended coverage to state and local government funded programs (Leuchovius & Parker, 2008). Together the ADA and the Rehabilitation Act ensure that all government-funded programs (federal, state, and local) are covered. The ADA further extends its applicability by providing rights protections for people with disabilities to the private sector. These ADA provisions apply to private businesses and nonprofits, requiring them not to discriminate against people with disabilities in providing goods and services, and must provide accessibility features when they can afford to do so. Increasingly the courts interpret this provision to include web-based applications, as electronic commerce and information resources (Klein, 2007). Recommendation. This misalignment of application coverage directly affects web accessibility. As the Rehabilitation Act explicitly addresses website accessibility, developing a new policy could ensure the same application coverage as the ADA. By extending the application coverage in this way, the mandate for accessible websites would more clearly extend to federal and state governmental agencies, and could extend to cover private businesses. This extension of application would further ensure that individuals with disabilities would be able to access web sites from all sources, public and private.

Language Style

Finding. The Rehabilitation Act used language that included precise standards and specific criteria. In comparison, the ADA language was broader in coverage, relatively imprecise, and in many cases used terms that were not clearly defined. Issue. The more broadly based language style used in the ADA was an attempt to provide wider coverage. This use of broad terms was intentional to counteract the narrow interpretations previously handed down by the courts. The Rehabilitation Act language was less broad in comparison, with the notable exception being Section 508. (Section 508 was a later amendment to the initial version of the Rehabilitation Act). The lessons learned from the success of the initial Rehabilitation Act, and the subsequent narrow court interpretations, led Congress to

150 be very precise in defining the standards and criteria specified in Section 508 (Leuchovius & Parker, 2008). In turn, the ADA influenced the Rehabilitation Act during its reauthorization in 1992. The 1992 amendment to the Rehabilitation Act reflected the language, goals and objectives of the ADA. The Rehabilitation Act adopted the ADA's emphasis on integration as its own, and translated the principles and policies of the ADA into government rehabilitation programs for people with disabilities (Leuchovius & Parker, 2008; Yukins, 2004). In comparing Section 508 text to the overall language used in the ADA, it is evident Congress left the fleshing out of the ADA details to the judiciary (Quinn, 1991). Recommendation. The recommendation is that any new attempts to develop accessibility policy should be both clear and precise. Such improved language would help to ensure obtaining the intent of the legislation, as the courts have drawn very narrow interpretations from conservative positions. While broad language may ensure broader coverage (as intended by the Congress), when specific objectives are the goal, the language used must be equally precise.

Costs Imposed and Cost Limits

Finding. The ADA did not place limits or restrictions on the costs that resulted from making the necessary modifications that would ensure achieving accessibility. However, when looking specifically at the topic of website accessibility, the Rehabilitation Act does address a limitation on such costs by including an ―undue burden‖ clause. Issue. The costs to implement accessibility compliance under the ADA are significant and not considered to be in balance when compared to the potential benefits (Leuchovius & Parker, 2008). The ADA approach of building-in accessibility for every item did not allow for decision-making based on such financial considerations as return on investment. The ADA‘s intent was deeply rooted in assuring accessibility as a universal civil right and afforded to everyone. With the ADA, there were really no limits on the costs incurred to produce or provide accessibility. By comparison, the Rehabilitation Act would provide accessibility in a way that gave greater attention to the associated costs. The Rehabilitation Act‘s approach of a case-by- case consideration to providing accessibility would serve to isolate and limit expenditures when providing accessibility. Section 508 also had a cost conscious characteristic, as it provided an

151 ―out‖ for entities that felt that they would incur undue expense in attaining accessibility. However, this clause proved to be an all too easy escape mechanism that entities could use to defer from complying with the intent of the law. Recommendation. The undue burden clause needs clarification and improved definition. Section 508 uses language that is clear and precise, but the lack of understanding surrounding this terminology is repeatedly identified as a major obstacle to assuring website accessibility. At the time when Section 508 was written the ―state of the art‖ for web development efforts required significant resources, investment, and talented manpower to create, modify, and maintain a website. It may be that such a costly scenario no longer exists, and similarly the ―excuse‖ of ―undue burden‖ is no longer a valid argument. The recommendation is to conduct research into determining if it is appropriate to delete the ―undue burden‖ clause in its entirety. If that research indicated that providing accessible websites could be costly, then guidance that is more specific is needed to clearly define the criteria for an ―undue burden.‖

Definition of “Disabled”

Finding. At one time, the terms "disabled" used in the ADA and "handicapped" used in the Rehabilitation Act were described by Congress as "comparable" (ASH, 2009). As public sentiment changed, the use of the term "disability" instead of "handicap" and the term "individual with a disability" instead of "individual with a handicap" represents an effort by Congress to make use of up-to-date and widely accepted terminology. Currently both the ADA and the Rehabilitation Act use the same terminology and the same definition of disability (ASH, 2009). Both acts define "disability" to mean a physical or mental impairment that substantially limits one or more of a person's major life activities, a record of such impairment, or the perception that one has such impairment (ASH, 2009). Issue. The two acts are in alignment regarding the definition of ―disabled,‖ as both the ADA and the Rehabilitation Act use the same terminology and the same definition of disability (ASH, 2009). Recommendation. None.

152 Action Stance

Finding. The ADA language indicates a stance that is reactive and adaptive, while Section 508 of the Rehabilitation Act uses language that is more proactive in nature. Issue. The requirements of the ADA specify that institutions must be responsive to the needs of individuals and make programs and services accessible to them on request. This differs from the proactive requirements of Section 508, which state that all technology purchased by the federal government must be accessible whether it is initially intended for use by a disabled employee or not (NCAM, 2009). Recommendation. The ADA should adopt language that is more proactive in stance. Section 508, by using proactive language, is a much more effective means of assuring website accessibility. The proactive nature helps to ensure that web developers build-in web accessibility features during the developmental phase, and not ―added on‖ as an afterthought. This is a marked difference from the ADA stance, where it appears that in many cases the ADA is working as a coercive law, not as a signal to build access to public environments (Baker, & Kaufman- Scarborough, 2001). Any effort to develop new web accessibility policy should similarly use language that takes a proactive approach. Making the ADA more proactive in nature would not necessitate extensive rewriting of the legislative act. A more pragmatic approach would be to recommend research that leads to supplemental guidance for agency webmasters. An approach similar to this was proven to be very successful when applied sections of the ADA that dealt with physical accessibility. The resulting supplemental guidance interpreted the ADA language and provided more specific proactive steps that helped to ensure a higher level of accessibility (Barron, 2010). For web accessibility, this approach would reflect comprehensive research of web accessibility issues, providing documentation and resources to promote compliance with requirements and build an awareness of user needs. This supplementary guidance would adopt a more proactive approach by instructing the web designer on specific tools, techniques and processes that promote web accessibility.

153 Summary of Comparisons

There are substantial differences between the accessibility provisions mandated by the ADA and by Section 508 of the Rehabilitation Act. Although the two Acts overlap, Section 508 applies significantly more stringent technical requirements for electronic information technology accessibility. While the ADA requires federal government entities to provide reasonably equivalent access to information, the method of delivery used to attain that access remains flexible. Under Section 508, federal entities must offer technology through its website that allows all individuals, with or without disabilities, to obtain the information directly through the site itself (Office of Compliance, 2001). Thus, Section 508 requires that the means to access information exist within the electronic medium itself. The side-by-side analysis of policy documents provides a structured method for comparison based on key evaluative criteria as ambiguity, contradictions, and inconsistencies (McClure, 2002). This type of analysis can help identify the gaps in policy coverage or areas of conflict that may suggest options that could strengthen or better align overall information policy environment. This comparison reveals that as the years progressed, information policy evolved to provide broader protections to individuals with disabilities. Individuals with a disability were increasingly perceived as people that should not just be accommodated, but should have equal and fair access to all the opportunities offered within mainstream society. While the ADA and the Rehabilitation Act had the intent of eliminating discrimination against individuals with disabilities, their approaches and concepts of implementation differed significantly. Section 508 language was both targeted and specific, when compared to the more broadly stated terms found in the ADA. Perhaps the most significant difference (as initially defined in each act) was in how these acts portrayed the intended recipient of these civil protections. The Rehabilitation Act perspective was that of accommodating an ―individual with a handicap,” versus the ADA perspective of providing universal accessibility to “individuals with a disability. ―

154 Analysis of Key Recommendations

While the side-by-side policy analysis reveals several key areas for improvement in federal policy on website accessibility, there are additional policy issues that need to be addressed. This chapter began by reviewing the existing federal policy environment surrounding website accessibility. A cursory analysis of that content reveals broad themes that were significant weaknesses in the current policy environment for website accessibility. At a minimum, the options for policy improvements must address the weaknesses identified regarding Section 508. As presented earlier in this chapter, Paciello (2007) and others (Jaeger, 2006; McKean, 2007) have identified specific weaknesses (identified herein) as being: failure of enforcement, lack of central oversight, government inaction, outdated standards, and the need for new standards and definitions. Presented are five recommended options that could improve the existing federal legislative environment that deals with website accessibility.

Option 1: Establish a Central Authority

This option proposes the establishment of a federal entity that would serve as a central authority for the administration of web accessibility policy. The organizational elements for this office would derive from the model of authority and structure that has proved successful for the Internal Revenue Services. The Internal Revenue Service (IRS) is a centralized authority/office for the administration of tax policy, to include the audit and enforcement of that policy (IRS, 2009). The operational model that the IRS uses to administer this policy is one of the world‘s most efficient (IRS, 2009). This model provides full authority to administer and enforce the internal revenue laws and empowers the agency to enforce these laws. The General Accounting Office (GAO) indicates that this successful model has attained 86-95% compliance with federal tax policy (GAO, 2009). This model should be adapted for the administration of web accessibility policy in the following way. The Executive Office should act to establish a centralized organizational entity with the authority to administer, audit, and enforce web accessibility policy within the United States. This proposed centralized office would need to be appropriately empowered by statue to

155 carry out these duties and have the ability to enforce the policies through the application of penalties that are appropriate to motivate compliance. This proposed office would conduct random audits of federal websites to test for compliance. This testing program could be a structured blend of accessibility testing approaches to include automated testing, expert testing, and true accessibility (user based) testing. Such a structured program would allow the organization to conduct testing in appropriate depth and thoroughness, while doing so in a cost effective manner. Part of the proposed enforcement process would require the direct support of the Office of Management and Budget (OMB, 2009). OMB would need to work hand in hand with this centralized office to apply the enforcement penalties that would likely take the shape of budget decrements for agencies found to be unable or unwilling to attain compliance with accessibility standards. Much like the IRS model, this agency would have the responsibility for providing various forms of assistance to web managers that would aid in their attaining compliance with web accessibility policy. This assistance could take the form of training, awareness programs, web- based resources, and other assistance necessary to support agency efforts to attain web compliance. The expectation is that implementing this type of model would have results similar to those attained by the IRS. This office could provide federal web managers with top quality service by helping them better understand and meet their accessibility related responsibilities, while applying accessibility policy with integrity and fairness to all (IRS, 2009).

Option 2: Improving Federal Agency Awareness and Training for Web Managers

Another proposed approach to assuring a higher level of compliance with the web accessibility policy is to better train our federal leaders and web managers. An awareness- building program would have the intent to institute an organizational culture change throughout the federal ranks. Executive training would advise leaders on the ethical and legal need for accessibility, while webmasters would receive training that addresses resources, testing programs, and technical tools that promote accessible web sites. By requiring each web manager to complete a comprehensive training course on web accessibility policy and resources, this

156 action should result in improved levels of compliance. Successful completion of the proposed training would result in the web managers receiving a web accessibility certification. Awareness building and training would appear to be a strong necessity, as some agencies‘ websites display an utter misunderstanding of accessibility law (Section 508) and what those laws require (Robb, 2001). Some agencies apparently were never clear about how or when they had to be in compliance (Tang, 2001). During the initial implementation of Section 508, little was done within the federal sector to make web managers aware of the imposed accessibility requirements (Jaeger, 2004; Tang, 2001). Further, the federal training centers for government information technology managers did not offer significant training in the legal requirements and the technical requirements regarding web accessibility (Tang, 2001; Yukins, 2004). Conducting awareness building and professional training courses is a successful method for modifying the attitudes and behaviors of professionals (Fender, Prentice, Gorst, et al., 1999). Bloom (2005) found that training was one of the most effective tools towards improving attitudes and subsequently changing the targeted behaviors. Other studies have confirmed this residual positive impact of training on subsequent behaviors (Silagy & May, 1997; Maclure, Allen, Bacovsky, et al., 2006). This type of behavioral modification approach has also been determined to be cost effective, when subjected to a formal economic analysis (Mason, Freemantle, Nazareth, et al., 2001; Raker & Boyd, 2007). Training generates the greatest amount of desired residual change (over the longer term) with the least amount of time spent with the target professional (Balas, Boren, Brown, et al., 1996). This proposed training/certification program could provide comprehensive coverage on all facets of web accessibility to include addressing the legal and ethical requirements; locating accessibility related resources; covering web accessibility development and testing tools; and providing guidance on developing an integrated accessibility testing program. The web accessibility training certification would likely be a popular opportunity for the federal web managers, as it would improve the professional stature of the web manager and serve to improve their career marketability. Additional inducements could help to ensure a high completion rate for this certification effort. Such an inducement could be the issuing of a mandate for completion of this training within a short but reasonable period. Subsequent salary increases would require successful completion of the training. Such a mandate should extend to include periodic re-training. OMB would administer the oversight and enforcement of this

157 mandatory training, as an addition to their existing responsibility for overseeing the government‘s investment in information technology (IT) resources. In this way, the OMB IT oversight role would extend to encompass the human resource aspect, towards ensuring the certification of web managers and that the mandated training occurred in a timely fashion.

Option 3: Review of Section 508

McKean (2007) asserts that the technical standards provided within Section 508 are outmoded and need update. Having outmoded accessibility standards will not appropriately serve the intent of the Section 508 legislation, which is to ensure full web accessibility. McKean‘s study was not a comprehensive examination of all the Section 508 criteria, nor did it address the creation of guidelines that will work with newly developing technologies. McKean indicated that many of the newly developed technological devices (as web 2.0, PDAs, Second Life) are not adequately addressed by Section 508. Also, new delivery technologies as Bluetooth and streaming video have made certain Section 508 criteria obsolete. Section 508 needs a more extensive review and overhaul, so that it takes a more forward looking approach towards accomplishing three important objectives: cull obsolete standards; develop standards that address the current state of technology; and provide standards and guidance that address burgeoning technologies. McKean‘s work makes a fundamental point very clear that the technological landscape has dramatically changed since Section 508 was originally developed. Research is needed that has the goal of updating web accessibility policy and standards to reflect the current and emerging technological environment. The Access Board recently completed a review of the existing Section 508 standards, but as yet these recommendations have not had Congressional action (McKean, 2007). While this is a step in the right direction, it is only one part of what needs consideration. Section 508 requires reviewing in its entirety, to include restructuring, so that it addresses the current and future state of technology. The indication is that it is time for a new policy effort. As a current policy document, Section 508 fails to address key elements that have emerged from technological developments, and suffers from a number of issues previously addressed. A new web accessibility policy could expand to cover not only the disabled, but needs to extend its coverage to include the

158 disadvantaged. Section 508 as currently written does not extend its civil rights protections to those individuals that fall into the classification as being disadvantaged. Such a policy development effort would extend to better define the term undue burden. Currently the poorly understood and often misapplied undue burden clause allows entities a much too convenient way of eluding the intent of Section 508.

Option 4: Ongoing Commitment to Research

Research has the potential to offer great potential for benefit towards improving web accessibility. A two-phase research effort could provide a current focus and a continuing focus on achieving accessible websites. Research efforts could address current accessibility concerns, while also staying abreast of the technological developments that will continue to impact web accessibility. The existing governmental organization that currently has the responsibility for Internet oversight (the Access Board) could administer these new research efforts. An Executive Direction to this Board from the Administration could initiate the program of activities required to accomplish these research efforts. Phase 1. Research in this phase would have an initial or current focus. The focus would be to determine the ―real‖ impediments that deter attaining accessibility. The research would attempt to identify the real world problems that webmasters and organizational leaders face in their attempts to comply with the web accessibility legislation. A combination of data collection methods (as surveys and interviews) could be the start of a systematic and rigorously structured research effort. The surveys could be exploratory in nature, to identify the types of problems faced by web managers and to identify the resources needed to overcome them. The research could then narrow to attempt to prioritize these problems. This prioritizing approach would help to ensure that resources go to the problems that would offer the greatest improvement in web accessibility. This research effort would have a current focus, as it would address the problems currently faced by the web managers. Committing to doing this kind of research on a continuing basis would ensure an ongoing effort to identify and address those ―current‖ problems that face web mangers. In this way, the research ensures a ―currency‖ or freshness of problem identification. From the start, this research would provide a valuable set of benchmarks and

159 metrics that could allow the ongoing evaluation of the effectiveness of these proposed policy options. A second part of the current focus would be to study the accessibility laws and initiatives of the various state governments in search of a set of best practices and procedures related to assuring web accessibility. A recent report on electronic government in the United States found that a number of the states have more accessible government websites when compared to federal level websites (West, 2002). West (2001) indicates that U. S. government websites are the most accessible when compared to other nations. The indication is that that there may not be too much to learn from other national governments‘ accessibility practices. While implementing accessibility in the federal government may be a much larger task than at the state level, there will likely be important lessons from those state and local government efforts to become fully web accessible. Phase 2. Research in this phase would have a future focus, by developing tools that would help ensure accessibility in new web development efforts. This research would assist in the design of improved web design tools that build-in web accessibility features into the development of the website. Not only could these tools provide accessibility features, but these tools could also include sophisticated prompts to the web developers that would serve as reminders to address web accessibility concerns, while the web programming code is undergoing development. These tools could also integrate internal accessibility testing, providing ongoing testing throughout the development process. This inclusion of testing throughout the development cycle is a best practice towards assuring accessibility (Slatin, & Rush, 2003). These research projects could fall under the responsibility of the centralized agency that is established to oversee web accessibility (as identified in Option 1) or by the current Access Board. The selected agency would need to have funding earmarked for conducting this ongoing research either internally or through the use of grant programs. These two approaches offer strengths and weaknesses that would need further evaluation before choosing one approach over the other. The optimal solution may even involve using a blend of internal (office staff) and external research activities (grant funded programs).

160 Option 5: Department of Justice Clarification

One simple but significant option that could serve to motivate compliance with Section 508 relates to the controversy in the courts over the applicability of the ADA concept of public accommodation. The proposal is that the US Department of Justice needs to intercede in this controversy by classifying websites as being public accommodations that are accountable to the standards of the ADA (Jaeger, 2004). If the ADA and ADAAA scope of application was indicated to clearly cover federal website providers, then the services they provide would have to be accessible equally to individuals with disabilities. To accomplish this change, the Department of Justice needs to make a definitive interpretation that asserts that the ADA applies to federal websites in the context of being public accommodations. The impact of such a clearly defined interpretation would serve to clarify the web managers‘ understanding as to why websites require accessibility features. Such clarification would also serve to permit the bringing of class actions suits against website providers that failed to comply with Section 508 criteria. Class action lawsuits are an increasingly pervasive force in today's world and would serve to support compliance (Gibson, Dunn, & Crutcher, 2009). Legal liability can be a powerful force in assuring compliance with the goals of information policy. One such example is a recent major class action suit filed by the National Federation of the Blind against Target Corporation, which is currently working its way through the California courts (Klein, 2007). The lawsuit alleges that Target failed to, and refused to, make its website accessible to the blind, in violation of the Americans with Disabilities Act. As recently as the fall of 2007, a federal district court judge determined that the case had merit as a class-action suit and ruled Target accountable for the accessibility claim. While similar litigation developed against other large corporations over the last decade, most of the cases resulted in private settlements (Klein, 2007). However, precedent-setting cases such as this Target case get headlines and quickly establish increased awareness throughout the legal and corporate communities, as well as the public sector. This increased awareness, propelled by concern over potential liability claims, would certainly help to motivate website managers to support and attain full accessibility.

161 Analysis of Policy Options

Each of these five options could serve to improve the federal information policy for web accessibility. The evaluation of these options will determine if any individual option or combination would best meet the objective of improving the accessibility of federal websites. The four criteria used in this evaluation process were effectiveness, feasibility, cost, and political impacts. Assessing the factors of effectiveness, feasibility, political impacts, and societal impacts requires making direct comparisons to similar applications of the proposed option. For example, if the proposed option were effective in a similar application, then the option would rate ―high,‖ indicating a strong likelihood for success. If similar applications of that option failed or have not been tested (unknown outcomes), then this option rates ―Low.‖ Effectiveness would also take into consideration the potential of this option to attain the goals of fairness and equity. Cost would be an estimate of the relative total dollar value of funding required to implement this option. Typically, policy options that promote the general welfare or advocate for equal rights do not garner the cost scrutiny that other programs may face (Yukins, 2004; McKean, 2007). However, the current economic climate would indicate that any new proposal would undergo intense scrutiny regarding the costs associated with implementation and sustainment. The rating of political impacts examines such factors as the disruption to the existing governmental structure; the possible controversy that may surround the option; and the current strength or commitment that exists to support or refute the benefits from this option. The analysis assesses these four criteria (effectiveness, feasibility, cost, and political impacts) against each option, towards developing ratings of low, medium, or high.

Effectiveness

The Awareness Training, Central Authority and DOJ Clarification options rated as ―high.‖ The Awareness Training could employ techniques that lead to very positive results when it comes to shaping behaviors. These reshaped behaviors will serve to improve the attention given to testing for accessibility and result in building accessibility into web development efforts.

162 Training programs also serve to build awareness of the legal requirements behind attaining accessibility. Training is a primary tool for effective organizational change (Mather, 2009). There are indications that the federal organizational culture does not fully embrace accessibility as an important goal (Jaeger, 2006; 2004), so this type of training would be instrumental in reshaping the federal culture, initially and over the longer term. The Central Authority option builds upon a successful model (the IRS model) that has a proven record of accomplishment for delivering a high percentage of compliance. This model provides a proven office structure and organizational cultural profile that could serve as a working example. That IRS model would offer an excellent road map for success towards the development of a Web Accessibility Office for the consolidated and integrated administration of web accessibility policy. The DOJ Clarification option rates as ―high.‖ If the DOJ provided an interpretation that clearly explains that the ADA provision for ―public accommodation‖ extends to websites, then federal websites would be required to attain compliance with Section 508. While federal websites must comply as stipulated in Section 508, non-compliance does not carry the potential for civil class action. This positive interpretation by the ADA would provide for redress actions from groups impacted by inaccessible websites. The threat of huge class settlements, combined with the potential for much politicized public embarrassment of governmental offices, would serve to instill a sense of urgency and commitment towards attaining accessible websites. For these reasons, this option rates as being very effective. The option for a Policy Review of Section 508 does not hold too much promise for delivering effective change. The original policy has fallen woefully short of its intent, and the mere rewriting of the policy does little to ensure improved compliance (Jaeger, 2006). That past record of failed accessibility policy would indicate that this option rates ―Low‖ in effectiveness. The Research option does offer more hope, showing potential by attempting to identify the real problems that impede attaining accessibility. This research stream would be the more beneficial of the two paths, and because it offers significant potential, this option rates ―high‖

163 Feasibility

The feasibility of the Central Authority option rates ―Low.‖ The current economic crisis will likely overshadow many important goals and programs that need attention. As this option would only serve to detract resources from the economic crisis, and offers no direct support to solving that crisis, it is unlikely that there will be sufficient political interest or support for this effort. This low rating also reflects the current trend for smaller government (Gallagher, 2003). There has been little serious concern for issue of civil rights and equity issue within the government lately and that trend will likely continue as we continue to face financial upheaval (Economic & Social Data Ranking, 2006). The advocacy and lobbying groups for disability rights are perhaps more organized and better prepared than ever, but their small numbers and declining power base do not favor pursuing this option (Pappin, 2002). The one ray of hope comes from the recent pronouncements by the newly elected Administration, that openly announced support for the ADA and it original intent of assuring fair and equal treatment in the broad sense (DisabilityNow, 2009). This reasoning also applies to the Policy Review and Research Program options, but other factors come into play that would lessen the overwhelming impact of the economy. While there are a few active advocacy groups lobbying for improved web accessibility (DRA, 2009), human equity and civil rights issues are not at the forefront in general and even less so during the current financial turmoil. However, these efforts require a very small commitment of resources. Given an effective champion, these options could implement relatively quickly and with a minimum of organizational stress. The Policy Review and Research Program options rate as ―Medium.‖ The DOJ Clarification option also has complications that may indicate that this plan is not something that would be easy to accomplish. There have been several court rulings that have addressed the issue of ―public accommodation‖ interpretation within the context of the ADA (Hart, 2007). Hart indicates that these rulings have generally not supported extending the ADA concept of public accommodation to websites. However, in Hart‘s review of applicable court cases, he noted that the DOJ did file a brief in which it asserted that websites are actually places of public accommodation under the ADA (Hart, 2007, p. 860). The indication is that the DOJ

164 could possibly take this opinion to the next level and issue a clarifying interpretation that extends public accommodation to all websites. This option rates as ―Medium.‖ The Awareness Training program rates ―High.‖ In the recent decade, the federal government has been paying increasing attention to improving the federal workforce (Grant Thorton LLP, 2009). Many training programs and certification opportunities exist and remain effective across much of the workforce. This momentum in supporting advanced technical and leadership training has had particularly strong roots in the information technology areas. A suitable infrastructure for training has been in existence for at least the last decade and has been very successful in providing top-level information technology training. This Awareness Training option has the potential for easy integration into the existing information technology training venues.

Cost

The Awareness Training, Policy Review, DOJ Clarification, and the Research Program options would all be relatively inexpensive options to pursue within the federal government. The incremental costs for the educational infrastructure would be minimal for the Awareness Training effort. If the training was developed and provided as a web based product, then the costs would even be less than a face-to-face version. As the level of required funding is reasonable, this option rates: ―Low.‖ The Policy Review option would require only a small team of experts that meet as an ad hoc panel over about 2-3 years to coordinate the collection of input and then formulate a proposed policy instrument. The Research option would similarly be a small team but their efforts would be ongoing. The sustained commitment to research would require a long-term funding stream, but the cost of such a small research team would indicate that these two options rate ―Low.‖ The cost of the DOJ clarification would also be minimal. The indication is that the current Administration would strongly endorse this action (DisabilityNow, 2009), which would serve to minimize resistance and thus lowering the cost of this option. While there would be a requirement for political negotiation and alignment of sentiment, the actual monetary costs would not be substantial, so this option rates ―Low‖ in cost.

165 The cost of the Central Authority option rates ―High.‖ This option would require the designation and staffing of a rather large organizational entity to encompass the complete administration of web accessibility policy. This would include hiring sophisticated professional personnel to conduct testing and audit activities. The requisite accessibility testing procedures would require top level technical and managerial experts that are expensive to obtain.

Political Impacts

The Central Authority would result in the biggest political waves and repercussions. It is not difficult to envision political power blocs working against each other to structure this office to serve agendas other that assuring website accessibility. If the office is not established with appropriate levels of independence and authority, then the political forces may abuse its audit and enforcement powers to influence other entities. This struggle could serve to render the office ineffective and mired in political upheaval. Such political wrangling is not uncommon, and evidenced recently when the Administration considered the elimination of the Office of Political Affairs (Politico, 2008). Also, note that the public focus has waned regarding issues of disability rights (Switzer, 2003). This topic is one that has an unpredictable ebb and flow as to political attention levels, and the current environment is one where concern is at the lower end of the scale (Switzer, 2008). The rating for this option is ―High.‖ The DOJ Clarification effort would require extensive political negotiation and involve substantial efforts to align factions in support of this effort. Hart (2007) has indicated that DOJ has already come down on the side of favoring extending ―public accommodation‖ to include websites, but at the same time failed to take the initiative to issue an official and binding interpretation. That reluctance to make a definitive call may speak volumes about the underlying political sensitivity that lies at the foundation of this issue. That perception of a strong political undercurrent surrounding this issue indicates that this option rates ―High‖ for political risk and impacts. The three options of Awareness Training, Policy Review, and Research Program, would all rate as ―Low.‖ These options offer little in the way of political influence or disruption of the current culture that surrounds the political arena. Web accessibility training courses and certification would merely be a small incremental addition to a training facility, requiring

166 perhaps 2 to 3 staff to administer the program. Such a small addition would harbor little potential for political difficulties. The Policy Review and the Research options would be so small in terms of government programs and projects that these options would easily fly under the radar of the political power blocs. These efforts would also have little to offer in the way of political haymaking or offer any type of significant controversy or embarrassment.

Impacts on the Key Stakeholder Groups

Each of the five options offers a unique set of impacts and benefits, along with placing specific requirements on each of the key stakeholder groups. The key stakeholder groups were: the website providers, the federal government, and the website users with disabilities. The discussion now examines each of the five options to assess what impacts will be levied upon each of these key stakeholder groups. For the stakeholder group ―individuals with disabilities‖, there is no significant direct negative impact of these options. The positive indirect impact is that each of these options would serve to provide increased web accessibility, which would serve to make websites more usable to these groups. Each of these options would contribute to removing the barriers that exist which impede their ability to fully use and interact with federal websites. Central authority. The impact on website providers will be to provide them with a consolidated source for all aspects of website accessibility issues. This would include being a focal point for guidance, training, best practices, and other resources. This central authority would promote consistency of policy, establish uniform criteria of evaluation, and attain conformity towards a single set of standards. There would be less possibility of conflicting direction or instructions from various sources. The impact on the Federal Government would be the necessity to establish an office resourced, staffed, and empowered to coordinate and administer all issues related to information policy on web accessibility. The office would need to have the authority and means to test, audit, and enforce the standards for web accessibility. Currently, the Access Board has the beginnings of these responsibilities, but is not empowered or resourced sufficiently to carry out this more encompassing role of managing web accessibility.

167 Awareness training. The impact on the website providers would be access to focused training covering website accessibility issue, policy, and tools. By mandating the accomplishment of this training in a timely manner, the web managers would develop an improved awareness of the legal and ethical reasoning behind the drive to improve web accessibility. Website providers would need to allocate funding to pay for this training and would need to make their staff available to obtain the training. Minimizing funding requirements is accomplishable by making the training available via an online instruction. Section 508 review. There would be minimal impact of a review of Section 508 on Website Providers, until the new recommendations take effect. One possible impact is that during the review, some interim findings may be suitable for application by the Website Providers that would improve the existing levels of accessibility. The impact on the federal government would center on the need to assemble an ad hoc task force to study Section 508 and make recommendations. The recommendations would likely be made to the Access Board and subsequently presented to Congress for the enactment of legislation to revise Section 508. Research program. Website providers would benefit from this research by having guidance that is more available and includes best practices obtained from reviewing the practices and procedures across all state and local entities. This research would also serve to improve federal training in website accessibility by including these best practices in the training required of all website managers during their certification process. The government impacts would focus on the establishment and resourcing of a research office that would conduct these research efforts. Resident staff could conduct the research, or efforts that are more specialized could be contracted using targeted vendors or by using grant programs. Department of Justice clarification. The Department of Justice (DOJ) should act to establish a clearer interpretation of ―public accommodation,‖ so that the application extends to websites. The impact would be that the DOJ would establish definitive legal obligation requiring websites to be accessible. That requirement would also provide for the filing of class action suits by groups impacted by the inaccessibility of websites. The motivation to prevent expensive litigation efforts and the possibility of significant judgments would result in websites that are more accessible to individuals with disabilities.

168 The federal government would need to take action to encourage the DOJ to consider this issue and make a favorable determination. The Administration could lead the charge by issuing an Executive Directive to the DOJ in this matter. Summary. These five options have varying impacts on the web providers and the federal government, but do share one common impact. That common impact is that each option results in providing an environment that is more conducive to attaining full compliance with web accessibility policies. The outcome would be websites that are more accessible to individuals with disabilities.

Conclusion and Policy Recommendations

Each policy option had its own unique set of factors that determined how the evaluative criteria rated their suitability for adoption. Table 4.3 provides a simple visual depiction of the proposed policy options, the evaluative criteria, and the associated assessed ratings. This side- by-side comparison is valuable for making a determination as to which policy option(s) to recommend for implementation. Table 4.3 provides a simple to understand approach to decision making, but lacks the ability to assess quantifiably the relative value of each option, especially if many such options were under consideration.

Table 4.3: Analysis of policy options. Assessment of Policy Analysis Options Policy Options Policy Evaluative Central Awareness Review of Research DOJ Criteria Authority Training Section 508 Program Clarification Effectiveness High High Low Medium High Feasibility Low High Medium Medium Medium Cost High Low Low Low Low Political Impacts High Low Low Low High

169 Table 4.4 offers the decision maker a more readily evident evaluation of the value of each option. In this table, the rating for each option is a quantified value that would correspond to the ―goodness‖ of the rating. As an example, an option rated as having high effectiveness would be a very good choice, and would be assigned a rating of 5. If an option had high political impacts, being perhaps too controversial to garner adequate support, it would not be ―good‖, and assigned a value of 1. Options rated as neutral or medium would score a value of 3.

Table 4.4: Analysis of policy options – Scoring table. Policy Analysis Of Options Example Rating Schema (No Weighting) Policy Options Policy Evaluative Central Awareness Review of Research Criteria Authority Training Section 508 Program DOJ Effectiveness High 5 High 5 Low 1 Medium 3 High 5 Feasibility Low 1 High 5 Medium 3 Medium 3 Medium 3 Cost High 1 Low 5 Low 5 Low 5 Low 5 Political High 1 Low 5 Low 5 Low 5 High 1 Impacts Totals 8 20 14 16 14

Table 4.4 employs this type of schema to evaluate the appropriateness of these five policy options. Note that this evaluative tool does not incorporate the application of weightings, although the model could be adapted to provide that type of analysis. In this application, each of the evaluative criteria was equally weighted. To determine the value of each policy option, the vertical columns with the assigned points sum to indicate a Total Point value for each option. The higher the Total Point score for the option indicates the greater overall appropriateness of that option for improving web accessibility policy. With all evaluative criteria being equally weighted, the indication is that the Awareness Training option would be the best approach to take, followed by the Research Program. The ranking of these policy options, from highly appropriate to least appropriate, would be: Awareness Training, Research Program, DOJ clarification, Policy Review of Section 508, and the Central Authority. The Awareness Training option scored well in each of the evaluative criteria categories, indicating high levels of effectiveness and good feasibility, while having low cost and minor political impacts.

170 The Research Program option had only medium ratings in effectiveness and feasibility, but scored well in the areas of cost and political impacts. The Awareness Training option and the Research Program option scored well overall, and were in the middle to upper range for effectiveness. For that reason, both of these policy options are viable programs for the improvement of federal information policy for web accessibility. It also turns out that these two options, in combination, can provide a strategic value that exceeds the individual value of each option.

Strategic Value, Benefits, and Impacts of the Recommended Options

The Awareness Training option rates as having a high level of effectiveness. This would serve to address the observed low levels of web accessibility as suggested by prior studies (Jaeger, 2006; West, 2004, 2002). By mandating this training and requiring completion in a timely manner, this option would serve to greatly improve accessibility compliance in a relatively short period. The requirements for ongoing training (re-certification, or to maintain certification status) would also provide a sustained and lasting effect. This ongoing training would also serve to keep web managers abreast of new developments in the tools, techniques, and legal requirements that evolve as this technology continues its rapid advance. Another benefit of this training approach is to re-shape the culture of the federal workplace to be more aware and accepting of the requirement for attaining accessible websites. Training is a very effective means for shaping the culture of the organization, and as the culture is re-formed, the impacts will include a long lasting attention to providing accessible websites. The value of the Research Program option lies in its ability to address the accessibility problem in a short and long-term approach. The short-term approach would identify the real problems faced by web managers that impede attaining accessibility. Once actual problems are identified, a better understanding of what changes in policy and resources would have the greatest impact on promoting accessible websites. In addition, this short-term approach could identify a set of best practices, and then make those practices available to all web managers through the Awareness Training courses. In this way, these two options leverage one another to become a more effective combined approach.

171 The longer-term research effort would help ensure accessibility in new web development efforts by creating or identifying web development tools that ―build-in‖ accessibility features during website design, programming, and testing processes. Once discovered by the research effort, these tools and processes could integrate into the Awareness Training programs to disseminate the information rapidly for immediate field application. It is appropriate to address the probability of effecting these changes in policy given the current political environment. The current political landscape is relatively fragmented, where the power and authority has become highly dispersed among the major national political institutions. This high level of fragmentation extends to the internal workings of the Congress. This high level of fragmentation hinders coherent action and obscures responsibility for government action. Such fragmentation will make any efforts to make effective change in web accessibility policy a very challenging matter. The key to success would depend upon the formation of a united and bipartisan bloc in order to implement new information policy. However, the current political landscape is characterized by increased polarization and decreased bipartisanship, making it exceedingly difficult to build the cross-party coalitions that are necessary for positive reforms. The pervasive gridlock jeopardizes the likelihood of making changes that would promote web accessibility. Political environments change, and within that change resides the hope for a coalition of support that will implement the policy recommendations proposed by this study.

Implementation of the Two Policy Recommendations

Ensuring Effective Implementation

McLaughlin (1987) indicated that policy implementation is incredibly difficult to accomplish in government, primarily because policy makers cannot mandate the detailed procedures that really matter. Policy does not speak to the implementers in sufficiently detailed terms to establish a clearly defined and restrictive set of procedures that guarantee compliance with the policy intent. McLaughlin (1987, p. 172) observes that implementers frequently ―do not always do as told,‖ and do not ―act to maximize policy objectives.‖

172 McLaughlin (1987) observed that that a successful policy implementation effort depends critically on two factors: local capacity and will. To promote local capacity, the proposed program office will need to be adequately staffed and funded. Staff would need to possess relevant experience and should receive appropriate training to execute their responsibilities skillfully. The program office must have the power and authority to execute the mission responsibilities necessary to achieve the goals and intent of the new policy. To build the will within a new office, or program effort, requires that the staff receive the training and informative resources that will build an awareness of the intent of the policy and convey a clear understanding of the means to attain that intent. McLaughlin (1987) further suggests that the newly formed staff should initiate efforts that promote the intent and mission of the office to all stakeholders. This collaborative effort with the key stakeholders serves to build the will within the office, and builds an external sense of will. Additional collaborative efforts would build a sense of community and shared purpose with other key stakeholder offices. As these recommended options (Research Program and Awareness Training Program) are merely extensions of the role or mission of an existing agency, Congressional action would be minimal when compared to the formation of a new agency (as the Department of Homeland Security) or the passing of a new legislative act. Presented next is a possible set of general steps that could occur to implement the two recommended options of establishing a Research Program and Awareness Training Program. Following these general steps is a discussion of more specific steps that could serve to improve the implementation of each of these program efforts. Presidential directive. The President could issue a memorandum that directs the establishment of these programs, and describes the expected mission and goals. The memo would also identify the organizational entity that acquires the responsibility. This directive would indicate top level Executive support for building awareness, while leading a culture change within the federal government. Congressional support. Congressional committees, or the overseeing Executive Agencies, may issue strategic and planning guidance to assist in the formation of the new office or program. Such implementing guidance could assist in further defining mission, goals, objectives, and in providing recommendations as to the members of an ad hoc formative committee. The Environmental Protection Agency (EPA) website provides an example of such

173 guidance (EPA, 1993). Congress should act to allocate immediate funding for these programs and determine ongoing allocations (ear-marking). Given the political significance of these new program efforts, Congress may establish an oversight committee to assist and provide high level direction during the formation of the office/program. These programs would likely not warrant such action, based upon their relatively low political impacts. The more likely action would be that the receiving organizational entities would form internal ad hoc steering and implementing committees that would: develop more detailed strategic plans regarding the formation and implementation of these programs; re-allocate resources and funding to staff these new programs; and begin to identify staffing requirements. These committees would likely have the responsibility for recruiting and staffing the initial key leadership positions for these new programs. New program office. Once key positions begin to fill, the new management team will develop more detailed operational and management plans, and continue to fill staffing requirements. The role of this new leadership team would include collaborating with related government agencies and with key stakeholders groups. As the office continues to develop, McLaughlin (1987) indicated that improving policy implementation will increasingly require the managerial focus to shift to the staff members. Successful implementation will require individual incentives, the building of will, and developing strong staff commitment to the policy goals. McLaughlin (1987, p. 174) stated that ―organizations don‘t innovate or implement change, individuals do.‖

Implementation of the Awareness Training Program

The requirement for training all agency leaders and web managers within the federal workforce will be a large undertaking. The Office of Personnel Management (OPM) indicated that the federal civilian workforce is in excess of 3 million individuals, but the number of federal websites or web managers could not be determined from the review of literature (OPM, 2009). An estimate of that number of websites (and thereby the number of web managers) can be derived by making a few conservative estimates. If each major organization has a website and consists of 300 employees, then over 10,000 websites may exit. If each website has a software developer, web project manager, and an IT oversight manager, then the total requirement for

174 training would likely exceed 30,000 individuals, and providing awareness building programs for the agency leadership would significantly increase that total. To accomplish training such a large number of individuals in a timely manner could be addressed by providing the training in an online mode. Online instruction is a very cost effective means of conducting training and is comparable in educational effectiveness to other common training methods (Bartley & Golek, 2004; Schmeckle, 2003). The online method offers the training provider such benefits as the ability reach a large audience, minimal staffing requirements, and offers significant cost savings by minimizing resources as classroom , trainer travel, and printed materials. For the student, the online method provides such benefits as convenient scheduling, modularized training to complete the training material incrementally, and minimized travel related costs. Online delivery of training to professionals also provides significant cost savings to the training provider, resulting in a higher return on the funding invested. Online training is the most effective way of providing training to students located in diverse areas, which is clearly the case with the geographically dispersed federal workforce (Taylor, 2002). Leveraging existing capabilities. The federal government has already made a great deal of progress in developing online training courses in general, and has developed several courses specifically designed to address the topic areas proposed for this training effort (GSA, 2009; Section508, 2009; USPTO, 2009). The indication is that the federal government has existing expertise and infrastructure related to providing online training, and more specifically has online modules already developed and in use that could be adapted for this training effort. Training courses and content. The proposed training could consist of two online courses that would target the agency managerial level (agency leadership) and to the more technical level (webmasters). Both the government and the commercial sectors have used this approach when providing Section 508 accessibility training (Section508, 2009; TecAccess, 2009; USPTO, 2009). Appendix G identifies the proposed course content and course durations, based on the topics found in similar government and commercial training courses. These initial course contents would undergo continuing improvement based upon participant feedback and other assessment efforts, and would adapt to the changing technological and legal environment for web accessibility.

175 Training materiel. This training effort may require the development of instructional materials that are not currently in the federal inventory. However, the government could develop or acquire such materials easily, as information on this subject/topic is widely available. There exists a multitude of commercially available resources and material needed for this proposed training effort. The federal government would have no difficulty in obtaining additional training materials from commercial sources should that need arise (GSA, 2008). Training administration. It has been indicated that numerous website accessibility training efforts have been developed by the federal government (Access Board, 2009; Section508, 2009, USPTO, 2009; GSA, 2008). This somewhat fragmented approach has likely lead to wasteful use of financial and human resources. In an effort to consolidate these efforts for a more efficient use of resources, and to ensure a standardized level of training across all federal agencies, the proposal is to task a single administrative entity to deliver and administer this training program. This entity would be an existing federal educational facility that has the skills and resources to deliver this type of information technology related training. Preferably, this entity would have experience in the delivery of online training and have and existing infrastructure in place to accommodate this requirements. One existing federal agency that meets these requirements for educational expertise, online course delivery experience, and possesses infrastructure to support the delivery of these online courses would be the National Defense University (NDU, 2009) in Washington, DC. While other appropriate federal training sites may exist, this site does offer the benefit of being a centralized entity that could readily absorb this role with a minimum of incremental resources. Training evaluation. As participants complete the training, assessment feedback would be solicited. The requested information could address such areas as the perceived quality, relevance, and appropriateness of the training, as well as soliciting ways to improve the training effort. Periodic follow-ups with both the participants and their managers will help to determine the impact the training has had raising the staff‘s awareness of web accessibility issues and in shaping behaviors to promote compliance with accessibility requirements. Additional assessment methods could further evaluate the training effort to identify areas for improvement. As the training is web based, the training course modules could be analyzed by using server log information. Server log information provides information that would help to identify errors in modules, by identifying where (web page) individuals would exit the website.

176 Participants that begin the training, but subsequently drop out, could identify their reasons for not completing the program. For those that did completed the training, a survey effort could be launched that would provide valuable feedback as to problems experienced by the participants or could solicit suggestions for improving the course. Impact assessment. It would be important to obtain an indication as to the impact of the training program. One way to measure impact would be to have the student identify their website during enrollment (baseline measure). The website could be tested for compliance with Section 508 criteria before the student takes the course and then one year after the course is completed. While individual results may vary, the overall trend should be that the number of errors would decrease. Lower observed rates of errors would indicate a positive impact, resulting in a generally higher level of compliance across federal websites. Inducement and enforcement. Participants would likely view the receiving of the certification (course completion) as a positive career advantage (AdvisorMax, 2007; Marcis & Bland, 2006). The training would enhance their career marketability, and career mobility would be enhanced allowing them to compete for other opportunities. Additional inducement would be a Presidential Directive issued to OMB that would task that agency to require: 1) the identification of all web managers and 2) would establish a mandate for attaining certification within two years of making this training initially available. The mandate would also require biannual re-certification training. Requiring that all subsequent pay increases for web managers be contingent upon the completion of the certification (and re-certification) process would serve to strengthen this mandate. As there is no central management of federal salaries, enforcement would be difficult. The General Accountability Office (GAO) could integrate (append) this certification requirement into their regular audits of federal personnel programs and procedures (GAO, 2009).

Implementation of the Ongoing Research Program.

The responsibility for managing the Research Program aligns closely with the current responsibilities of the Access Board (Access Board, 2009). The Access Board is a federal agency with responsibilities that include developing and maintaining accessibility requirements, and providing technical assistance and research on accessibility guidelines and standards. The

177 Access Board would need to expand in scope, resources, and personnel when formally assigned this responsibility. Formative committee. An initial part of the implementation of this policy effort would be for the Access Board to establish an ad hoc planning committee. This temporary committee would have the responsibility of developing the detailed implementation plans that initially form the office (Research Program) and provide the strategic direction. One of the first duties faced by this committee would be to define overall resourcing and staffing plans related to creating the Research Program office. This committee would have the responsibility for recruiting and interviewing the initial key staff members, and then transition that responsibility as the key positions are filled. The importance of this formative committee lies in the needs to initially provide strong leadership and establish a clear strategic vision. The ―Reengineering through Information Technology (Gore, 1993)‖ report identifies important formative elements that should be present to ensure the successful integration of an information technology change within the federal government:  Strong and effective leadership  Organizational strategic planning  Active interaction with stakeholder groups  Integration with the functional agencies through collaborative efforts Reinventing government is a serious and complex undertaking that depends on leadership. Only good leadership, which combines vision and commitment, can ensure the success of information policy that includes the redesign of federal business practices (Gore, 1993). Initial activities. The senior leadership would have the requirement of filling the key positions according to the staffing plan. Other early-on activities would include formal and informal coordination with the related stakeholder groups. These stakeholder interactions would include soliciting input on the stakeholders‘ concerns, needs, and other issues. Another early-on activity would be the promotion of the office to the federal web community (awareness building activities). This new office will effectively implement a significant change to the businesses processes ingrained in the federal culture. There will be the need to conduct extensive awareness activities to inform the federal community of this new federal entity. These activities will help build a collaborative environment that will result in a more effective set of working relationships.

178 This importance of building these strong working relationships and minimizing cultural resistance are not to be underestimated (Schien, 2004; Gore, 1993). Impact assessment. It would be important to obtain an indication as to the impact of this Research Program. One way to measure impact would be to conduct ongoing assessment of the levels of accessibility compliance. Examining a statistically representative sample of federal websites each year could indicate the overall compliance status. A positive year-to-year trend would support continuing the research program and indicate a successful outcome.

Chapter Summary

The intent of this chapter was to conduct a policy analysis of the federal policy environment on website accessibility. The chapter began by examining key policy problems, issues, and assumptions that are noted concerns. The chapter then conducted a side-by-side comparison of the two legislative acts that most closely relate to website accessibility. This analysis revealed key differences in how these two documents attempt to ensure the accessibility of federal websites. That analysis led to several recommendations that could improve the effectiveness of these two legislative acts. Having examined the key policy documents, the analytic focus turned to an examination of the key weaknesses attributed to attaining website accessibility. By identifying key themes in the discussion of policy problems, the analysis searched for opportunities that could rectify these areas of weakness. The findings from that analytic effort led to the development of five policy options that could significantly improve the effectiveness of the federal information policy regarding website accessibility. Assessing the merits of these options involved the use of several evaluative criteria: effectiveness, feasibility, cost, and political impacts. That evaluation produced a recommendation for pursuing two interrelated options:  Requiring all federal leadership and agency website managers to participate in awareness building and training programs.  Committing to ongoing research efforts to determine the impediments to attaining web accessibility and to identify the best practices that promote attaining web accessibility.

179 These options are interrelated, as the knowledge gained from the ongoing research effort would serve to as feedback loop into the certification training programs. This feedback provides for continuous improvement in the skills, knowledge, and abilities of the federal web managers. These recommended policy options would help the federal government to better comply with the intent of the existing legislation and to ensure fully accessible federal government websites. While the federal government has been progressing toward equal participation in its government for individuals with disabilities for three decades, the federal government must now focus on assuring that all individuals can have fair and equal participation in the new frontier of cyberspace.

Closing Recommendation

The selection of the two recommended options depended greatly upon the criteria used to evaluate the options and the weightings assigned to each of those criteria. In this case, the weightings of the criteria were equal. However, this issue of web accessibility falls into the category of being a human rights issue (Jaeger, 2006, 2004). Political decision-making on issues of civil rights, human equity, and fairness typically do not rest on such simple models. Traditionally these issues are decided primarily on effectiveness and on the expressed desires of the citizenry. While there is not a strong call to arms by the citizenry related to advocacy for web accessibility, this issue deserves the respect and dignity that is required of such human rights issues. To that end, there needs to be an ongoing consideration for the option that proposes the establishment of a Central Authority Office for attaining web accessibility, which would include the administration of the training and policies that are associated with web accessibility. Such an option would serve the short-term and long-term need to effectively design, implement, and administer all the programs that support the attaining of web accessibility policy. As the National Council on Disability (2001, p. 28) has noted, "…the costs of doing nothing may be greater than the costs of any reasonably foreseeable measures."

180 CHAPTER 5

ANALYSIS OF THE WEB CONTENT REVIEW

As a method of research, the content review of documentation can provide a useful description of the information within those documents (Babbie, 2001). It is a valuable tool for discovering trends and patterns in documents (Schutt, 2006). This study used a content review to examine federal health care websites for content typically known as a website accessibility statement. The intent of this study was to examine and compare the website accessibility statements of this selected group of websites in order to understand current practice and usage. Findings from this examination resulted in generating recommendations for best practices, and in raising issues and questions about accessibility statements that further research could address.

Accessibility Statements

Websites are increasingly providing web accessibility statements to identify the level of accessibility the website aims to achieve. Typically, these statements take the form of a short informational section or page of text accessed by use of an active link from every page of the website. A website accessibility statement increases public awareness of web accessibility, while helping the user choose the websites that they will visit (eAccess, 2011). This is only one of several reasons for providing a web accessibility statement. Watson (2009) indicated the web accessibility statement has two primary functions. First, it presents information that describes the level of accessibility for the website. This presentation may include describing the methods used to attain accessibility, or to identify the accessibility features that are present or supported within the website. Analysis of the information contained in the web accessibility statement may identify areas within the website where accessibility goals are not met, and provide alternatives that would help assure the content is somehow made available to individuals with disabilities. Secondly, Watson (2009) indicated that the website accessibility statement can be an announcement attesting to the commitment to provide an accessible website. This announcement could be a declaration of the intent to adhere to the standards present in Section 508, or other similar comprehensive guidelines that attempt to ensure high levels of accessibility.

181 Such a statement would provide a reassurance to the users of the website that accessibility is a critical concern of the website provider. Watson (2009) identified the following key components as being essential to a good website accessibility statement:  An open commitment to providing a website that is fully accessible,  An identification of the formal accessibility guidelines or standards that the website follows,  Any known exceptions or deficiencies in provided accessibility features, and  A means to report accessibility problems. The open commitment should appear in the first paragraph to the website accessibility statement. That commitment should state clearly and concisely the intent to provide a website that is accessible, and the intent to adhere to the available standards and guidelines for accessibility. In indicating the formal guidelines, the accessibility statement should identify the specific standards used to ensure web accessibility. Section 508 criteria, the Web Content Accessibility Guidelines, and W3C technologies are the more commonly used guidelines and standards used by website developers.

Purpose of the Web Accessibility Statement

Providing a website accessibility statement accomplishes several purposes. The web developer can customize the content of the statement to address the specific accessibility standards used on that site and should identify features provided to make the website more accessible. EvenGrounds (2010) identified the most common reasons for providing a web accessibility statement are to express commitment; state the standards of compliance; provide instructions for users of assistive technologies; and to address issues of non-accessibility. The web statement affords the website manager with an opportunity to inform visitors of the organization‘s concern for accessibility. That message is one of intent to deliver a website that is fully accessible by providing key features that support accessibility for those individuals with disabilities. Meeting specific standards for accessibility may be an internally generated goal brought about by an intentional organizational commitment, or it may be a requirement imposed

182 externally in an effort to comply with a state or federal laws. In the latter case, it is likely that the organization adopted a specific set of standards in order to attain conformity to the law. In the former case, the organization may use multiple sources of guidance to assure full accessibility of the website. In either case, the accessibility statement is the appropriate place to identify the standards used to ensure that the website is accessible to individuals with disabilities. The web accessibility statement should be a resource for instructions on using assistive technologies to access the information contained within the website. Accessible websites often implement specialized features that serve the needs of individuals with disabilities. As an example, the web developer may have included a set of scripting codes to assist users that require a screen reader to obtain the content found on the website. In this case, the accessibility statement should provide instructions on how to download these embedded scripts and provide the user with access to any required instructions needed to execute these scripts. This section of text within the accessibility statement should also explain how the scripts will assist the user or enhance the user‘s web browsing experience. There may be instances where the website may not be fully accessible in every case or to every user with disabilities. Such a condition may arise when using an emergent technology for which the appropriate accessibility feature does not yet exist. While every effort should be made to avoid using technologies that lead to inaccessibility, there may be the case where the developer (perhaps by contractual requirement) must interact with specific technologies that are outside of his control. In this instance, the accessibility statement should address this lack of accessibility. However, that alone is insufficient; the accessibility statement should provide instructions that allow the user to exercise an interim solution that affords equivalent access to the content or service offered. Website users do not frequently view accessibility statements, in comparison to the other pages in a website. When a website is well constructed and tested using commonly available accessibility guidance, the need for referring to this accessibility statement is likely to be a rare occurrence. That is not a weakness, but a strength of the web accessible design approach. ―When a site is easy to use, fully accessible, and functionality is obvious, people won‘t feel the need to read it‖ (EvenGrounds, 2010, p. 1).

183 Existence of Web Accessibility Statements

This section presents the analysis related to the web content review effort that examined each website to determine if that website meets the criteria for providing a web accessibility statement. Identified in this section are the key findings, issues, and the associated recommendations resulting from that analysis.

Findings – Existence of Web Accessibility Statements

For the purposes of this research effort, the determination as to whether an accessibility statement exists within a website is contingent upon meeting several criteria. To qualify as an accessibility statement, the content must be a specific page (or designated page section) of the website that is devoted to accessibility content, and contains at least one of these three criteria:  Affirms the website‘s compliance to accessibility standards and legislation,  Provides instructions to users of assistive technology, or  Addresses any inaccessible parts and features of the website. Using these criteria for defining the existence of a web accessibility statement, the study found that within this sample of 20 websites:  Eighteen websites (90 percent) provided a section on the topic of accessibility.  Thirteen websites (65 percent) indicated conformance with formal standards or guidelines for web accessibility.  Nine websites (45 percent) described the accessibility features found on the website.  Six websites (30 percent) addressed areas of the website that were inaccessible.  Fifteen websites (75 percent) provided content that met the criteria for providing an accessibility statement. Overall, there was a good indication that the agency web managers are making serious efforts to provide accessibility statements on their websites. Table 5.1 presents the findings for each websites examined. In conducting this web content review, the accessibility statement was often hard to find. In some cases, the link to the accessibility statement was visually indistinct or otherwise difficult

184 Table 5.1: Findings: Does the website have an accessibility statement? Findings: Does the website have an accessibility statement? Condition 2. Condition 1. Is any one of these three criteria met? Does this Criteria 1: Criteria 3: Is there a Criteria 2: website qualify Does it Addressed Are both Name of the website Provides as having an confirm any conditions website section instructions accessibility compliance inaccessible met? devoted to on the use of statement? with existing parts of the accessibility assistive standards or website content? technology? legislation? addressed? Office of the Surgeon Yes Yes Yes Yes Yes Yes General Social Security Yes Yes Yes No Yes Yes Agency OSHA Yes Yes No No Yes Yes OPM Yes Yes No No Yes Yes OPHS Yes Yes Yes Yes Yes Yes NLM Yes No No No No No Navy No No No No No No Medicine HHS Yes Yes Yes Yes Yes Yes Healthy Yes No No No No No People Health Finder Yes Yes Yes Yes Yes Yes EPA Yes Yes No No Yes Yes Consumer Product Yes Yes No No Yes Yes Safety Army Yes No No No No No Medicine Department of Yes No Yes No Yes Yes Agriculture Air Force Surgeon No No No No No No General Health.gov Yes Yes Yes Yes Yes Yes Women‘s Yes Yes No No Yes Yes health Veterans Yes Yes No No Yes Yes Admin. TriCare Yes No Yes No Yes Yes Healthier Us Yes Yes Yes Yes Yes Yes

to find. In some cases, the location of the link was not readily observable, lacking prominence or separation from the other content.

185 Issues – Existence of Web Accessibility Statements

Using these criteria for defining the existence of a web accessibility statement, the study found that within this group of 20 websites:  Two websites (10 percent) failed to provide a section addressing the topic of accessibility.  Seven websites (35 percent) did not indicate conformance to any formal standards or guidelines for web accessibility.  Eleven websites (55 percent) did not describe any accessibility features of that website.  Twelve websites (60 percent) failed to indicate areas of the website that were inaccessible.  Five websites (25 percent) did not meet the criteria for providing an accessibility statement. With 25 percent of the websites failing to meet the criteria for providing an accessibility statement, there is concern regarding a consistent commitment by federal health care entities to provide fully accessible websites. Section 508 does not require that websites contain an accessibility statement. However, leading authorities on web accessibility have indicated that these statements provide significant value to website user (eAccess, 2011; EvenGrounds, 2010; Lemon, 2005; Tomlinson, 2008). Tomlinson identified three benefits of including a web accessibility statement on a website:  Demonstrate how to use the accessibility features provided,  Detail known barriers, and  Provide a point of contact. The web user would experience an improved browsing experience, if properly informed of these accessibility related features. As an example of an improved browsing experience, the user may require larger text to make the web pages easier to read, or may require instructions on connecting a speech output program. Such information on the use of these features would empower the user to interact with the website in a way that enhances their session experience. Instruction on these features should

186 include clear and concise directions that may include specific references to external sites that provide detailed content. It is important that the website makes users aware of known barriers to web accessibility. The web accessibility statement is the appropriate place to disclose those areas of inaccessible content. Along with the identification of inaccessible areas, the statement should provide specific remedies or alternative ways of obtaining that information. Even when the developer makes concerted efforts to assure that a website is fully accessible, there may be instances where the users are unable to access information, or may be unclear as to the instructions that make certain content accessible. In those instances, the user would need to have a means to receive additional assistance, or to report problems related to accessibility. In these cases, the user would benefit from dedicated accessibility contact information. Tomlinson (2008) indicated that these types of user queries should directly route to the person or team responsible for the accessibility of the website. This direct routing should help ensure the resolution of the problem in an expeditious and effective manner.

Recommendations – Existence of Web Accessibility Statements

Based on the findings of this study, the following recommendations would help ensure that webmasters include effective web accessibility statements. These recommendations include guidance that would help to better define the location or find-ability of the accessibility statement. Recommendation: Each website should provide an accessibility statement. Expert opinion indicates that these statements are valuable resources to improve the usability of a website (eAccess, 2011; EvenGrounds, 2010; Lemon, 2005; Tomlinson, 2008). The author provides more specific recommendations as to content and structure in a subsequent section of this chapter (see Recommendations – Thematic Content). Recommendation: Web developers should provide a visually clear link to the Accessibility statement. Whether a user reads an accessibility statement may be dependent upon where the statement is located within the website. The website accessibility statement should be readily available to the initial visitor of the website, indicating the need to place the link to the statement on the website home page. Another good option is to place the link in the main ―Help‖

187 section of the website. The statement often appears as an active link located at the bottom of each web page. The most common (and recommended) location for finding the accessibility link was to place the link at the bottom of each web page next to the links often provided for privacy, copyright, and disclaimer statements. Wherever it is located, the statement must not be overly technical, preferably using language that is clearly stated and simple to understand. The link to the accessibility page should be available from each page of the website and located consistently across all web pages. If a specific Help page exists within the website, then the accessibility statement link belongs on that page. Accessibility links should be distinct from the normal text content. Making the text larger than normal text size and by ensuring a higher level of contrast between the text and the background colors will help make the link more distinct. Recommendation: Make the link to the accessibility statement available from all web pages, and readily visible in the main content area. The most common (and recommended ) location for finding the accessibility link was at the bottom of each web page next to the links for privacy, copyright, and disclaimer statement. Recommendation: If the website is internally searchable from its own web search utility/application, then the accessibility pages should be readily available upon entry of the term “accessibility.”

Quality of Web Accessibility Statements

This section presents the analysis related to the web content review efforts that examined each website to assess the quality of its web accessibility statement. Identified in this section are the key findings, issues, and the associated recommendations resulting from that analysis.

Findings – Quality of Web Accessibility Statements

The determination of the quality of the website accessibility statement was contingent upon meeting several criteria. The researcher examined the websites using four criteria developed by Lemon (2005), from a study that determined the qualities of an effective accessibility statement.

188 Lemon identified the following four criteria as being important to providing a high quality accessibility statement:  Indicates a commitment to accessibility or to providing an accessible website,  Provides a means of contacting the webmaster to resolve problems,  Describes accessibility features or explains how to use the features provided within the website, and  Provides a statement as to intent to conform to specific accessibility guidelines (e.g., Section 508, W3C/WAI). This study used these four criteria to determine if the federal health care websites in this sample provided quality accessibility statements. The researcher examined each website accessibility statement and assigned quality ratings as follows: statements meeting all four criteria were HIGH quality; statements that meet two or three criteria were MEDIUM quality; and statements meeting only one criterion were LOW quality. Table 5.2 presents the findings for each of the websites examined. Within the 20 websites, only 15 websites (75 percent) met the criteria for having a web accessibility statement. Using these criteria for defining the existence of a web accessibility statement, the study found that within this group of 15 websites:  Fifteen websites (100 percent) expressed a commitment to accessibility.  Twelve websites (80 percent) provided a means to resolve problems, such as the ability to contact the webmaster  Nine websites (60 percent) described the use of accessibility features available with that website.  Thirteen websites (87 percent) indicated a commitment to conform to an acknowledged set of accessibility standards (e.g. Section 508 criteria, WAI guidelines) In assigning quality ratings to these 15 websites, the results were:  Seven websites (47 percent) scored as HIGH quality by meeting all four criteria.  Eight websites (53 percent) scored as MEDIUM quality, by meeting two or three of the criteria.

189  No website (0 percent) scored as LOW quality. Overall, of the websites that met the criteria for providing accessibility statements, the findings indicate that the sampled websites had a good showing in the quality assessment. As Section 508 does not require the provision of an accessibility statement, the provision of this statement is voluntary and at the discretion of the website provider. Also of note, the literature review did not reveal any prior studies that conducted a rigorous or systematic investigation into determining a user centric set of guidelines for the content and structure of an accessibility statement.

Table 5.2: Findings: What is the quality of the accessibility statement? Findings: Determining the quality of the accessibility statement. Quality Rating: Criteria 4: Criteria 2: Criteria 3: Criteria 1. Does it state Does it provide Are 4 criteria met = High Seq. Name of the Is there a conformance a means to accessibility 2-3 criteria met = # website commitment to to resolve features Medium accessibility? accessibility problems? described? 0-1 criteria met = Low standards?

Office of the 1 Surgeon Yes Yes Yes Yes High General Social 2 Security Yes Yes Yes Yes High Agency 3 OSHA Yes Yes No Yes Medium 4 OPM Yes Yes No Yes Medium 5 OPHS Yes Yes Yes Yes High 6 NLM Does not meet the criteria as having an accessibility statement. N/A Navy 7 Does not meet the criteria as having an accessibility statement. N/A Medicine 8 HHS Yes Yes Yes Yes High Healthy 9 Does not meet the criteria as having an accessibility statement. N/A People 10 Health Finder Yes Yes Yes Yes High 11 EPA Yes Yes No Yes Medium Consumer 12 Product Yes Yes No Yes Medium Safety Army 13 Does not meet the criteria as having an accessibility statement. N/A Medicine Department 14 Yes No No Yes Medium of Agriculture Air Force 15 Surgeon Does not meet the criteria as having an accessibility statement. N/A General 16 Health.gov Yes Yes Yes Yes High 17 Women‘s Yes No Yes No Medium Health Veterans 18 Yes Yes No Yes Medium Admin. 19 TriCare No No Yes No Medium 20 Healthier Us Yes Yes Yes Yes High

190 Issues – Quality of Web Accessibility Statements

Within the 20 websites examined, five websites failed to contain a web accessibility statement, as defined by the criteria used in this study. Using these criteria for defining the existence of a web accessibility statement, the study found that within this group of 15 remaining websites:  Three websites (20 percent) did not provide a means to resolve problems, such as the ability to contact the webmaster.  Six websites (40 percent) failed to provide instructions on the use of the accessibility features available on the website.  Two websites (13 percent) did not provide a stated commitment to conform to an acknowledged set of accessibility standards (e.g. Section 508 criteria, WAI guidelines). Overall, there was significant variance in how these websites provide quality related features on their websites. The findings from this review center on three topic areas: 1) accessibility commitment, 2) awareness of Section 508 requirements, and 3) providing content that is usable and easily understood. The following sections address each of these topics. Accessibility commitment. It was encouraging to see that 100 percent of the websites that had a web accessibility statement provided some sort of commitment to assuring accessibility. This expression of a commitment to providing an accessible website indicated a strong awareness of the need for accessible websites, and indicated an understanding of accessibility features. That commitment indicated accessibility awareness, social leadership, and knowledge of the policies and procedures that serve to promote accessibility (WebAim, 2011). Section 508. While the overall commitment to accessibility was strong (93 percent), the identification of adherence to a specific set of guidelines was not nearly so robust. With only 87 percent of the websites identifying which guidelines were used to ensuring accessibility, the indication may be that there is a lack of awareness as to the legal requirements to comply with Section 508 criteria. That lack of awareness may suggest that the web developers need training as to the array of accessibility guidelines that exist, and to the relative strengths and weaknesses of each set of guidelines.

191 From Table 5.3, the thematic content analysis (described later in this chapter) indicated that only 60 percent of the websites indicated compliance with Section 508 guidelines. This seems to be an odd situation in that only 60 percent indicated compliance with Section 508, while 67 percent provided links to content about Section 508. Clearly, there is an awareness of Section 508 policy even though the adherence to those guidelines is not explicitly stated. Accessibility features and instructions. It was surprising that as many as 60 percent of the websites provided content regarding the available accessibility features found on the website. However, no website provided actual instructions on the use of these features. While it is valuable to the website user to have knowledge of what features are present, it would likely be even more valuable to have helpful instructions on how users can access and make use of the features during a web browsing session. Overall, from this part of the content review, two general themes are indicated. First, there is a lack of consistency in the content found within the web accessibility statements. Secondly, there is a significant variation in the levels of quality of these web accessibility statements. These problems suggest two underlying issues: 1) a lack of guidelines for web managers that indicate the right content and structure for web accessibility statements and 2) a lack of awareness on the part of web managers regarding the need to provide high quality accessibility statements on their websites.

Recommendations – Quality of Web Accessibility Statements

These findings point to two recommendations that would address the issues identified:  Establishing a standard structure and content for accessibility statements.  Providing increased awareness for web managers through education. Taking steps to integrate accessibility statement creation into the website development process would increase the quality and occurrence of these policy statements, resulting in an improved user experience. Part of that integration effort would be to ensure that web managers are aware of the existing resources and guidelines that help to ensure high quality accessibility statements. Recommendation: The Access Board should conduct user-based research to determine the user value of a web accessibility statement, along with the appropriate design and structure of these statements. The goal of this research would be a determination as to whether web

192 accessibility statements should become an additional standard that could be appended to the existing standards within Section 508. While the World Wide Web Consortium provides guidelines for producing web accessibility statements (W3C, 2011), these guidelines are not part of the Section 508 technical requirements for accessible websites. As such, when web manages refer to Section 508 for guidance on compliance issues they may overlook other sources of guidance that address the content and structure recommendations for high-quality web accessibility statements. Recommendation: Increase the web manager’s awareness of the need for high quality accessibility statements through promotional activities, education, and training. The above recommendation identifies that web managers could benefit from additional guidance on making websites more accessibility. While that guidance would take the form of a recommended structure (layout) and content, there may be an accompanying need to make the web managers aware of these available guidelines and resources. Awareness building efforts, as training, conferences, and certification programs, are just a few activities that could promote awareness of the resources that are available to web managers related to attaining website accessibility. This recommendation ties into the recommendation made in chapter 4 that addresses the need to provide accessibility training for agency web managers.

Thematic Content of Website Accessibility Statements

This section presents the analysis related to the web content review efforts that performed a thematic content analysis to determine the types of information found within these examined website accessibility statements. Identified in this section are the key findings, issues and the associated recommendations resulting from that analysis.

Findings – Thematic Content of Website Accessibility Statements

From the literature review, the researcher found no prior similar studies that systematically examined web accessibility statements for content and structure. The researcher determined that it would be beneficial to examine the content of the web accessibility statements to look for the types or categories of content provided within these statements. Using a practical

193 thematic analysis framework (Aronson, 1994), the researcher examined the content of the web accessibility statement provided within each website, to identify recurring themes, categories, or types of content. The researcher decided to add this thematic content analysis, in order to inform the findings provided by the other analytic methods within the web content review. This thematic content analysis was an informal effort to understand more completely the issues that derived from the findings of the other analytic methods used in this web content review. As previously indicated, of the 20 websites examined, only 15 websites met the criteria used by this study for providing a web accessibility statement. The researcher subjected each of those 15 websites to a thematic analysis to identify the key categories or types of content found. Table 5.3 indicates the findings from that thematic analysis. That analysis found that within the 15 sites that provided website accessibility statements:  Fourteen websites (93 percent) provided a commitment to assuring accessibility.  Eleven websites (73 percent) asserted compliance with some specific set of accessibility guidelines.  Nine websites (60 percent) indicated compliance with Section 508 standards.  Eleven websites (73 percent) provided additional content specifically relating to Section 508.  Ten websites (67 percent) supplied active links to content about Section 508.  Nine websites (60 percent) identified specific accessibility features provided by the website.  No website (0 percent) provided instructions on the use of features offered within that website.  Fourteen websites (93 percent) identified a point of contact for help in using the website. Twelve websites (80 percent) provided active web links to external resources on accessibility related topics.  Three websites (20 percent) provided a date of last modification for the website. Overall, these findings indicate that these statements provide a wide range of content, covering several key topics related to website accessibility

194 Table 5.3: Types of content found in the web site accessibility statements. Web Content Review Types of content found within website accessibility statements Type of Website (identified by sequence number) Content 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Total Provided a commitment x x x x x x x x x x x x x x 14 to assuring accessibility. Asserted compliance with specific x x x x x x x x x x x 11 accessibility guidelines. Asserted compliance x x x x x x x x x 9 with Section 508. Accessibility features x x x x x x x x x 9 described. Provided instructions on how to 0 use the features. Identified a means to x x x x x x x x x x x x x x 14 obtain help. Provided links to external x x x x x x x x x x x 12 accessibility content. Provided content x x x x x x x x x x x 11 about Section 508. Provided links to content x x x x x x x x x x 10 about Section 508. Provided a date of last x x x 3 modification. Count of types of 8 6 7 5 8 8 8 7 4 5 9 6 3 1 7 content

.

195 The analysis indicated 10 major types of content within in this group of websites. The number of types of content ranged from a low of only one type of information present to eight types of content found in these web accessibility statements (Mean 6.1, Standard deviation 2.2). Looking across the 15 websites that met the criteria for providing a web accessibility statement, there is a wide variability in the types of information presented.

Issues – Thematic Content of Website Accessibility Statements

As previously indicated, of the 20 websites examined, only 15 websites met the criteria used by this study for providing a web accessibility statement. Related to the above findings are the following issues for these 15 sites:  One website (7 percent) failed to provide a commitment to assuring accessibility.  Four websites (27 percent) did not assert compliance with some specific set of accessibility guidelines.  Six websites (40 percent) failed to indicate compliance with Section 508 standards.  Four websites (27 percent) provided no additional information relating to Section 508.  Five websites (33 percent) did not provide active links to content about Section 508.  Six websites (40 percent) did not describe the accessibility features provided by the website.  No website (0 percent) provided instructions on the use of features offered within that website.  One website (7 percent) failed to identify a point of contact for help in using the website.  Three websites (20 percent) did not provide any active web links to external resources on accessibility related topics.  Twelve websites (80 percent) failed to indicate the date of last update for the accessibility page. Overall, these findings indicate that the website accessibility statements inconsistently represent important elements about the accessibility of a website.

196 This web content review revealed some qualitative findings that require addressing, with the intent being to provide recommendations for higher quality accessibility statements. These additional findings are:  The type of information presented in these accessibility statements typically focused on technical versus usability content.  Statement content indicated a great assumption of user knowledge.  Content was often complex or very technical in nature. While these findings are subjective determinations, they are concerns that need consideration when the goal is to provide information in a way that is understandable and easily used by the prospective user of the website. Accessibility information typically focused on technical content. The bulk of the content was of a technical nature or presented from a technical point of view. Accessibility features, when described in technical terms, were often overly scientific or complex for the average website user. This technical focus seemed to overshadow the importance of meeting the real needs of the user, who would more likely have a need for accessibility information that supported or explained the use of the website features. For example, the visually impaired user would need instructions on how to use the text enlargement function. The technical information often only covered a few features, or only discussed features related to a single type of disability- related impairment. The content provided within the web accessibility statement displayed too great an assumption of user knowledge. Several sites presented information that required a user to have a great deal of knowledge about technical aspects of the computer or software used. The terminology used relied heavily on highly technical language that may be unfamiliar to the average user. As an example, the accessibility statement for the www.health.gov portal was checked using the Flesch-Kincaid Grade Level indicator, revealing a reading skill requirement at a grade level of 16 (www.health.gov, 2009). That grade level is far above the recommended reading level of 7 to 8 (Microsoft Corporation, 2011). Content is often too technical. Accessibility statements typically present an overview of the accessibility related features that exist on a website. These statements usually concentrate on the technical aspects of the accessibility features available. However, this information will have every little meaning of little significance to the bulk of the visitors of a website. Technical

197 detail would be appropriate if the intended audience was web developers and designers, but this content would not be appropriate for the public. Accessibility statements should present information that allows the individuals with disabilities to get the most benefit from the feature provided, with information presented in a format that is readily understandable to the user community. The final discussion issue addresses the wide variability of the types of content found within the individual website pages. The number of types of content found within these individual pages varied significantly (Range: 1-8), indicating the lack of a consistent presentation and a wide variability in the types of information presented. This inconsistency from website to website may indicate that the user of the website does not always have the information necessary to ensure full accessibility of the website.

Recommendations – Thematic Content of Website Accessibility Statements

The following are recommendations related to the content and structure of web accessibility statements, based on the analysis and findings from the thematic review of the sampled websites. Recommendation: Web developers should provide information on the use of accessibility features in a way that is task focused. The information on accessibility features should align to the needs of the user. In some cases, the websites presented a listing of accessibility features, or provided descriptions that were all too brief. In some cases, the information lacked clear and concise explanations as to the use of the features. Adequate ―how to‖ information was lacking, resulting in the probable inability of the user to employ the accessibility feature. If the feature is unusable, due to a lack of clear instructions, then the site remains inaccessible. In some cases, the content assumed that the user knew such technical information as the type of browser used. The required level of technical knowledge may be beyond the actual level of technical expertise of the average user. In this example, the site should provide additional information that will help the user determine the browser type in use. Recommendation: Web developers should provide links to externally available accessibility related information. The user may need additional or more detailed information regarding the use of accessibility features. The accessibility statement should provide links to

198 other websites that provide authoritative content and additional resources. Examples of such links could include such websites as those by the W3C or Section508.Gov (W3C, 2010a; Section508.gov, 2011). By providing such external resources, this has the benefit of minimizing the amount of content initially provided and subsequently maintained by the website provider. Recommendation: Within the accessibility statement, the web developer should provide content that is clearly presented and readily understandable. Arrange content by use of section headings, to allow the users to scan quickly the various topics presented on that page. Some recommended guidelines for improving the content include:  Provide content that clearly and fully explains the use of each accessibility feature provided. Additional links to external sources for additional resources and information would be valuable.  Solicit feedback from website users in positive way. Users should receive a welcoming and positive request for feedback. Feedback is an important means of identifying problems. Moreover, the problems identified in this way are usually the ones that may most significantly impede use of the website by those individuals with disabilities.  Present a means to contact the appropriate organizational representative, if additional help is necessary. That representative should be someone who is trusted to correct the action in a timely manner. Multiple means of contact would be preferred as some users may be limited in their ability to effectively communicate due to the nature of their disability.  Include a section that addresses known accessibility problems, or areas within the website that are not fully accessible. Provide information on how the users can overcome these barriers and obtain the information or service.  Avoid overly technical presentations. Use clear and concise terminology that fully describes the accessibility issue and associated solution. Present the information in a ―how to‖ format, focusing on the exact steps that the individuals needs to take in order to resolve the issue. These simple guidelines will help assure that accessibility statements are more user-centric. The intent of the guidelines is to provide the users with information that is readily understandable and easily used in resolving accessibility related concerns.

199 In examining the web content of the sample websites, it was clear that there is a need for common standards for the content and format of the accessibility statement. The recommendation is that the Access Board should develop a formal set of standards and guidelines for the development of accessibility statements. The guidance should then be integrated additional accessibility related criteria, which is formally a part of Section 508 of the Rehabilitation Act. At the minimum, a template could serve to indicate the appropriate content areas, language, and headers for an accessibility statement. A template would provide a consistent presentation to users of the website, allowing important topic sections to be quickly located. By using a common set of terms and phrases, the user would get a quicker and more defined understanding of the information presented. As noted previously, the literature review did not reveal any rigorous or systematic investigation into determining a user centric set of guidelines for the content and structure of an accessibility statement. Such an approach would be a great stride forward toward ensuring that the content and structure most closely meets the needs of the individuals who rely on these statements. As an example of the benefit of a user based approach, this study observed that the accessibility statements often begin by including sections that address conformance to specific set of guidelines. While conformance to guidelines is an important feature, the actual users of the website may have very little interest in the technical guidelines or standards used to develop the website. From that observation, the following recommendations to web developers are made: Recommendation: The identification of guidelines and conformance standards that are used should be located at the end of the accessibility statement, thus avoiding making the user trudge through this information in order to get to content that may actually help them use the features that are provided within the website. A common finding is that accessibility statements often provide a listing of steps taken to ensure that the website‘s content is accessible. Examples of this may include statements similar to (Lemon, 2005):  Style sheets are used for presentation.  Font sizes are specified using relative units of measurement. While these may be valuable features to promote accessibility, the mere identification of these steps is of little value to the users. The website users will be readily able to determine if these

200 features exist on the website, without the accessibility statement telling them. The user would be quickly aware if these features were missing from the website presentation. Recommendation: The description of these features should also be located nearer the bottom of the accessibility statement area. Placing this information near the top of the accessibility statement would only serve to dissuade the individual from continuing to plod through this low value content in order to get the content that is of value to the user that needs relevant instructions of the features available to him and the instructions as to how to use those features. Recommendation: Accessibility statements should have instructions near the top of the page that explain how the user can use the accessibility features provided. As the above recommendation reveals, there are several organizational sections within the overall accessibility statement. These sections should be emphasized by provide bold headings that identify the subject matter content provided within each section.

Summary Conclusion

Accessibility statements are the appropriate place and means to boost the capabilities of the visitors to the website. Many times accessibility statements are excessively technical, and may fail to convey the information that the visitor actually needs in order to use the site. The statements may fail to provide the information that a user with disabilities actually needs. The study indicated that the content of the accessibility statements are not standardized and rarely are they organized in a way that makes it easy to use the information. The recommendations indicate on the need to make accessibility statements a formal requirement within Section 508. The guidance related to that requirement should derive from a rigorous and systematic investigation that takes a user centric approach to a standardized design covering content and structure. To ensure compliance with any such new requirements, there would need to be an associated mechanism for monitoring and enforcing compliance. In this way, this study provides support for the policy recommendations previously discussed in chapter 4.

201 Chapter Summary

The research within this chapter presented a detailed analysis of the use, quality, and contents of the website accessibility statements found in federal health care websites. This examination was a web content review of federal health care websites using a checklist-style data collection worksheet. The research used a checklist approach to establish the presence or absence of a statement; to assess the quality of the information presented; and to identify the types of information found within the website accessibility statement. The research found that there were considerable differences in the presence and quality of the accessibility statements across the studied group of websites. It also found a wide variation in the contents provided by these accessibility statements. The research findings led to the development of recommendations that addressed improved content and structure for a more effective accessibility statement. Also recommended is the need to conduct more user-centered research to determine the optimal design and content of the web accessibility statement. This study raises concerns over the presence and quality of web accessibility statements. When made available on a website, web accessibility statements provide an indication to the user regarding the accessibility of a website and at best provide users with instruction on the use of the accessibility features of that site. In this study, only 18 of the 20 websites (90 percent) in the sample featured a findable accessibility statement, but the statements differed significantly in the types of information that they provided. The content, presentation, and organization of the material within the accessibility statements differed greatly from website to website. Few of the accessibility statements contained specific information about the website‘s accessibility features, although all but one site provided a mechanism for users to notify the webmaster about accessibility problems. As Section 508 does not specifically require the provision of accessibility statements within a website, web managers may often overlook the importance of providing an accessibility statement. Integrating the requirements for web accessibility statements into the provisions of Section 508 may serve to imbed the process of providing web accessibility statements into website development. By providing a user centric accessibility statement as a formal and well defined requirement with the provisions of Section 508 may result in an overall increase in the

202 quality and quantity of these statements within federal health care websites, resulting in an improved user browsing experience.

203 CHAPTER 6

ANALYSIS OF THE AUTOMATED TESTING

This research effort examined federal health care websites to evaluate Section 508 compliance. The study employed automated testing as an unobtrusive approach to determining website accessibility. This approach employs a software program to examine a web page or an entire website to determine accessibility. Automated testing provides quantified measures that indicate compliance errors, using a selected set of accessibility standards (e.g. Section 508 standards). These measures are the counts of errors (violations of Section 508 criteria) and the types of error (the rule violation as categorized within Section 508). This study applied automated testing to a purposive sample of websites to check for Section 508 violations. By looking at this sampling of sites, the findings provided an indication of accessibility compliance across all federal health care websites. The automated testing effort provided insight into the range of accessibility errors found within this group of websites. As such, this testing method allowed the identification of higher/lower performing websites. By finding a broad range of errors, the researcher can compare characteristics of the higher performing group (fewer errors) to the lower performing group (more errors).

Methodology

As detailed in chapter 3, the automated testing method used an inclusive sample taken from the population of websites identified in the www.health.gov portal. The www.health.gov portal site and all federal health care websites linked to that portal underwent automated testing (6 websites). For each website, the purposive sample included the home page and five additional web pages within that site. Because the home page of a website typically serves as the initial gateway to all subordinate content within the website, the testing of the home page provided a meaningful representation of the accessibility for the site. The five additional web pages was a purposive selection of web pages taken from the active links and navigational menus provided on the website‘s home page. In this way, the researcher attempted to capture a sample of those

204 pages that are most used, or that provided content that the website provider considers to be of greatest importance to the intended audience. The determination to include six web pages from each website developed from studies by Guest, Bunce, and Johnson (2006) for determining non-probabilistic sample sizes. That work indicated that basic elements for themes present in as early as six reviewing efforts. The Guest, Bunce, and Johnson (2006) study findings were consistent with other findings that indicated 80% of meta-themes identify within the first five reviews (Brinck, Gergle, & Wood, 2002; Virzi, 1992). With such purposive sampling, it is important to avoid any pretense that such an approach is probabilistic, and hence representative. Such a sample will only provide indications, and cannot support generalizations of the population. While the findings may suggest broader indications for all federal health care websites, the study cannot purport wider generalization beyond this sample population. The researcher‘s choice of using six total pages per website also stemmed from pragmatic concerns regarding the time involved in conducting the testing. The automated testing method yielded quantified data as to the counts and types of accessibility errors found within the tested websites. This method ensured the provision of the data needed to address this study‘s research questions. The automated testing yielded quantified measures that represent the accessibility of the websites examined, thereby contributing directly to each of the three research questions posed in this dissertation. Research Question 1 seeks to determine the common error types and frequency of those errors. From collecting the counts and types of errors, this method supports the identification of the more prevalent errors types (Section 508 violations). By knowing the most frequently occurring errors, the researcher placed increased focus on providing targeted recommendations that would mitigate those more commonly occurring error types. Research Question 2 seeks to gather information about the restrictions, deterring issues, or barriers that may prevent web managers from attaining full accessibility. Automated testing is a code-focused approach that primarily supports finding technical issues that may impede accessibility. The automated testing provided insight into the technical barriers to accessibility that currently exist within federal health care websites. Research Question 3 addresses the changes that could serve to improve website accessibility. These changes could range from additional legal awareness, training, expanded budget authority, or improved software tools that automate the features needed to assure

205 accessible websites. By using automated testing to identify the relative frequency of occurrence of the error types, it is possible to better shape the recommendations in a way that more directly addresses the mitigation of these accessibility problems. By knowing the type and frequency of errors, it may be possible to generate policy-related changes that could improve the effectiveness of the legislation.

Findings

The study examined six web pages from each of six websites, using automated accessibility testing. Table 6.1 shows the number and types of errors detected for each website tested. The key findings from that testing indicated that:  Thirty three percent (2/6) of the websites passed the test (had zero accessibility errors).  Of the sites with errors, the testing found 12 errors.  Of the 12 errors discovered, there were five types of errors (error types directly relate to a corresponding rule category within Section 508).  Of the five error types, the most common error type was a Rule 1.1.2 violation. The Rule 1.2.2 violation indicates that the web page had INPUT elements that did not contain the ALT attribute or use a LABEL. The indication resulting from the findings is that significant accessibility problems may exist across all federal health care website. This section discusses the above findings from an overview perspective. Then the narrative more narrowly focuses on those findings and recommendations that specifically address each of the websites examined by the automated testing method. The key indication is that these government health care websites have poor accessibility, as only 33% of the websites passed the test (had no errors). The counter point is that 67% of the federal health care organizations failed to provide fully accessible websites. This finding suggests that a significant number of federal health care websites may have accessibility problems. The mere passing of Section 508 has not resulted in a high level of compliance.

206 Table 6.1: Identification of the errors found within the websites tested. Listing of Errors by Website

Name of the Errors Section Passed/ Description of error website found 508 Rule Failed Rule: 1.1.6 - All IFRAME elements are 1194.22 (a) required to contain element content. No 1 www.health.gov Failed 1 IFRAME elements found in document body.

2 Healthfinder.gov Passed No errors

Rule: 1.1.2 - All INPUT elements are Healthy People required to contain the alt attribute or use 3 Failed 6 1194.22 (a) a LABEL. Failure - INPUT Element, of Type TEXT, found. Rule: 1.1.9 - When EMBED Elements are used, the NOEMBED element is required in the document. Failure - EMBED 2 1194.22 (a) Element(s) are found in document and the NOEMBED element is not.

4 Healthcare.gov Failed Rule: 6.3.7 - When EMBED Elements are found in document, the NOEMBED element is required in the document. 1 1194.22(m) Failure - EMBED Element(s) are found in document and the NOEMBED element is not. Rule: 1.1.1 - All IMG elements are 5 required to contain either the alt or the Womenshealth.gov Failed 2 1194.22 (a) longdesc attribute. Failure - IMG Element found.

6 MedlinePlus® Passed No errors

Four of the six websites had errors (67%), with 12 errors being detected encompassing five error types. Of the sites with errors, the average number of errors was three errors per website (12 errors detected / 4 websites found to have errors) or 0.5 errors per page tested (12 errors detected / 24 web pages examined). As these errors violated five categories of Section 508 criteria, the indication is that federal health care websites suffer from a range of accessibility issues.

207 Healthfinder.gov and MedlinePlus

The Office of Disease Prevention and Health Promotion and its health information referral service, the National Health Information Center, coordinate the Healthfinder.gov website. This website offers information and tools to help individuals stay healthy. The intent is to provide a high quality resource that provides the latest, most reliable health related information. Healthfinder.gov provides resources on a wide range of health topics selected from over 1,600 government and non-profit organizations. MedlinePlus is a website provided by the National Institutes of Health and produced by the National Library of Medicine. The site offers information about diseases, conditions, and wellness issues in clearly understandable language. Medline Plus‘s goal is to provide an online resource that offers reliable, up-to-date health information, anytime, anywhere, and for free. The content includes the latest treatments, information on a drug or supplement, medical definitions, medical videos, and illustrations. The site provides links to the latest medical research on clinical trials for specific diseases or conditions. Finding: The automated testing indicated that the Healthfinder.gov and MedlinePlus websites had no accessibility errors. www.health.gov

This website is coordinated by the Office of Disease Prevention and Health Promotion, Office of the Assistant Secretary for Health, Office of the Secretary, U.S. Department of Health and Human Services. The site provides several links to resources related to general health, covering a variety of topics. The site serves primarily as a portal site, providing multiple links to other government health websites. The site has the intent of promoting a healthy nation by improving health communication and helping individuals utilize health technologies. To achieve that intent the Office of Disease Prevention and Health Promotion has assembled key tools, research and reports, and resources for public health and health communication professionals. Finding: Testing of the www.health.gov website test revealed one accessibility error, indicating that an image within the page did not appropriately display explanatory text when the user hovers over an image.

208 Issue: An example of this error condition is the use of an image to indicate a functionality option within a navigation bar. When the user hovers over this image, a text description should display informing the user as to the functionality provided by this image. In this situation, the descriptive information related to the image is not accessible to visually impaired and blind users, and to users of non-graphically enabled browsers. The impact is that the user would have no way of knowing that this navigational capability exists. This error condition corresponds to Rule 1194.22(a) of Section 508 that states ―a text equivalent for every non text element shall be provided.‖ Recommendation: A relatively simple fix is required, where the web developer would edit the source code for this web page. The web developer would locate the source code related to that image element and then amend that section of code to add a text description of the functionality provided by that specific image.

Healthy People

The Healthy People organization identifies and measures science-based national objectives for improving the health of all Americans. The organization establishes benchmarks and monitors progress over time in order to encourage collaborations across health sectors; provides information that assists individuals toward making informed health decisions; and measures the impact of prevention activities. In support of that mission, this website provides data and tools to enable individuals across the country to achieve a healthier lifestyle. Finding: The automated testing indicated that the Healthy People website had six errors. The errors were all of the same type, where a non-text element within the tested page did not appropriately display explanatory text or provide a descriptive label when the curser hovers over designated area for user input. Issue: As an example, this type of error could occur when a search block (a user input area) lacks a text description that alerts the user to the functionality offered by this input area. This error in accessibility would occur when the user hovers over this specific user input area on the web page, and does not receive explanatory text describing the functionality. The impact on the user is that without the descriptive text the user would be unaware of the availability of the

209 search block. This error condition corresponds to Rule 1194.22(a) of Section 508, which states ―a text equivalent for every non text element shall be provided.‖ Recommendation: The solution would be for the developer to add a title attribute as "Search our website." This descriptive text would make the search box accessible. To correct these errors in the Healthy People website, the web developer would locate the source code related to that non-text element, and then amend that section of code to add a text description of the functionality provided by that specific user input area.

Healthcare.gov

Healthcare.gov is a federal government website managed by the U.S. Department of Health and Human Services. This website provides the public with a "one-stop shopping" place for health insurance. The intent of the website is to provide unprecedented transparency into the health care marketplace. The website helps people sort though both public and private health insurance options by asking them a few basic questions in a multiple-choice format. In addition to the focus on evaluating options in health care coverage, the site provides information and resources related to practicing preventative health care, understanding the Affordable Care Act, and assessing the quality of health care. Finding: The automated testing indicated that the Healthcare.gov website had three errors. Two of the three errors were of the type described as the failure to provide a text equivalent for every non-text element. Discussion of the remaining error is in the finding section that immediately follows. Issue: This type of error occurs within web pages that load plug-ins using the EMBED coding. Web developers should surround any plug-in with a set of EMBED tags, which afford the developer the capability of providing additional text narrative that describes the content or functionality offered by the plug-in. The impact on the user is that without the descriptive text the user would be unaware of the content or functionality provided by the plug- in. This error condition corresponds to Rule 1.1.9 of Section 1194.22(a) of Section 508, which requires ―a text equivalent for every non text element shall be provided.‖ Recommendation: To correct this error, the web developer would edit the source code for this web page. A suitable solution would be for the developer to place the inserted plug-in

210 between two EMBED tags within the source code. To correct these errors in the Healthcare.gov website, the web developer would amend that section of code to add a text description of the functionality provided by that specific plug-in. Finding: The automated testing indicated that the Healthcare.gov website had an error described as: When a page requires that an applet, plug-in, or other application is present on the client system to interpret page content, the page must provide a link to a plug-in or applet that provides the corresponding accessibility. Issue: This specific type of error relates to the use of EMBED elements within the source code of the web page. EMBED elements are typically used to insert (embed) multimedia content. The use of EMBED elements is one of the most reliable ways for web managers to to insert a video into a web page. To ensure compliance with Section 508, the inserted video would be placed between two EMBED tags within the source code. While the video may display properly with only one EMBED tag, the use of only one tag will not ensure accessible functionality with assistive devices. The impact on the user is that without both EMBED tags used, the user may be unable to access the content of the video. By using both EMBED tags, web developers allow the media to open in the stand-alone media player where the controls are much more accessible using a screen reader or keyboard. This error condition corresponds to Rule 6.3.7 of Section 1194.22(m) of Section 508, indicating ―a text equivalent for every non-text element shall be provided.‖ Recommendation: To correct this error, the web developer would edit the source code for this web page. A suitable solution would be for the developer to place the inserted video between two EMBED tags within the source code.

Womenshealth.gov

The Office on Women's Health in the Office of the Assistant Secretary for Health at the U.S. Department of Health and Human Services (HHS) owns and maintains this website. The mission of the Office on Women‘s Health is to improve the health and sense of well-being of all U.S. women and girls. This organization leads and coordinates the efforts of all the HHS agencies and offices involved in women's health. The website presents content related to the agency‘s programs that focus on the health of women, along with providing resources that

211 educate physicians and other health professionals. The website serves to educate and motivate the public to live healthier lives by giving them trustworthy and accurate health information. Finding: The website testing revealed two errors of the same type. The error type is that an image within the page did not appropriately display explanatory text when the user hovers over that image. Issue: An example of this error condition could be an image used to convey instructions or to provide an example of something discussed within the text of the web page. When the user hovers over this image, a text description should display. The display would inform the user as to the content and meaning provided by this image. The text description is needed by visually impaired users who rely on assistive devices (e.g. screen readers) to provide audio descriptions of the image content. The impact on the user is that the user would have no way of knowing what information this image provides. This error condition corresponds to Rule 1.1.1 of Section 1194.22(a) of Section 508, which states ―a text equivalent for every non text element shall be provided.‖ Recommendation: A relatively simple fix is required, where the web developer would edit the source code for this web page. The web developer would locate the source code related to that image element, and then amend that section of code to add a text description of the content, meaning, or functionality provided by that specific image.

Relationship between Accessibility Statement and Error Counts

In this chapter, the researcher examined a set of six websites to identify accessibility errors. In chapter 5, the researcher indicated an approach for evaluating websites for the existence and quality of an accessibility statement. The opportunity exists to compare the results from both analytic methods to see if there is any indication of a relationship between the quality of the accessibility statement and the number of errors found on these websites. The automated testing effort provided insight into the range of accessibility errors found within this group of websites. As such, this testing method allowed the identification of higher/lower performing websites, based on the total number of errors that existed on the six web pages examined for each website. In looking at the accessibility statement assessment for these six websites, the researcher compared characteristics of the higher performing group (fewer

212 errors) to the lower performing group (more errors). The intent is to see if a relationship exists between the quality of the website accessibility statement and the number of accessibility errors found on a website. Table 6.2 summaries the key data elements needed for this comparison effort.

Table 6.2: Relationship between accessibility statement and error counts.

Relationship between Accessibility Statement and Error Counts

Did the What was the Relative Number of website have quality of the Name of the accessibility errors an accessibility website based on found. accessibility statement? error count. statement?

MedlinePlus® 0 High No N/A

Healthfinder.gov 0 High Yes Medium

2 Medium Yes Medium Womenshealth.gov

www.health.gov 2 Medium Yes Medium

Healthcare.gov 3 Low Yes Medium

Healthy People 6 Low Yes High

The two websites with lowest error counts (high accessibility) were the MedlinePlus and HealthFinder.gov. Both sites had no errors. The Medline Plus site did not provide sufficient accessible content that would qualify as providing an accessibility statement based on the criteria established in chapter 5. The Healthfinder.gov website did meet the criteria for providing an

213 accessibility statement, but that statement was only assessed a medium rating based on the content found. The expectation was that those websites with fewer errors might provide higher quality accessibility statements. For this group of two websites, the findings do not support this relationship. The two websites with highest error counts (low accessibility) were the Healthcare.gov and HealthyPeople.gov. These sites had three and six errors, respectively. The Healthcare.gov site provided sufficient accessible content to qualify as providing an accessibility statement; however, that statement only rated a medium assessment. The HealthyPeople.gov website met the criteria for providing an accessibility statement, and that statement received a high rating based on the content found. The expectation was that those websites with a greater number of errors may provide lower quality accessibility statements. For this group of two websites, the findings do not support this relationship. Now the examination will focus on the entire collection of six websites, and consider the expectation that websites with fewer errors would have higher quality accessibility statements. In looking a Table 6.2 from top to bottom, as the number of errors increases the quality of the accessibility statement actually seems to decline. The quality of the accessibility statement ranges from no statement for MedlinePlus (a site with no accessibility errors) to a high quality statement by HelathyPeople.gov (a site with 6 accessibility errors). From this very limited sample, the relationship between error counts and the quality of the accessibility statement is actually the inverse of what might be expected. Due to the very small sample size, the researcher strongly recommends additional study of this phenomenon, using sample sizes that would produce statistically significant results.

Discussion

Supporting their efforts to create accessible websites, web developers have a wide variety of helpful resources at their disposal. Many of these resources are readily available online and are free to use. These resources include technical documentation and software tools, such as the one used in this automated evaluation of compliance with the requirements of Section 508. Despite the availability of these resources, many studies have found that websites remain

214 inaccessible, with some recent studies indicating that websites may even become less accessible over time (Hackett, Parmanto, & Zeng, 2005; Lazar & Greenidge, 2006 ). This study has found that a significant number of the tested website had accessibility problems. As a group, the percentage of websites that failed to be fully accessible was quite high. The indication is that many federal health care websites may be inaccessible. This significant number of inaccessible websites indicates that the legal requirement for accessibility as mandated by Section 508 of the Rehabilitation Act has not been met. In order to improve the accessibility of federal websites overall, it is important to identify the accessibility problems that occur most often and determine which accessibility errors may have the greatest affect on the user. This method of automated testing helped determine which accessibility errors occurred most frequently. By analyzing the nature of those errors, the study provided meaningful insight as to how these errors may affect the user of the website. Websites are rarely static in nature. More typically, they are dynamic creations, ever changing and often undergoing major restructure. Increasingly common are websites that update in real time, immediately reflecting breaking news or events of interest to the user community. To mitigate the effects of periodic change, the data collection for this method spanned a relatively short nine day period. As such, the analysis reflects a snapshot in time. If accessibility tests occurred on an ongoing basis, then important information would be available from periodic examinations. Such longitudinal studies could reveal trends and support projections that would help identify possible accessibility issues not revealed by single testing efforts. In general, automated tools used to conduct accessibility testing may underestimate or overestimate the number of errors on a given web page (Brajnik, 2006). This study used the software product called CynthiaSays (HiSoftware, 2010). This product has become the de facto standard for conducting automated accessibility testing, when the intent is to use a proven product that is freely available in as an online application. Influencing this choice of testing product was the decision of the American Council of the Blind to endorse CynthiaSays as an excellent resource for testing website accessibility (HiSoftware, 2010). Using the most popular testing software package available supports comparisons of the findings of this study to those attained by other studies that used the same testing product. A study by Kane, Shulman, Shockley, and Ladner (2007) indicated a high correlation among the most commonly used

215 accessibility testing products, and indicated that the use of multiple products was unnecessary to support higher reliability. A significant limitation of the automated testing approach is that it does not necessarily indicate the severity of accessibility problems on individual web pages. However, the number of accessibility errors on an individual web page provides some understanding of the affect that an individual with disability may have, when trying to comprehend the information presented or in using the functionality provided by the web page. A significant limitation related to using an automated testing product to examine the accessibility of web pages is that these automated testing products do not capture all accessibility errors. This limitation results in the study findings and conclusions focusing solely on those problems that are measureable by automated testing. While the use of automated testing facilitates looking at a significant number of pages in a easy and timely manner, the approach inappropriately places its attention on addressing what may be only a portion of all the accessibility errors that exist on a tested web page or website. This focus may overlook many significant accessibility problems. The implication is that while automated testing has its place in a comprehensive evaluation of a website‘s accessibility, it has significant limitations that the researcher must address in order to get a complete and accurate assessment. Automated testing also fails to perform even the simplest of logical analysis. As was indicated in the findings of this study, many of the accessibility error related to missing text descriptions. However, there may be cases where text description existed, but those descriptions lacked relevance or accuracy. In such a case, the text description would be just as inaccessible as if it had provided no description at all. In another similar case, the alternative text may not provide equivalent information. As an example, a website may provide an image of the corporate logo. That image may contain the organization‘s logo design, acronym, and the full name of the organization. This image may be the only place on the web page where the full name of the organization exists. If the alternative text only identifies the image as ―logo‖ or as ―IBM,‖ then the visually impaired user would not be aware of the organization‘s complete identification as ―International Business Machines.‖ Cases as these, point out the weaknesses of the automated testing approach. Automated testing can be an effective means to identify accessibility problems quickly and easily, but it should not be the sole means of testing a website for accessibility. The web manager should

216 manually check the site for accessibility problems that are beyond the scope of the automated testing approach.

Context for these Findings

Internationally, studies in the United Kingdom (Disability Rights Commission, 2004) and the European Union (Marincu & McMullin, 2004) have revealed that about 80 per cent of the websites studied had significant accessibility problems. Within the United States, accessibility compliance has also had a poor showing, indicating that 77 percent of the websites had major problems (West, 2006). Jaeger (2006) examined 10 United States electronic government websites and found that every site had major accessibility problems. From the findings of this research effort, 67 percent of the websites examined had significant accessibility problems. Table 6.3 provides a context towards gaining a better understanding as to the accessibility findings from similar studies regarding website accessibility. Such comparability is not ideal as study methods and target populations differ significantly; however, there is the indication that the findings of this research are not significantly out of line when compared to other website accessibility studies. While this study found low accessibility for this group of websites, the study findings are comparable to other similar studies of website accessibility.

Table 6.3: How does this study compare? Comparative Findings of Website Accessibility This study of Electronic US US government Worldwide USA government health care websites websites Percentage of websites 80% 77% 100% 67% NOT accessible

A review of the findings of this study indicates that the web manager could easily fix the accessibility errors detected by the automated testing. The findings indicated that the most common problem was a lack of text descriptions. If these text descriptions were provided, then 5 of the 6 sites would have no errors, resulting in only 18 per cent of the sites (one site) having a

217 major accessibility problem. In other words, this type of minor change would result in a dramatic improvement in website accessibility. This analysis supports the findings of a prior work that indicated that websites that are well designed with accessibility in mind are no more expensive or difficult to create and maintain than poorly designed ones (Lilly & Van Fleet, 2000).

Relationship of Findings to Research Questions

The intent of this section is to convey how well this relatively simple method can address the very fundamental questions posed by this study. The findings from the automated testing approach contribute significantly to each of the research questions posed by this study. Table 6.4 identifies the key findings as they relate to the following research questions:  RQ1. Are federal health care websites compliant with Section 508 requirements?  RQ2. What are the issues and barriers to attaining accessibility?  RQ3. What changes would promote websites that are more accessible? For each research question, a discussion of the implications of the findings follows. The discussion addresses the bearing that the findings have on the four key stakeholder areas: policy, web managers, organizational entities, and the users with a disability.

RQ1. Are Federal Health Care Websites Compliant with Section 508 Requirements?

The legislative goal of Section 508 was to ensure that federal agency websites were fully accessible to all users. As only one third of the sites tested were compliant, the policy-related indication is that the intent of Section 508 has not been fulfilled. The objective of that legislation was to promote the accessibility of websites, and the indication is that federal health care websites remain to a large degree inaccessible. The affect on the user with a disability is that these problems result in websites that are significantly inaccessible, effectively denying these individuals fair and equal access the federal health care information and resources. The indication is that strengthening of the policy environment is necessary in order to make Section 508 compliance a reality.

218 Table 6.4: Relationship of findings to research questions. Relationship of Findings to Research Questions Research Key Findings Questions RQ1.  Only one third of the sites tested were compliant. Are federal health care  Websites had significant accessibility problems. websites compliant with  Testing indicated a variety of error types. Section 508 requirements?  Most errors would be easy to fix.  Most errors related to missing alternative text. RQ2.  Implication of low organizational importance for What are the issues and accessibility barriers to attaining accessibility?  Web developers do not have access to or fail to use software tools that prompt for accessibility features.  Automated testing was effective organizational tool in finding errors. RQ3.  Web developers should use software development What changes would promote tools that prompt for accessibility features websites that are more  Organizations need to implement a more accessible? comprehensive accessibility testing programs  Section 508 needs strengthening to ensure fully accessible websites

RQ2. What are the Issues and Barriers to Attaining Accessibility?

The automated testing approach examined six web pages from each website, and found a significant number of errors, and a variety of errors, during the examination of those few web pages. The indication is that the lack of compliance with Section 508 criteria stems from a variety of problems that impede web managers from providing fully accessible web pages. Most of the errors related to the instance where alternative (descriptive) text was missing from the web source code. Further examination of those problems revealed that the most of the issues would be easy to fix. The indication is that any policy related effort to improve web accessibility should have a major focus on the web manager and the organizational entity. The simple automated testing employed by this study was a fast and effective way to identify key accessibility problems. As this testing method is readily available on the Internet and is free to use, the indication is that web managers have not adopted this testing method into their ongoing web development and testing practices. This deficiency could be a common problem among many web managers and/

219 or could be a common organizational level weakness, where agencies have not adopted and documented detailed and comprehensive practices that assure a thorough and ongoing testing program that promotes accessible websites. The implication is that within the organization there may be a low organizational importance given to assuring accessible websites.

RQ3. What Changes would Promote Websites that are more Accessible?

As fully two thirds of the websites tested were not fully accessible, the current policy environment that attempts to promote accessible website is insufficient to attain that goal. The policy environment must be stronger in order to make Section 508 compliance a reality. Based on the types of errors discovered in this study, changes in the policy environment should focus on the web manager and the organizational entity. The web manager would benefit by having development tools that prompt for the addition of accessibility features during source code creation. Such tools are available in the marketplace, but it is unclear as to the degree to which web managers are using these tools, or are keeping these tools up to date. Additional research would clarify the proliferation, use, and benefit that these tools can provide. At the organizational level, federal agencies must take steps to institutionalize comprehensive and ongoing testing procedures that promote accessible websites. By adopting and documenting testing programs, the agency takes a significant step toward assuring that a detailed and available knowledge base exists that addresses web accessibility. This knowledge base would be an invaluable resource for existing web managers and would provide a permanent resource for subsequent managers due to normal workforce turnover.

Benefits

Automated testing is just one method used to evaluate the accessibility of websites. This quick and easy to use method can identify where problems exist and may indicate improvements in practices and guidelines that would help to ensure accessible websites. The insight provided by this method can shape the creation or modification of web development tools. By knowing the more common types of accessibility errors, designers of web development software could

220 modify these software products to include prompts that remind the web developer to address accessibility issues. In this way, these products would serve to help ensure web accessibility. As an example, this study found that the most common accessibility error was the lack of descriptive alternative text. To address that problem, the developers of commercial software products for web designing could modify these products so that when the software user inserts an image file into a new web page, the software product could provide a prompt that reminds the developer to include appropriate alternative text. By having the developer reminded of the need for alternative text, two objectives are accomplished. First, the added text would serve to make the web page more accessible by providing an informative description about the image. Second, the ongoing prompting would be an effective means of training the web developer to think continually about the accessibility of his websites. This persistent reminder could be a small but significant way to build an awareness of accessibility throughout the community of website developers. This improved awareness could help shape behaviors that would result in websites that are more accessible. Automated testing is a relatively simple method that supports the discovery of accessibility problems and the identification of where these problems occur within a web page. These are two important pieces of knowledge that could be instrumental in reshaping how web page design software prompts for accessibility-related corrective actions. By making the web page authoring products more ―accessibility friendly‖, web page designers would be better able to create more accessible websites.

Chapter Summary

In addition to the graying of America, the number of individuals with disabilities is growing. These individuals require full access of all government services that are available via the Internet. Moreover, the government is providing services increasingly in an online mode and in several cases, the Internet is becoming the primary means of acquiring these services. Individuals with disabilities must have equal access to all the government has to offer, to include access to the health care information and services provided online. The importance of this study lies in its intent to assure that federal health care websites are accessible to everyone, in a fair and equitable manner. The criteria used to determine

221 accessibility are the standards defined in Section 508 of the Rehabilitation Act. The method used in this study is an automated testing of six federal health care websites. Measurements from this study indicated the number and types of accessibility errors found in these websites. The analysis revealed that the federal health care websites had low accessibility, with only one-third of those tested passing the automated testing. That is, only two sites had no errors when examined by the automated testing product. The accessibility errors were diverse in specific type, but generally shared a common theme of failing to provide alternate text to describe content or functionality provided within the web pages. The findings suggest that a significant number of federal health care websites have accessibility problems, and that these problems vary in nature (multiple types). Recommendations are provided that specifically address each type of accessibility error. The analysis of the error types indicated that many of the errors related to the lack of descriptive text to inform the user of the nature or content of images or functional areas that the website provides. As Section 508 mandates accessibility of federal websites, the performance of these federal organizations is of concern. The implication for website developers is that the testing of accessibility should be an initial and recurring part of website management. Additional efforts are needed that will motivate website developers to improve the accessibility of these websites. This study demonstrates that automated accessibility testing can be a cost effective approach that federal entities can use to increase their website‘s accessibility. Federal agencies should employ automated testing as an important part of an overall testing program to help assure their websites meet the criteria established in Section 508. The findings of this study indicate the need for agencies to incorporate accessibility testing into their current design, development, and web management processes. The institutionalization of accessibility testing as a fundamental process will serve to ensure ongoing compliance with the standards that promote accessibility and ultimately result in websites that are more accessible. By using automated testing, this examination of federal health care websites indicates the need for web managers to be more attentive in their efforts to comply with the criteria mandated by Section 508. This study has shown how automated testing can be effective as a quick and easy way to identify basic issues of non-compliance. The analysis of the errors found in this study indicates that the majority of errors are readily correctable with a minimum of efforts or

222 resources. The indication is that automated testing can be a valuable tool toward ensuring the accessibility of these websites.

223 CHAPTER 7

ANALYSIS OF THE EXPERT REVIEW

This chapter provides a discussion of the testing of selected federal health care websites by use of the expert review approach. An expert review (also called expert testing) refers to having a trained authority on website accessibility review the website for errors. The expert researcher employs a checklist designed and tested to evaluate the accessibility of websites (Jaeger, 2006). The major benefit of using expert testing lies in its integration of a human element. This human element (or human centric focus) allows the identification of problems, but also supports an improved understanding of the identification and examination of the underlying causes of the problems. In this way, the findings from the expert review help to shape the associated recommendations by providing insight into the severity of those problems. From this improved understanding, the analysis provided an indication as to which accessibility problems present the biggest deterrent to users with disabilities. The goal of the expert review was to provide an indication of the degree to which these sites comply with the standards established in Section 508. This testing provided a more user based and comprehensive assessment when compared to the automated testing approach used in chapter 6. In comparison to automated testing, the expert testing employs a human reviewer who is able to identify a broader range of potential accessibility problems. The human expert is more qualified to interpret the severity or impact of the problems for those users with disabilities. While the expert review will likely not uncover all possible accessibility errors, it does provide a more thorough examination that better describes the depth and breadth of accessibility problems on a website (Jaeger, 2006; Lazar & Greenidge, 2006).

Findings

The expert review of the websites focused on finding accessibility issues that related to the 10 questions posed in the data collection worksheet, presented in Table 7.1. Those 10 questions address the 16 criteria for accessibility as defined in Section 508 of the Rehabilitation

224 Table 7.1: Expert testing data collection instrument. Data Collection Instrument - Expert Testing Site URL: Score Category (0 = No, Criteria Grouping Number 1 = Somewhat, 2 = Yes) 1 Provides an equivalent for elements related to content and 0 1 2 services?  Provides equivalent alternatives to auditory content?  Provides equivalent alternatives to visual content?  Alternative formats of elements of multimedia presentations synchronize to the appropriate parts of the presentation? 2 Does not rely on color alone? 0 1 2  All information conveyed through color also conveyed without color? 3 Content clear and organized so as to be readable to any 0 1 2 user? 4 Provides context and orientation information at all times? 0 1 2 5 Provides clear navigation mechanisms? 0 1 2 6 Has tables and other formatting that transform gracefully? 0 1 2  Identifies row and column headers on tables?  Title frames with text identifies frame and facilitates navigation? 7 Does not rely on moving pictures or flash to convey 0 1 2 content?  Designs pages to avoid flicker rates above 2 Hz or below 55 Hz? 8 Works comprehensively with assistive technologies? 0 1 2  All electronic forms allow users with assistive technologies to access the information, field elements, and functionality required for completion and submission of the forms, including directions and cues?  Text-only equivalent page available for every page that cannot otherwise be made completely compliant with all other requirements? 9 Ensures user control of time-sensitive content changes? 0 1 2  Users not timed out of applications?  Users given an alert message and the option to indicate that more time is necessary? 10 Ensures direct accessibility of embedded user interfaces? 0 1 2 TOTAL SCORE:

225 Act of 1973. The intent of this expert review was to determine the accessibility of this set of websites as determined by their compliance with the standards set by that legislative act. The following discussions address the findings of the expert review of each website. www.health.gov (Score: 19/20)

This website is coordinated by the Office of Disease Prevention and Health Promotion, Office of the Assistant Secretary for Health, Office of the Secretary, U.S. Department of Health and Human Services. The site provides several links to resources related to general health, covering a variety of topics. The site serves primarily as a portal site, providing multiple links to other government health websites. The site has the intent of promoting a healthy nation by improving health communication and helping individuals utilize health technologies. To achieve that intent the Office of Disease Prevention and Health Promotion has pulled together key tools, research and reports, and resources for public health and health communication professionals. For this website, there were no instances of audio, video, or multimedia presentations. The site used images to support the content, but the use was selective and added to the quality of the presentation. In most cases, but not all, images provided an ALT tag that described the content provided within the image. The site did not use color exclusively to convey content. The color scheme provided good contrast, with the text being visually distinct and readable. The color scheme varied across all six pages, but each individual page scheme provided a good contrast of colors. In testing the site for color impairments, the grayscale simulation indicated that a significant portion of a web page became unreadable. Figure 7.1 shows the image as viewed through a browser, with Figure 7.2 showing the same web page simulated for monochromatic vision (achromatopia). The area indicated by the red circle highlights the section of the web page that ―disappears‖ under the grayscale simulation. This problem also existed when the page simulated the three most common forms of color blindness (proanopia, tritanopia, and deutanopia). Overall, the layout of the individual web pages varied significantly from page to page. Within the six web pages examined, there were three different main navigation menus used.

226

Figure 7. 1: Unfiltered version of the web page (http://health.gov).

Figure 7.2: Web page simulated as monochromatic (http//health.gov).

227 Some pages used a side-column navigation menu, while other pages used a top navigation bar. As orientation aids, two pages provided a breadcrumbs feature that would support navigation and provide a sense of location within the website. Overall, each page provided some sort of navigation mechanism. Only three of the six pages shared a common page style, providing a consistent page layout, column structure, header use, and navigation style. Even though the pages varied in layout, the pages presented content in a well-organized format that was easy to understand and use. Page titles and section headings helped to organize the content for better understanding. None of the pages provided a search block, or any link to a search feature. The website did not provide a site map feature. One navigation problem related to the provision of a home navigation feature, as only five of the six pages provided a home tab in the navigational menu/bar. However, the use of the home key was confusing. While each of the pages appeared to be a hierarchically subtending page from the Health.gov domain, in some cases the home keys would take the user to the home page of that section of web pages, and not to the home page of the Health.Gov website. This could be very confusing and frustrating to a user trying to get back to the top-level home page for the Health.gov website. The researcher tested the content of the pages for the comprehension level required to understand the material provided. Across all major content sections presented in the web pages, the Flesch-Kincaid grade level was 13.7, with one page rated as 16.9. This finding is of significant concern, as the generally recommended level for web page content is six. This high level may seriously impede understanding by an average web user, and be of even greater problem for anyone viewing the sight with a cognitive impairment. The site worked well with the screen magnifier and the screen reader. The researcher found no accessibility problems related to the use for forms, tables, or timing out features. The site did use scripting to support embedded user interfaces. Testing the pages with the scripting disabled revealed that an equivalent accessible format was available to the user.

Healthfinder.gov (Score: 19/20)

The Office of Disease Prevention and Health Promotion and its health information referral service, the National Health Information Center, coordinate the Healthfinder.gov

228 website. This website offers information and tools to help individuals stay healthy. The intent is to provide a high quality resource that provides the latest, most reliable health related information. Healthfinder.gov provides resources on a wide range of health topics selected from over 1,600 government and non-profit organizations. This site made good use of ALT tags to describe the content within the images found on the web pages, although not every image displayed a corresponding ALT tag description. There were no instances of other audio, visual, or multimedia presentations found on these web pages. The use of images was selective and added to the presentation of the content, and the number of images was not excessive to overwhelm the page layout. There were no instances where color alone conveyed information. The color scheme used on the website was consistent across all web pages and provided a good level of contrast. Regarding color impairments, the researcher noted that some of the images did not appear, and that part of the navigation menu disappeared. Figures 7.3 and 7.4 indicate the missing section of the navigation menu under deutanopia simulation. The grayscale simulation also had problems, with much of the content in the header and footer not displayed properly and the active links disappeared. During the color blindness simulations, on most pages the left column navigation bar was not visible until the mouse hovered over the menu entries, and in some cases the menu entries did not appear. The website was very understandable and well presented. Use of the site was logical and straightforward. The use of section headings helped to organize the presentation of the content. The Flesch-Kincaid grade level was 7.0 for a composite selection of text from the web pages, indicating a good level of understandability for the text information presented. Navigation was consistent on 5 of the 6 web pages, with one page using active links to connect the user to additional resources. That page did not provide the direct access to the other web pages, as provided by the navigation menu used on each of the other pages. Web pages that were hierarchically below the home page provided a breadcrumbs feature, which helps the user orient themselves within the structure of the website. Each page provided a search block, with the block consistently located in the header section. Every web page included a link to a site map, consistently located in the standardized footer section. Each page presented a home link as an entry on the navigational menu. In

229

Figure 7.3: Healthfinder.gov: Page as it appeared in browser.

Figure 7.4: Healthfinder.gov: Page as it appeared under the deutanopia simulation.

230 addition, a home link was available on the five pages hierarchically below the home page, by use of the breadcrumbs feature. The site worked fairly well with the screen magnifier and the screen reader. The researcher noted that for several tested images the screen reader provided speech that was unintelligible. Four of the pages provided a form for acquiring personal data from the web user, in order to get personalized information on health related topics. A review of the source code indicated the lack of ALT tags or LONGDESC tags that would provide descriptive text to explain the functioning of the form elements.

HealthyPeople (Score: 20/20)

The Healthy People organization identifies and measures science-based national objectives for improving the health of all Americans. The organization establishes benchmarks and monitors progress over time in order to encourage collaborations across health sectors; provides information that assists individuals toward making informed health decisions; and measures the impact of prevention activities. In support of that mission, this website provides data and tools to enable individuals across the country to achieve a healthier lifestyle. In scanning the website for audio and video presentations, the researcher found one video clip. In supporting accessibility for that video clip, the web page included an equivalent text transcript in PDF format. Each image on the website included an ALT tag that provided a description of the image‘s content. The website used a consistent color scheme across all web pages, and that color scheme provided appropriate contrast to ensure good readability of the text. There were no instances of the website using color alone to convey information. Color shading designated sections within a web page. A grayscale simulation indicated that some images were not clearly readable. Loss of those images would impair a user‘s ability to navigate effectively. Under color blindness testing, these areas became difficult to recognize or indiscernible. This testing also revealed that the usability of the color based menu bar became problematic. Figures 7.5 and 7.6 indicate the problems encountered with the menu bar.

231

Figure 7.5: HealthyPeople web page as viewed in the browser.

Figure 7.6: HealthyPeople web page as viewed in the Tritanopia simulation.

232 All pages used a consistent page layout, adopting standardized footers and headings. The website was easy to use and readily understandable. Page titles were clear and indicative of the content provided on the page. Subheadings helped to categorize the content within a web page. In conducting the Flesch-Kincaid grade level test, the overall finding was that the site was grade level 12. This is significantly higher than recommended for website viewing and may significantly impede understanding by someone with a cognitive impairment. Use of a standard navigational menu enhanced user navigation of the website. The menu consistently provided a link to the home page. Each web page provided a breadcrumbs feature to assist in orientation. The website did not include tables, flashing images, scrolling text, or forms. The website worked well with the screen reader and magnifier assistive devices. In the examination for embedded user interfaces, the researcher found a script that provided an equivalent function with scripting disabled. Interesting though was the fact that the imbedded user interface feature did not work correctly when tested, while the equivalent function did perform correctly. Additional testing for embedded interfaces revealed that the video clip and search block functionality remained operational with scripting disabled.

Healthcare.gov (Score: 19/20)

Healthcare.gov is a federal government website managed by the U.S. Department of Health and Human Services. This website provides the public with a "one-stop shopping" place for health insurance. The intent of the website is to provide unprecedented transparency into the health care marketplace. The website helps people sort though both public and private health insurance options by asking them a few basic questions in a multiple-choice format. In addition to the focus on evaluating options in health care coverage, the site provides information and resources related to practicing preventative health care, understanding the Affordable Care Act, and assessing the quality of health care. In reviewing the website for audio and visual media, the researcher found three video clips. Each of the video clips provided closed captioning displays. Image use was not excessive, with all images providing ALT tag descriptions. The website used a consistent color scheme, providing adequate contrast for good text readability. Color was not the sole means to convey information on any of the examined web

233 pages. During the color blindness analysis, the researcher discovered that the search block area on each web page became indistinguishable from the surrounding area. This problem existed in the grayscale, protanopia, deutanopia, and tritanopia simulation testing for the common forms of color blindness. Figures 7.7 and 7.8 indicate this problem by comparing screenshots of normal browser viewing to the grayscale simulation of the same web page. The website had a professional appearance, with a standard page layout used across all pages. Header, footers, and the navigation bar were consistent across all pages. The site was user friendly and very understandable. The Flesh-Kincaid test indicated a grade level of 12.6 for the site, indicating that a high school graduate would have a good understanding of the material presented. However, this level is higher than generally recommended. There was good organization of page content, using page titles and subheadings to organize the topics and resources. Adequate spacing separated the content areas within a given page. The navigation bar was consistent across all pages, and included a link to the home page. The standardized header included a search block. There was a problem noted in the use of embedded user interfaces. When the researcher examined the page with the scripting disabled, the text overlapped in areas, significantly impairing the readability. Figures 7.9 and 7.10 present screen captures that illustrate this problem. This problem occurred on three web pages that used a similar form to collect user feedback.

Womenshealth.gov (Score: 18/20)

The Office on Women's Health in the Office of the Assistant Secretary for Health at the U.S. Department of Health and Human Services (HHS) owns and maintains this website. The mission of the Office on Women‘s Health is to improve the health and sense of well-being of all U.S. women and girls. This organization leads and coordinates the efforts of all the HHS agencies and offices involved in women's health. The website presents content related to the agency‘s programs that focus on the health of women, along with providing resources that educate physicians and other health professionals. The website serves to educate and motivate the public to live healthier lives by giving them trustworthy and accurate health information.

234

Figure 7.7: Healthcare.gov: Website under normal browser viewing.

Figure 7.8: Healthcare.gov: Website under grayscale simulated viewing.

235

Figure 7.9: Healthcare.gov: Normal browser view of web page.

Figure 7.10: Healthcare.gov: Browser view with scripting disabled.

236 In reviewing the web pages, there were no instances of audio or video presentations found. The use of images was sparing and did not clutter the pages. All images except one included corresponding ALT tags that provided descriptions of the image content. There were no instances of color used as the sole means to convey information. The color scheme was consistent throughout the site, providing appropriate levels of contrast supporting the readability of the text. The Flesch–Kincaid grade level testing indicated a grade level of 15.9 for these pages. This is of concern, as this elevated grade level requirement may impede comprehension of the site by individuals with cognitive impairments, or for users for whom English is not their first language. Under grayscale testing, some areas of text became very difficult to discern. Three of the pages had similar contrast issues, and this problem existed in the simulations for protanopia, deutanopia, and tritanopia. The web pages used a consistent layout, providing standardized headers and footers. The pages were readily understandable with the content presented clearly. Pages used descriptive titles and employed subheadings to organize the content. In those pages that had sectioned areas of content, the use of graphical separators more distinctly indicated the borders between sections. A standard navigational menu appeared on each page, along with a breadcrumbs feature to assist in providing the user with an indication of location within the website. A search block was part of the standardized header, appearing on each web page. There was no indication of the availability of a site map. In examining the homepage, when the researcher attempted to enter an email address into a user interface area, the web page experienced a redirecting error. The researcher retested that function, but observed another failure. A similar user interface was present on three other pages and those interfaces functioned properly. The redirecting error is a functionality failure that impedes the accessibility of this site.

MedlinePlus (Score: 20/20)

MedlinePlus is a website provided by the National Institutes of Health and produced by the National Library of Medicine. The site attempts to offer information about diseases, conditions, and wellness issues in clearly understandable language. The goal of the MedlinePlus website is to provide an online resource that offers reliable, up-to-date health information,

237 anytime, anywhere, and for free. The content includes the latest treatments, information on a drug or supplement, medical definitions, medical videos, and illustrations. The site provides links to the latest medical research on clinical trials for specific diseases or conditions. The home page presented a slide show using changing images, with each slide accompanied by a descriptive ALT tag. The use of images on the website was selective and appropriate for the content presented. Each image included a corresponding ALT tag. The site‘s color scheme was consistent across all web pages and provided appropriate color contrast. There were no instances where the web page solely conveyed information by the use of color alone. The site tested well when simulated for achromatopia, protanopia, tritanopia, and deutanopia. The site used a consistent page layout, employing standardized headers and footers. The site was clearly understandable, well organized, and easy to use. The web pages provided descriptive titles and used section headings to categorize the content. The Flesch-Kincaid test indicated a grade level of 15.4. This is of concern as it may pose a problem for the cognitively impaired, and may be overly complex for the average web user. Except for the home page, all other pages used a standardized navigation bar and provided a breadcrumbs feature. However, on one of the pages the breadcrumb feature incorrectly identified itself as being the home page. The navigation bar and the breadcrumbs feature were consistently located directly below the standardized header on each page (except the home page). Each page, except the home page, included a link to the home page in both the navigation bar and the breadcrumbs feature. A search feature was included in the standardized header used on each page. An active link to the website map was included in the standardized footer as the base of each web page.

Summary of Findings

Overall, the sites scored an average of 19.17 out of the 20 possible points (95.83%). Table 7.2 presents a ranked listing of the site based on their scores. The indication is that the sites performed well, although Section 508 requires full compliance with all the standards. Only two sites studied (HealthyPeople.gov and MedlinePlus.gov) scored 100% of the available points. With the sites scoring in a range of 90-100%, it indicates that these federal organizations have

238 Table 7.2: Websites ranked by expert review scores. Websites Ranked by Expert Review Scores Website Score HealthyPeople.gov 20/20 (100%) MedlinePlus.gov 20/20 (100%) HealthCare.gov 19/20 (95%) HealthFinder.gov 19/20 (95%) Health.gov 19/20 (95%) Womenshealth.gov 18/20 (90%) Average Score: 19.17/20 (95.83%)

placed a priority on meeting the accessibility criteria of Section 508. The overall average of 95.83% indicates that a great deal of the health care content provided by these websites is accessible to persons with disabilities. Table 7.3 identifies the key accessibility issues identified by the expert review for each of the websites examined. These issues are the problems that relate directly to Section 508 criteria used to assess the accessibility of the websites. These issues primarily affect users with visual, auditory, and cognitive disabilities. This study examined the web pages for additional accessibility issues that were not part of the scoring of the website. Table 7.4 presents that list of findings.

Recommendations

The expert review revealed the key accessibility issues that indicate noncompliance with the criteria established by Section 508. Based on the criteria used in this study, the accessibility issues that directly relate to Section 508 criteria fall into two categories:  Not all images provided an ALT tag description.  An embedded user interface was not directly accessible. These issues primarily affect users with visual, auditory, and cognitive disabilities. To correct the above problems, the researcher provides two recommendations. First, all images should include ALT tag descriptions. Second, the web manager should ensure that embedded user interfaces are directly accessible. By implementing these two recommendations, the website would be compliant with the standards mandated by Section 508. Compliance with

239 Table 7.3: Key findings as part of the scoring effort. Key Findings Directly Related to Section 508 Criteria Website Section 508 Criteria Finding

Not all images provided an ALT tag Health.gov §1194.22 (a) description.

Not all images provided an ALT tag HealthFinder.gov §1194.22 (a) description.

This website met all tested criteria related to HealthyPeople.gov Section 508. An embedded user interface was not directly HealthCare.gov §1194.22 (f,l,m) accessible.

Not all images provided an ALT tag §1194.22 (a) description. Womenshealth.gov An embedded user interface was not directly §1194.22 (f,l,m) accessible. This website met all tested criteria related to MedlinePlus.gov Section 508.

Table 7.4: Key findings that were not scored. Accessibility Issues Identified During the Expert Review Website Potential Accessibility Problems  Not all images had ALT tags or LONGDESC information  Parts of website not visible for some types of color blindness Health.gov  Navigation mechanisms were inconsistent  Website required a high grade level to assure comprehension  Not all images had ALT tags or LONGDESC information HealthFinder.gov  Parts of website not visible for some types of color blindness  Navigation mechanisms were inconsistent  Parts of website not visible for some types of color blindness HealthyPeople.gov  Website required a high grade level to assure comprehension  Parts of website not visible for some types of color blindness HealthCare.gov  Website required a high grade level to assure comprehension  Imbedded interfaces failed to provide full content accessibility  Not all images had ALT tags or LONGDESC information  Parts of website not visible for some types of color blindness WomensHealth.gov  Website required a high grade level to assure comprehension  Imbedded interfaces failed to provide full content accessibility MedlinePlus.gov  Website required a high grade level to assure comprehension

240 these standards would help assure that these websites would be more accessible by individuals with disabilities. The web manager has several ways to assure that images are accessible to individuals with disabilities. One solution is to provide short text equivalents for images, through the use of ALT tags. The webmaster should provide a text equivalent for every non-text element (e.g., via ALT or LONGDESC tags). This recommendation extends to cover: images, graphical representations of text (including symbols), image map regions, animations (e.g., animated GIFs), applets and programmatic objects, ASCII art, frames, scripts, images used as list bullets, spacers, graphical buttons, sounds (played with or without user interaction), stand-alone audio files, audio tracks of video, and video. As an example: If the website provided a search block presented by the use of an image, then the source code should look like the following;

Example: Search Where the ALT tag is the alt= “Search” string, and the descriptive text is the word Search.

One key consideration is that the ALT tag be appropriately descriptive so ensure that the content of the image provides sufficient detail to appropriately convey the meaning. In some cases, the use of a LONGDESC tag may replace or enhance the ALT tag. When a shorter text equivalent would not suffice to adequately convey the function or role of an image, the webmaster could provide additional information in a file designated by the LONGDESC attribute:

Example: search In searchblock.html: This search provides the website user with the capability of searching through a collection of over 300 resources related to the topic of nutritional cooking for diabetics. These resources are limited to the official government surveys and reports issued between 1997 and 2010.

Selection of the use of the ALT tag or the LONGDESC depends on how much information is necessary in order to provide the user with an appropriate understanding of the content of the related image.

241 All embedded user interfaces must be directly accessible to users with disabilities. This study tested web pages for directly accessible scripts. That testing examined the web page functioning to ensure that pages are usable with scripting disabled. User with disabilities may rely on assistive devices to view web pages, and these devices may often have problems with web pages the use scripting. Web designers should provide source code that will accommodate the use of assistive devices by providing equivalent functionality or equivalent information on an alternative accessible page. A script (set of code instructions) invokes when a certain event occurs within a web page viewing session. As an example, when the mouse hovers over a section of the website text, the text may highlight or change color. This color change or highlighting provides a visual sign that this section of text provides a unique functionality, such as allowing the users to access a predefined form. The following provides an example of how developers can create web pages that work with those user assistive devices that do not support scripting. In developing the source code for a web page, the NOSCRIPT element allows for alternate content when a script cannot execute. This NOSCRIPT section of code allows content to be available to users when the user‘s assistive device is unable to evaluate scripts. In the following example, the web page contains scripting that provides some dynamically created data in the web page.

Example:

Access the data.

In this example, the script may be generating a form for the user to complete in order to get more personalized or customized data from a search engine application. If the user‘s assistive device does not support scripts, the user may still retrieve the data through the alternate link. These recommendations relate specifically to the criteria used within this study to identify compliance with Section 508. The study also produced findings that indicated additional

242 issues that may indicate accessibility related problems. The following presents recommendations that address those issues. Websites that had problems related to color blindness included Health.gov, Healthfinder.gov, HealthyPeople.gov, Healthcare.gov, and Womenshealth.gov. The color blindness simulator indicated that sections of the web pages would become unreadable when tested for visual imparities related to color blindness. The recommendation is for web developers to examine each page for issues related to color blindness. Addressing this issue could be as simple as adopting the technical best practice of not using color to indicate something specific on your page. As an example, the web page may present the users with a form that has required fields. Indicating the required fields by using red text may not be a big enough distinction for a color blind person. The web developer should add another visual cue, such as an icon or other text based descriptive element to indicate that the field is required. Another effective practice would be to avoid the use of color combinations that provide low levels of contrast. One such example would be to avoid red/green combinations that would provide little contrast to individuals with red/green color blindness (protanopia). While adopting such practices help ensure more accessible websites for individuals with color blindness related problems, the more appropriate solution is to include testing that simulates the various forms of color blindness. User based testing is another approach to examining for color blindness related problems. However, the availability of individuals with specific forms of color blindness would likely make this approach generally impractical. A more feasible approach would be to use software products that simulate the various forms of color blindness. As an example, this study used an online service specifically configured to conduct these types of tests (Colorfilter.wickline.org, n. d.) That service is free and provides a relatively fast and practical way to indicate possible accessibility problems related to the more common forms of color blindness. Vischeck (Vischeck, 2011) and 456 Berea Street (456 Berea Street, 2011) are informative website that identify and discuss the benefits of similar testing products. The Health.gov and HealthFinder.gov websites used an inconsistent presentation for their navigation mechanisms. The recommendation is for these sites to redesign their navigational bars or menus, so that there is a standardized and consistent presentation across all pages within the website. Navigation mechanisms provide a means to direct the user to key sections and/or pages within a website. These mechanisms should clearly state the available destinations and

243 should indicate the current position of the user within the website. These navigational tools should present in a manner that is consistent visually and structurally from one web page to another. A uniform and logical presentation helps the user quickly grasp an understanding of the navigational options and the operation of the mechanism. Consistency of operation and presentation helps to avoid confusion or disorientation. To achieve a standardized navigation mechanism, the web developer should adopt the following design practices. In order to ensure a uniform presentation of the navigational mechanism, the developer should:  Establish a consistent visual presentation across all pages;  Maintain a standardized sequential ordering of the navigational options;  Use language and terminology in a consistent manner; and  Maintain consistency in the location of navigation controls. These practices support the smooth transition from page to page, enabling the user to navigate easily and quickly. This standardized presentation supports the quick and easy understanding of the navigational mechanism and of the options available. Additionally, the navigation menu should always include a link to the site's homepage. The researcher tested all websites in the sample for an indication of the educational grade level needed in order to assure comprehension of the content of the website. The Health.gov, HealthyPeople.gov, WomensHealth.gov, and MedlinePlus.gov websites had very high grade levels required for comprehension. The recommendation is for the web manager to rewrite the text sections, so that the comprehension level aligns with the recommended reading level. The goal should be to present the user with text that is clear and easily comprehended, using language that is simple and straightforward. The Flesch-Kincaid grade levels for these four websites ranged from 12.0 to 15.9. These scores were significantly about the average readability for web users in general and greatly exceeds the recommended reading level (a level of 8.0) for health care information (JuicyStudio, 2009; Paasche-Orlow, Taylor, & Brancati, 2003). Improving the user‘s ability to comprehend the text readily on a web page will allow the site to reach a larger audience. Language should be concise, and not rely on overly technical terminology. Terminology that may be unduly complex should have definitions or explanations offered. General practices for assuring a high level or readability include:

244  Terminology is clear and concise,  Sentences are short and arranged into compact paragraphs, and  Definitions accompany difficult terms. Several readily available services and techniques are available to the web developer to check the comprehension level associated with a section of text. While user based testing is one option for assessing the readability and understandability of the website, this study used the Flesch-Kincaid assessment tool. A more complete discussion of readability and comprehension, along with a discussion of other assessment tools and techniques, is available in the work by Gemoets, Rosemblat, Tse, and Logan (2004). The expert review relied on a data collection instrument that provided a general indication of compliance with the accessibility criteria within Section 508, and identified potential additional accessibility problems. These potential problems varied in nature, and could negatively affect the use and comprehension of these websites for individuals with a variety of disabilities. In assessing the level of compliance with Section 508 criteria, the sites generally performed well. However, in examining the sites for other accessibility related issues, the study indicated many more potential problems. The number of problems revealed by this testing method indicated its usefulness as a tool for determining the accessibility of a website, or a group of websites. As such, this approach and the associated data collection instrument provided valuable information about the accessibility of a website. However, the data collection instrument does have weaknesses. The evaluation process relies heavily on expert opinion, and the ―quantification‖ that appears in the scoring may be significantly influenced by the knowledge and skills of the accessibility expert administering the test. This approach is also reliant upon picking the right set of criteria that go into evaluating each of the 10 questions used in the data collection instrument. In developing those criteria for this study, the researcher made appropriate efforts to select criteria that were representative and accomplishable. The researcher acknowledged that the data collection instrument and the criteria used, do not include all possible tests for accessibility and provide only indications of problems areas. At the organizational level, federal entities, organizations should adopt this method as an integral part of an overall program of website accessibility evaluation. The data collection tool and the criteria used in this testing approach produced an array of findings that indicate the

245 accessibility of this group of studied websites. These findings indicate the value of the expert review as a method of indicating compliance with the standards put forth within Section 508. Moreover, this approach produced interesting and informative findings for accessibility issues that extend beyond those criteria directly related to Section 508. The expert review is an effective tool for assessing accessibility, and web managers should employ this testing method to support both formative and summative evaluation efforts. The expert review method produced findings that go well beyond those detected by the use of automated testing. More significantly, in comparison to the automated testing, the findings from the expert review produced greater insight as to the affect that these problems would have on an individual with a disability.

Chapter Summary

This chapter provided an analysis of the detailed results from the data collection effort to locate accessibility problems in federal health care websites. The primary focus was to look for indications of noncompliance with the accessibility standards identified in Section 508 of the Rehabilitation Act of 1973. The data collection effort employed an expert review of federal agency websites. The data collection effort used a set of ten questions that collectively addressed all 16 of the accessibility standards provided within Section 508. The researcher examined six federal health care websites, looking at six pages in each website. The data collection questionnaire guided the researcher‘s examination of the web pages to assess how well the website complied with Section 508 standards. The researcher was the sole examiner for this study. In assessing the overall quality of these websites, the sites scored an average of 19.17 out of the 20 possible points (95.83%). The indication is that the sites performed well, with two sites (HealthyPeople.gov and MedlinePlus.gov) scoring 100% of the available points. The sites‘ scores ranged from 90% to 100%, indicating that these federal organizations have placed a priority on meeting the accessibility criteria of Section 508. The overall average of 95.83% indicates that a great deal of the health care content provided by these websites is accessible to persons with disabilities.

246 The expert review revealed two key accessibility issues relating to noncompliance with the standards established by Section 508. These accessibility issues are 1) not all images provided an ALT tag description, and 2) an embedded user interface was not directly accessible. These issues could significantly affect users with visual, auditory, and cognitive disabilities. Two recommendations addressed these issues. First, all images should include ALT tag descriptions. Second, the webmaster should ensure that embedded user interfaces are directly accessible. By implementing these two recommendations, the website would be compliant with the standards mandated by Section 508 and would provide greater accessibility to federal health care information.

247 CHAPTER 8

CONCLUSION

This chapter provides an analysis of the study findings to indicate the accessibility of federal health care websites. Included in this chapter is an analysis of the study approach, an assessment of the methodology, and the identification of implications for the conceptual framework. Also presented are key recommendations for federal organizations and web managers that would promote improved accessibility for federal health care websites. The chapter concludes by offering suggestions for future research on web accessibility.

Implications of Study Findings

Chapters four through seven presented findings from the separate data collection efforts, and included the identification of key issues and recommendations that addressed the accessibility of the federal health care websites examined by this study. Each of those chapters presented a specific method of data collection analysis that provided a unique perspective on accessibility. The analysis covered issues related to policy, content, compliance, and aspects of website presentation related to accessibility. The information presented in those four chapters identified broad issues that related to national policy, organizational practices, web development techniques, and the use of these websites by individuals with disabilities. The following presents a discussion of the implications that flow from those discoveries. The discussion identifies the implications that arose from each method of analysis. Table 8.1 identifies the key implications from the findings, categorized by stakeholder group.

Implications from the Policy Analysis

The intent of Section 508 was to promote the full accessibility of federal websites. Based on the findings presented in this set of studies, as a national policy that objective has not been fully achieved. The policy review indicated several problem areas within in the two laws that

248 Table 8.1: Summary of implications. Summary of Implications Stakeholder Group Implications  Consolidate existing policies to clarify terms and intent.  Expand policy to include audit and enforcement of Section 508 provisions. Federal policy makers  New policies that commit to: for web accessibility o A program of awareness building, o Training and certification, and o Ongoing research.

The continued existence of accessibility errors may imply that these agencies would benefit from:  Increased awareness of the legal and ethical need to comply Federal health care with the law. agencies  Training on how to institutionalize the best managerial and technical practices associated with assuring accessibility and with conducting accessibility testing.

The continued existence of accessibility errors may imply the that web managers could benefit from:  Awareness training on legal mandate. Federal health care web  Development tools that promote accessible websites. managers  Training in accessibility testing programs.  Technical best practices and success stories.

Users can play a more active and contributory role.  User feedback would provide web managers and agency Users of federal health organizational leaders with feedback on actual accessibility care websites issues discovered.  Become active advocates for change.

most directly attempt to promote web accessibility. The expert review and the automated testing looked for accessibility errors using the criteria established in Section 508, and found several instances where the websites were not accessible, based on those standards. The content review indicated that these websites did not consistently provide accessibility statements that would be of high value to the users with disabilities. These indications point to a conclusion that the national policies that attempt to ensure accessibility have clearly failed to meet their goal of full accessibility. Meeting the goal of full accessibility for the nation requires changes to these national policies.

249 In calling for changes to the national policy surrounding web accessibility, it is convenient to assemble the issues and recommendation into a themes or categories. The first category discussed relates to funding issues. The language used in the Section 508 amendment includes the term undue burden, which could allow site owners to ignore providing full accessibility. Policy critics often deride Section 508 as another unfunded mandate forced upon government agencies. In this current technological environment, it is unclear whether providing accessibility during web development requires any significant effort or adds any undue cost to the project. Additional research may well determine that the current generation of web development tools provides adequate features that act to ensure the creation of accessible websites. That research may also determine that implementing an effective and ongoing accessibility testing program would not constitute an undue burden. The issue is that there may be no need for an undue burden clause given the current technological environment, as there are no significant costs associated with providing and ensuring web accessibility. The undue burden clause could be stricken from the current national policy language. This policy change would encourage all providers to ensure accessibility and serve to promote the outcome of fully accessible websites. In attempting to understand why some websites remain inaccessible, the study examined the two national policies that promote web accessibility. The indication was that the addition of language that promoted education and training related to accessibility would strengthen these policies. This training would target the organizational leaders who establish key institutional policies that enact procedures that would ensure accessibility, and would include those individuals (federal web managers) who implement those institutional policies. These two groups (organizational leaders and web managers) have the greatest influence towards incorporating accessibility into federal websites. The conclusion is that the establishment of a national program to train and certify federal web managers in the area of accessibility is appropriate. A web accessibility training program would inform organizational leaders and web developers about the legal and ethical foundations for providing accessible websites. The program would serve to educate the organizational leaders as to the best practices for institutionalization (formalization, adoption, and embedding changes within the organizational culture) as formal policies and procedures. One key training topic would be the importance of testing for accessibility throughout the website development

250 cycle. Educating the organizational leadership on the importance of providing accessibility would integrate a change into the corporate culture. By mandating training, website developers would benefit by having access to the latest procedures, technical tools, and programming techniques that serve to promote web accessibility. Overall, the findings of the policy analysis are consistent with many of the criticisms made in other formal reviews. Golden and Buck (2003) found that webmaster‘s had significant concerns over the interpretation of Section 508 and the American‘s with Disabilities Act. The concerns dealt with the lack of specificity in the language used and the lack of clarifying guidance. The findings of this study are consistent in identifying the same problems. Additionally, this study proposes that training can be an important means to address these problems. The findings and recommendations made within the study are also consistent with the work by Jaeger (2006, 2004). Key areas of agreement include the need to build awareness, provide training, disseminate best practices, and provide better oversight and enforcement. This study also identifies the need for building accessibility awareness as part of that training. This need is consistent with the analysis and recommendations made by ATT508.com (AT508.com, 2010), a national association for promoting accessibility. In chapter 4, this study provided additional details as to how to implement an enforcement office that could improve the accountability to, and enforcement of, the standards mandated by Section 508. This study also identified those programs that may be the most feasible to implement given the current political and economic environment.

Implications from the Web Content Review

This study used a content review to examine federal health care websites for content typically known as a website accessibility statement. The intent of this study was to examine and compare the website accessibility statements of this selected group of websites in order to understand current practices, usage, and to assess the quality of the statement. Key findings within the examined group of 20 websites were:  Only 15 websites (75 percent) met the criteria for having a web accessibility statement.

251  Seven websites (47 percent) scored as HIGH quality by meeting all four criteria.  Eight websites (53 percent) scored as MEDIUM quality, by meeting two or three of the criteria. Overall, there was a good indication that the agency web managers are making serious efforts to provide informative accessibility statements on their websites. The literature review found no similar studies that systematically assessed the existence and quality of web accessibility statements. The criteria used within this study to determine the existence and quality of web accessibility statements derived from expert opinion. The researcher also noted that no prior studies involved conducting user-based assessments to determine what content and presentation format is appropriate for web accessibility statements. Chapter 3 discussed the development and identification of the specific criteria used within this study. As such, this is the first study that systematically attempts to determine the existence and quality of web accessibility statements. The implication is that this work provides an initial framework and set of criteria that could be used as a standard method for evaluating the existence and quality of web accessibility statements. However, this researcher recommends that any such standard method should include criteria validated by user-based research. The lack of prior research in this area prompted the researcher to look more closely at the content themes (types of information) presented in the examined web accessibility statements. This use of thematic content analysis is appropriate, as this was an exploratory investigation. This thematic analysis provided findings that indicated changes that could result in websites that are more accessible. The implication is that this thematic content analysis provided an initial understanding of the content common to this class of websites and provided fundamental knowledge from which to base future research. That future research could explore the issues related to content and format of presentation from a user based point of view. Additional research aimed at obtaining a user based assessment of web accessibility statements would be appropriate in order to ground the design and content upon the users actual needs. Such a user centric perspective could result in web accessibility that better serves the needs of the intended user, being the individual with disabilities. The web content review noted that most web accessibility statements included content that identified an accessibility point of contact. Such a venue for communication between the user and the provider of the website could be a valuable feedback mechanism that would serve to

252 promote accessible websites. The implication is that by providing a point of contact, the organization has a positive outlook towards assuring the accessibility of the website. User feedback would provide web managers and organizational leaders with information on actual accessibility issues discovered during the use of the website. In this way, the users of these federal websites can play a important role in the ongoing promotion of accessible websites.

Implications from the Automated Testing and Expert Review

Automated testing provided quantitative measures related to the accessibility of the websites, as counts of errors and the types of errors found. This was also true for the expert review. The automated testing was a more code-focused examination, while the expert review provided a user centric focus on quantitative and qualitative measures related to the accessibility of the website. In comparing the two methods of evaluation, based on the detailed findings previously presented, the expert review identified all of the errors discovered by the automated testing. The value added by including the automated testing method was that it underscored how a relatively fast, simple, and cheap method is suitable for providing an initial indication of accessibility problems that may exist within a website. In employing the automated testing and expert review methods, the intent was to locate accessibility problems in the sampled federal health care websites. The primary focus was to look for indications of noncompliance with the accessibility standards identified in Section 508 of the Rehabilitation Act of 1973. The key findings from these two methods included:  In assessing the overall accessibility, these websites scored well, with an average of 19.17 out of the 20 possible points (95.83%).  The sites‘ scores ranged from 90% to 100%. The implication is that these federal organizations have placed a priority on meeting the accessibility criteria of Section 508, but are not in full compliance. Another implication is that a great deal of the health care content provided by these websites is accessible to persons with disabilities. In examining the types of accessibility problems found, the methods of evaluation revealed two key accessibility issues relating to noncompliance with the standards established by Section 508. These accessibility issues are 1) not all images provided an ALT tag description,

253 and 2) embedded user interfaces were not directly accessible. The implication is that these issues could significantly affect users with visual, auditory, and cognitive disabilities. Two recommendations addressed these issues. First, all images should include ALT tag descriptions. Second, the webmaster should ensure that embedded user interfaces are directly accessible. By implementing these two recommendations, the websites would be compliant with the standards mandated by Section 508 and would provide greater accessibility to federal health care information. Another key implication is that these federal agencies are not carefully testing and reviewing their websites in an effort to discover and correct accessibility problems. The automated testing method discovered accessibility problems in a quick and easy manner using a readily available online tool. If federal agencies had employed such a simple procedure within their web testing and development process, the accessibility problems discovered in this study could have been avoided. The expert review also implied a failure by agencies to conduct accessibility testing. The expert review revealed errors that the web developer could have corrected prior to publishing the web page. While the expert review requires a more skilled evaluator and sophisticated evaluation process, it is clear from these findings that such an evaluation is not part of the organizational practices and policies that go into the agencies‘ web development procedures. The implication is that agencies have not implemented a sufficiently comprehensive accessibility testing program as part of their website management efforts. This deficiency in their testing programs could stem a number of reasons that may include a lack of knowledge of accessibility requirements, the lack of training on accessibility testing methods, or from a conscious decision to ignore the legal requirement. Further examination of the issues behind this observed lack of adequate and appropriate accessibility testing is required. Most studies of website compliance with the criteria established by Section 508 used automated testing. While the specific software used to measure compliance varied in those tests, the method of analysis has been similar across the majority of the studies, and the indication is that regardless of the specific software product that was employed, the results would be similar (Jaeger, 2006; Thatcher, Kirkpatrick, Heilmann, et al. 2006). In this study, the automated testing revealed that four of the six websites had errors (67%), with 12 errors detected, encompassing five error types. As these errors violated five

254 categories of Section 508 criteria, the indication is that federal health care websites suffer from a range of accessibility issues. These findings are consistent with prior studies that used automated software testing. Jaeger (2006) also noted that many studies have used automated testing to assess the accessibility of websites and that the accessibility was generally poor. Several studies found that websites had major accessibility problems. Lazar, Dudley- Sponaugle, and Greenidge in 2004 indicated poor accessibility in studies of private and non- profit websites (Lazar, Beere, Greenidge, & Nagappa, 2003), for-profit commerce websites ( Sullivan & Matson, 2000), United States state websites (Ceaparu & Shneiderman, 2002), and even U.S. federal websites (Stowers, 2002) were found to have major accessibility problems. The results of this study are consistent with those prior efforts, indicating that ongoing monitoring of compliance continues to be an important issue. Only one prior study of website compliance with Section 508 used expert testing as the basis for investigation. That study by Jaeger (2006) examined 10 federal government websites that provided electronic government resources and services. Jaeger found that the expert testing revealed many accessibility problems. This study used a similar approach as was used by Jaeger‘s work, in that the websites were scored using specific accessibility criteria from Section 508, and looked at additional accessibility features that fell outside of the expert testing scoring criteria. In considering the larger set of accessibility criteria, this study also noted many accessibility issues for the sites tested. This study and Jaeger‘s (2006) study looked at criteria that related directly to Section 508 standards and scored the studied websites to get an indication of the relative ranking of accessibility of the examined websites. For the ten sites evaluated by Jaeger, the scores (out of a possible 20 points) ranged from 9 to 17 points, with the average score being 10.8. For this study (also using the 20 point scale) the scores for the studied websites ranged from 18 to 20, with an average of 19.17. This study made significant efforts to reflect accurately the assessment criteria used in the Jaeger study to allow for comparability of findings. In comparing the results of the Jaeger study to the finding of this study, the indication is that there may be a general improvement in the accessibility of federal websites; however, the two studies examine two distinctly separate categories of websites. In addition, it is unclear if they criteria for scoring in the Jaeger (2006) study and this study were identical. As such, the comparability of the two

255 works may be inappropriate. Later in this chapter, the section entitled ―Assessment of Methodology‖ provides additional discussion of this issue.

Summary of Implications

This study has implications that extend to address national policy, web development practices, organizational policies, and the users of federal health care websites. The policy environment could benefit from efforts to consolidate existing policies, clarify the terminology used in the key legislative acts, and from adding provisions that serve to address the need to audit for compliance and provide a more effective means to enforce compliance. By strengthening the policy environment, the expected outcome would be a higher level of compliance that would lead to more websites providing accessible content. While the web content review indicated that federal health care agencies are doing a good job at providing high quality web accessibility statements, there is concern that the basis for determining quality is not firmly rooted in user based assessments research. Expert opinion was the source of the criteria used to determine the existence of the web accessibility statement and the assessment of the quality of that statement. There is a need to conduct additional research that determines the content and format of presentation for these statements, based on actual users‘ needs. From that fundamental research, a better understanding of the actual ―quality‖ aspects would develop, leading an improved set of assessment criteria for determining the existence and quality of these statements. This research effort found the studied class of websites to have several accessibility problems. The policy analysis revealed issues and problems with the current policies that attempt to ensure accessible websites. The key implication being that these policy issues may impede federal agencies‘ efforts to provide websites that are accessible to all individuals. Moreover, the existence of such accessibility problems suggests that agencies are not fully testing their websites prior to release into production. This lack of testing could stem from a lack of training and awareness by the web manager, as to the legal requirement for providing accessible websites, or from the lack of knowledge regarding the available tools and techniques that would assist in the successful production and testing techniques that help ensure accessibility. Further, the existence of these accessibility errors could indicate that federal

256 agencies have not established internal organizational policies and procedures that would serve to ensure sound business practices that promote a managerial environment that is conducive to the delivery of fully accessible websites. From chapter 6, the automated testing indicated a relatively poor level of compliance when looking at the percent of websites that were fully compliant with the Section 508 standards. As the expert review (chapter 7) examined accessibility by looking at a greater number of criteria, the performance of the websites overall was good. The indication is that while the web sites are not fully compliant, the agencies are making significant efforts to comply with the standards put forth by Section 508, and are creating websites that are, for the most part, accessible to users with disabilities. In aligning these findings with the findings of the side-by- side policy analysis, the indication is that the policy issues (indentified in chapter 4) have not severely deterred the agencies‘ efforts to provide websites that are to a large degree accessible.

Assessment of the Research Questions

This study posed three research questions that relate to website accessibility. These three questions were:  Are federal health care websites compliant with Section 508 requirements?  What are the issues and barriers to attaining accessibility?  What changes would promote websites that are more accessible? This section summarizes the key findings of this study and indicates how these findings contribute toward a better understanding of these three research questions. Table 8.2 indicates the relationship between the methods and the research questions.

Are Federal Health Care Websites Compliant with Section 508 Requirements?

The automated testing and the expert review provided quantitative and qualitative data that identified accessibility problems. These problems were instances of non-compliance with the criteria within Section 508. The key findings from these two methods included:

257 Table 8.2: Relationship of research questions to methods. Relationship of Research Questions to Research Methods Does the Method Address the Research Question? Research Methods Research Side-by-side Website Automated Expert Questions Policy Content Testing Review Analysis Review Are federal health care websites compliant with Yes Yes Section 508 requirements? What are the issues and barriers to attaining Yes Yes Yes accessibility? What changes would promote websites that are more Yes Yes Yes Yes accessible?

 Most sites (67%) were not in full compliance, with 4 of 6 sites having problems.  In assessing the overall accessibility of these websites, the sites scored an average of 19.17 out of the 20 possible points (95.83%).  The websites‘ scores ranged from 90% to 100%.  Accessibility problems were varied in nature, but usually simple to correct. The high level of scoring in the expert review suggests that federal organizations have placed a priority on meeting the accessibility criteria of Section 508, but have not achieved the intent of Section 508, which is to provide fully accessible websites. Because of this lack of complete compliance with Section 508, a significant amount of the health care content provided by these websites may not be accessible to persons with disabilities.

What are the Issues and Barriers to Attaining Accessibility?

The side-by-side policy analysis, automated testing, and expert review methods each contributed information that addressed this research question. The policy analysis found substantial differences between the accessibility provisions mandated by the ADA and by Section 508 of the Rehabilitation Act. The broad language of the ADA left much room for interpretation, which was then adjudicated by the courts on a case by case basis. Such a case

258 specific approach did not provide a unified and clear mandate, particularly for such issues as scope of coverage. Section 508 was more technical and specific in nature, but still had ambiguous or unclear areas, as the definition of the undue burden clause. This lack of policy clarity may have resulted in varied interpretations and inconsistent implementations at the practical level by the implementers of federal websites. These problems could explain why federal entities may have been hesitant to embrace web accessibility and slow to institutionalize practices and testing programs that would help to assure accessible websites. The indication is that the policy environment does provide issues and barriers to attaining accessible websites. The automated testing and expert review methods, as discussed in the preceding section, indicated that federal organizations have placed a priority on meeting the accessibility criteria of Section 508, but have still not provided fully accessible websites. The study findings indicate that websites that are more accessible are attainable, if an effective accessibility testing program were a part of the federal agencies‘ website development process. As this study found accessibility problems that are correctable with only minor resources or efforts, then federal agencies should be able to do the same. The indication is that federal agencies are not conducting rigorous and systematic accessibility testing. The deficiency in testing could stem from several possible causes. Such causes could include inadequately trained web managers; or poor internal management oversight. Poor management oversight could be a result of the lack of internal policies and practices that should be imbedded within the web development process.

What Changes Would Promote Websites that are more Accessible?

Each of the four methods provided information that led to recommendations for changes that could make websites more accessible. Policy recommendations suggested aligning the language of the two legislative acts, so that the specific intent of the legislation would be clearer and more explicit. Additionally, the policy terminology used was sometimes not well defined. One example was the undue burden clause. The recommendation for this issue included conducting additional research to determine if that clause was still appropriate. That clause had the intent of assuring that federal agencies would not incur excessive costs in order to attain accessible websites. As technology and software tools have advanced significantly since Section

259 508 became law in 1990, attaining accessible websites may no longer require an undue burden for web developers. The web content review contributed significant knowledge related to the existence and quality of the accessibility statements. One of the more interesting facts revealed was the lack of academic research behind the formatting and content of these statements. While the intent of these statements is to provide assistance to the user with a disability, in many cases the statements lacked adequate technical guidance that would assist the user in making use of the accessibility features provided within the website. From this research effort, the recommendations addressed the need for additional research that would include user-based studies. User based studies would help to assure that these statements more closely meet the actual needs of users with disabilities. This web content review produced recommendations that called for higher quality in web accessibility statements. These recommendations addressed the need for web accessibility statements that would:  Focus on content that was more usability centered versus technically oriented.  Refrain from assuming too high a level of user technical knowledge or reading comprehension.  Present content using less complex terminology, avoiding overly technical terms. While these recommendations are subjective determinations, they reflect concerns that need addressing when the goal is to provide information in a way that is understandable and easily used by the prospective user of the website. The automated testing and the expert reviews revealed the more common types of accessibility errors found within these websites. By understanding the types and frequency of accessibility problems, it was possible to tailor recommendations. The use of automated testing was particularly beneficial as it indicated the value of conducting even the most simple accessibility testing. By using a readily available, free, and easy to use software tool, this study quickly identified important accessibility issues that may impede website use. By testing with such a simple automated test, the indication was that federal agencies were failing to conduct any sort of rigorous accessibility testing during the web development process. As an expert review provides greater thoroughness compared the automated testing, the expert review provided greater insight into the scope and depth of the accessibility issues found

260 within this group of websites. The recommendations related to need for providing descriptive text when the web page uses images to convey information, and to provide equally accessible functionality when embedded interfaces are present within a web page. By implementing these two recommendations, the websites would be compliant with the standards mandated by Section 508. Compliance with these standards would help assure that these websites would be more accessible by individuals with disabilities.

Summary Assessment of Research Questions

The four methods used in this study provided findings, and subsequent recommendations that addressed each of the three research questions. The study posed multiple questions in order to examine the issue of accessibility from various perspectives. By using multiple methods the study better addressed the questions, and ultimately produced findings and recommendations that supported the intent of the study, which was to evaluate the accessibility of federal health care websites, and to understand the factors that may impede attaining accessible websites.

Analysis of the Study Approach and Conceptual Framework

Key elements of the study approach included the overall intent of the study, the conceptual framework, and the methodology of investigation. Each of these elements will be addressed in within this section, with the conceptual framework and methodology assessed more completely in subsequent sections of this chapter. Also presented in this section of the chapter are suggestions for an improved study approach for future research efforts. The problem this study addressed was to determine how well federal health care websites comply with Section 508 of the Rehabilitation Act of 1973 (29 U.S.C. § 794d.). The purpose of the research was to conduct an evaluation of the accessibility of federal health care websites for individuals with disabilities and to understand the factors that may impede attaining accessible websites. The goal of this research effort was to improve the accessibility of the federal websites that provide health care information. The review of literature revealed only one prior study that rigorously dealt with the issue of website accessibility, being the Jaeger (2006) study. Most other works that examined

261 accessibility were cursory examinations of web accessibility using automated testing. Such examinations would not provide the depth and breadth of scope that would more completely reveal the degree of compliance with Section 508 criteria, and would not yield adequate insight into the issues and problems that may impede attaining full website accessibility. The Jaeger (2006) study provided insight into how to construct a more comprehension compliance-based study of web accessibility, and significantly informed the design and construction of this research effort. To date, the Jaeger (2006) work was the most serious effort to study web accessibility and the only study to explore an appropriate conceptual framework. The Jaeger work provided an extensive discussion of the Technology Acceptance Model (TAM) and described modifying the basic TAM model for use in the study of web accessibility. This study adopted Jaeger‘s modified TAM model as the conceptual framework for this research effort. That modified TAM model provided a sound basis in helping to identify and understand the complexity of the factors that influence the adopting and implementation of web accessibility features. Based on the results of the data collection efforts and the subsequent analysis of that data, the study approach supported by the TAM conceptual framework proved to be appropriate and useful. For this study, the extent to which that modified TAM was useful was somewhat limited in comparison to the Jaeger (2006) study. The reason for this more limited extent relates to the fact that this study did not employ a webmaster questionnaire that solicited information regarding the ―Acceptance of the value of Section 508 compliance‖ (referring to Figure 1.4 Conceptual Framework diagram on page 36). The study approach had the intent of determining the accessibility of the studied websites by conducting a compliance-based examination. That intent was consistent with the research questions that addressed website compliance, impediments to compliance, and recommendations for improve accessibility. The findings contributed insight into the factors that affect the adoption of Section 508 standards, and indicated the accessibility of the studied websites. These findings validated key elements of the TAM conceptual framework. The methods provided information on the compliance errors and types of errors, which also provided input to the modified TAM framework. The overall study approach proved to be a successful combination of intent, conceptual framework, and methodologies, which in combination were appropriate for, and successful in, answering the research questions posed.

262 The following section presents a more comprehensive analysis of how the study findings contributed to the conceptual framework used for this research effort. The analysis focuses on how this study contributes to the factors identified in the conceptual framework, and discusses changes to the modified TAM model that would enhance its value as a conceptual framework for use in future web accessibility studies.

Assessment of the Conceptual Framework

The literature review revealed only one prior study that included an extensive discussion of a conceptual framework used in the examination of web accessibility. Jaeger‘s (2006) work provided significant insight into the selection of a conceptual framework for this study. Jaeger successfully used a modified version of the Technology Acceptance Model (TAM) as the basis for his work on web accessibility. His work further modified the TAM based framework, and this study adopted that resultant framework for use within this study of web accessibility. The TAM provides a framework for understanding how users adopt and use new technologies (Davis, 1989; Davis, Bagozzi, & Warshaw, 1989). The underlying assumption of the TAM is the fundamental belief ―that perceived ease of use and usefulness can predict attitudes toward technology‖ (Lederer, Maupin, Sena, & Zhuang, 2000, p. 269). The user‘s perception of the ease of use of a given technology, combined with the perceived utility of the technology in a given context, together create a user centric attitude about the technology. The resulting attitude then influences the user‘s behavior, which subsequently affects the user‘s decision to employ or not employ that technology for a specific application. As an overarching assessment of the conceptual framework, it is important to note the lack of research efforts that addressed the topic of conceptual frameworks when examining website accessibility. More study is necessary to assure that the conceptual framework is fundamentally sound and complete, and to understand the relationships among the component factors Figure 8.1 presents the modified TAM conceptual framework that resulted from Jaeger‘s (2006) study, and which guided this study. The following discussion sections address how the findings of this study influenced that conceptual framework. The goal is to assess how well the findings served to enhance the conceptual framework for the analysis of website accessibility.

263 More specifically the goal is to determine if the findings validate the elements and factors of the modified Technology Acceptance Model (TAM), and to determine if these findings reveal new factors that improve the framework and extend its application for web accessibility research. Each of the four research methods used in this study may contribute to refining the conceptual framework. This assessment addresses the value of the framework in conducting web accessibility studies similar to this research effort, and will identify possible refinements of the framework. In assessing the conceptual framework, the discussion examines the elements of Figure 8.1 using a top to bottom approach. That is, the discussion will begin by addressing the top level sections of ―Political Climate‖ and ―Pressure to Adopt Section 508 Guidelines,‖ and then move downward to address ―Education and Training,‖ and so forth. The discussion incorporates findings from the literature review and the study methodologies, when they provide support for the existing elements in the model or offer new insights.

Political Climate and Pressure to Adopt Section 508 Guidelines

The conceptual framework begins by indicating that technology acceptance has two key factors, being Law and Policy and Public Pressure. This study confirmed that these factors were part of technology acceptance. The findings of the literature and the side-by-side policy analysis did indicate that the existing key policies provided compelling motivation, or pressure, to adopt the Section 508 guidelines. The policy analysis findings indicated that the two policies examined in this study appropriately expressed the intent of promoting accessibility, with Section 508 providing rather specific criteria related to website accessibility. The policy findings led to recommendations that addressed how these two legislative acts could be more effective in delivering the pressure to adopt Section 508 guidance. Key recommendations included the need for clearer language within the acts, and proposed that the addition of stronger audit and enforcement provisions could significantly enhance the effectiveness in a pragmatic way. Additional recommendations addressed the need for awareness building and training at the organizational level and the technical level. These training efforts would increase awareness of the need for compliance, and would better inform Pressure to organizations regarding web accessibility issues. Providing such a channel of

264

- Conceptual Framework -

Modified Technology Acceptance Model

Political Climate Laws Public and Policy Pressure

Pressure to Adopt Section 508 Guidelines

Education and Training of Web Staff

Perceived Perceived Value Costs

Agency Acceptance of the Actual Mission Value of Section 508 Costs Compliance

Other Priorities Staff Time and Interests and Skills

Level of Implementation of Section 508 Criteria User Monitoring Feedback and Enforcement Amount of Types of For Whom Accessible Accessible Content is Content Content Accessible

Level of Compliance with Section 508

Figure 8.1: Conceptual framework diagram.

265 communication, may indicate that agencies are embracing of the intent of the federal policy. The automated testing and expert review findings further affirm the appropriateness of these two laws as factors for providing pressure. The findings indicated good accessibility of these studied websites, even though full compliance with the Section 508 criteria was not always attained.

Education and Training

The findings from the literature review did indicate that the model would benefit from expanding upon the conceptual framework factor of ―Education and Training of Web Staff.‖ The efforts to educate and build awareness of the need for accessible websites would likely extend beyond the web staff. The effort to educate and inform on the legal and ethical need for accessible websites should extend to include all management tiers within the organization, addressing the entire chain of command that extends from top-level agency leaders throughout each hierarchical level down to web development team. Such a comprehensive awareness building effort would help assure that the appropriate top-level attention is brought to the matter with the intent to instill an institution-wide commitment, supported by the top leadership. In this way, the organizational environment evolves to be more conducive to establishing and implementing the necessary policies and practices that would assure more accessible websites. This new environment would also serve to promote better informed decision making regarding setting priorities and providing resources to address accessibility issues. The expectation is that such education and awareness activities would result in an organizational culture change. The new culture would be more accepting of the accessibility guidelines, resulting in websites that are more accessible.

Acceptance of the Value of Section 508 Compliance

In the ―Acceptance of the Value of Section 508 Compliance,‖ the impact of training and awareness would be realized. Training and awareness activities could have significant effects at the individual employee level and that the organizational (cultural/managerial) level. The activities could dramatically change the state of the ―Perceived Value for Attaining Compliant

266 Websites.‖ Increased awareness could result in the realignment of organizational resources and priorities to focus more effort on attaining compliance. Educational efforts and exposure to the latest technology developments could assist web manager in producing websites that are more accessible. These educational activities could result in providing web managers with a more accurate understanding of the perceived and actual costs associated with producing compliant websites.

Level of Implementation and Level of Compliance

The conceptual framework accurately depicts what the findings revealed from the automated testing and the expert review. Agencies appeared to vary significantly in the degree of compliance with Section 508 guidelines. The findings revealed a range in the number and types of errors discovered. The findings indicated failure by organizations in implementing the Section 508 guidelines, resulting in varying amounts of accessible content, and several types of accessibility problems, which in combination affect for whom the content is accessible. These factors, each indicated in the conceptual framework, have an impact on the ―Level of Compliance,‖ which in the findings of this study varied significantly from one federal agency to another. The conceptual framework identified user feedback as a key component. The web content review noted that the web accessibility statement often provided the user with agency contact information in order to resolve accessibility related issues. This channel of communication could be an important venue for improving the accessibility of websites. The user has the necessary information to provide the web manager with information as to the usability and functionally of the website from the unique perspective of an individual with a disability. Such information could be critically important in improving the accessibility of the website and in increasing compliance with the Section 508 guidelines. The framework identifies monitoring and enforcement as another key factor in determining compliance. The findings from the expert review and automated testing indicated that monitoring is clearly lax or non-existent within some of the federal agencies. The indication from the findings was that the simplest of monitoring for accessibility would have identified accessibility problems prior to placing the website into production for public use. Enforcement

267 of the Section 508 guidelines also appears to be inadequate or missing. The policy analysis indicated the need for effective audit and enforcement measures, and noted the significant potential for such measures to improve compliance with Section 508 guidelines.

Summary of the Study Approach and Conceptual Framework

The modified TAM proved to be a conceptual framework that was appropriate and effective for understanding the relationship of technology acceptance to the resulting level of compliance with Section 508 guidelines. The findings of this study supported many of the individual factors identified within the conceptual framework. Moreover, the findings added insight and depth of understanding as to the nature of the factors. Neither this study nor the study by Jaeger (2006) had the primary intent of developing a detailed and extensive understanding of the conceptual framework. This study does provide substantial insight into the conceptual framework, but hesitates to modify further the initial framework used in this study. The findings provided by this study indicate new avenues of research for a more intensive examination of the conceptual framework. There is a great opportunity for future research to explore the factors and relationships that comprise this framework, in a more rigorous and systematic fashion. The need for such a framework is important to the study of web accessibility, as it would promote the ability to generalize about the process that constitute technology acceptance relating to web accessibility. The further development of the framework could lead to theory that is more general, and models that may better promote an understanding relating to achieving websites that are more accessible.

Assessment of Methodology

Design

The study approach employed multiple methods in order to examine the issue of accessibility more comprehensively, and to gain a fuller and richer understanding of the problems that may impede attaining fully accessible websites. The methodological approach began by analyzing the two key federal policies that attempt to assure website accessibility. This

268 side-by-side policy analysis informed about problems that may stem from poorly constructed legislative efforts that effectively inhibit webmasters from implementing accessible websites. The study‘s methodological approach then conducted a content review of the websites to gain insight into the accessibility from the users‘ point of view. The content review examined the web accessibility statement, as these statements can provide valuable assistance to the user of the website in matters related to accessibility. The study also employed automated testing and an expert review in order to understand the accessibility problems that relate to technical and usability issues. To a degree, Jaeger‘s (2006) work influenced the methodological design for this research effort. Jaeger used a multimethod approach to examine a specific category of federal websites, being those that offered electronic government goods and services. Jaeger‘s work included a law and policy analysis, expert testing, user testing, and a webmaster questionnaire. That methodological design was a systematic and methodologically comprehensive approach to examine a category of websites for accessibility issues. That study was instrumental in guiding this research study, which also sought to conduct a thorough multimethod examination of a category of websites. Specific research questions posed by this study influenced the choice of the methods used for this investigation. Practical issues and limitations on resources also shaped the selection of methods. In comparison to Jaeger‘s work, this study narrowed the policy analysis to the two legislative acts that serve to promote web accessibility. This afforded the opportunity to more rigorously excoriate those two key policies for any deficiencies, inconsistencies, or ambiguities that may impede attaining accessible websites. This study and Jaeger‘s (2006) work both employed expert review/testing to examine the websites for accessibility issues. The use of automated testing and expert reviewing supported the finding of accessibility problems that represented deviations from the intent of the legislative acts. By identifying these problems, the findings provided an indication of the degree of compliance. In this study, the findings indicated that full compliance with the intent of the legislative acts has not yet been achieved.

269 Alignment of Methods to the Study Purpose

The purpose of this study was to conduct an evaluation of the accessibility of federal health care websites for individuals with disabilities, and to understand the factors that may impede attaining accessible websites. The methodological design of this study addressed the stated purpose by examining the accessibility features provided within a selected set of websites, and by examining the key policies the attempt to promote accessible websites. In determining how well this group of websites complied with the current federal policy that promotes accessibility, the study used the accessibility guidelines put forth within Section 508 of the Rehabilitation Act of 1973. The nature of the research questions required that the research methodology utilize multiple methods to evaluate accessibility from various perspectives. To address the research questions and get a broader understanding of the accessibility, the study needed to address the federal policy environment, compliance with federal web accessibility guidelines, and the accessibility content and features provided on the websites. The side-by-side policy analysis provided findings that identified possible issues or barriers to federal agencies in the attempts to provide accessible websites. The web content review examined the websites for accessibility features that could be of value to users with disabilities. The automated testing and expert review method provided findings that supported the assessment as to how compliant these websites were with respect to the Section 508 standards. These two methods also provided insight as to the problems that impeded attaining accessible websites, and supported the identification of changes that could promote greater accessibility. The side-by-side policy analysis provided a great deal of insight into how the key legislative acts promoted web accessibility, and allowed the identified of issues or problems areas that could impede attaining accessible websites. This type of policy analysis was very effective at providing a solid understanding of the basic legal foundation that promotes website accessibility for federal agencies. The policy analysis method was critically important to identifying such shortcomings as unclear or undefined terminology, failure to provide adequate audit and enforcement provisions, and revealed how the two acts differed in approach. The different approaches led to serious confusion as to the exact scope of coverage of accessibility legislation, which the federal court system has yet to clarify.

270 The web content review also contributed a policy analysis approach, in that this review examined the websites to indicate the organizational policy on web accessibility. The web content review was an effective means for determining if these websites provided a web accessibility statement, and supported the determination as to whether that statement was of good quality, to be of significant value to users with disabilities. In addition, this content review methodology, using thematic content analysis, provided additional insight into what accessibility features existed on these websites. From that insight, the researcher gained important knowledge as to the structure and content elements that these web accessibility statements provided to users with disabilities. The automated testing method was included as it provided an example of how quickly and easily it is to test a website for accessibility problems. While this method is not a thorough examination for accessibility related issues, the method does provide an unobtrusive approach that yields an indication of accessibility, and is useful in examining websites for problems related to the Section 508 guidelines. Automated testing proved to be a useful, but simplistic, tool for easily determining accessibility related problems. In comparing the automated testing method to the expert review it is clear that the expert review provides a more complete and extensive indication of the website‘s accessibility. The automated testing method did successfully provide quantitative measures of the instance of non- compliance with the Section 508 guidelines. As such, the method was valuable for providing an indication of relative accessibility, by comparing the number and types of Section 508 errors. By identifying the type and number of errors, the analysis was better able to tailor recommendations that addressed the more commonly occurring errors. Correcting the most common errors may provide the biggest increase in the overall accessibility of the websites. However, a user based assessment would be necessary to more accurately determine this outcome. The expert review method provided a great deal more accessibility related information than the automated testing. The expert review provided quantitative and qualitative indications as to the accessibility of the websites. One significant benefit in using the expert review method was its addition of a human element or human centric focus. This human focus allowed for better identification of accessibility problems and provided greater insight and understanding as to the cause of the problem.

271 Future Methodological Considerations

Part of the expert testing method involved the use of a scoring sheet. The intent behind including this scoring sheet was to replicate work done in a prior accessibility study (Jaeger, 2006), and provide some degree of comparability between this study and that prior work. However, the worksheet and related study details had insufficient detail in that prior work to support direct comparability. This study made extensive efforts to define more clearly the testing criteria used and to detail more precisely the procedures employed. For future research efforts, users of this scoring sheet approach should make serious efforts towards clearly defining criteria and procedures, in order to promote comparability. Additionally, future researchers should carefully test any such criteria and procedures to assure validity and reliability, as by conducting pretesting, and supported by Krippendorff‘s Alpha and Cohen‘s Kappa statistics. The complexity of the findings revealed by this study indicates the value of the multiple method approach. Combining side-by-side policy analysis, web content review, automated testing and expert review provided a comprehensive evaluation of the accessibility of federal health care websites. The multiple method approach supported a much broader look at accessibility issues, and by doing so, the findings provided a more comprehensive understanding. While this combination of methods was appropriate for addressing the specific research questions posed within this study, this unique combination of methods may not be optimal configuration for all accessibility studies. Given the research questions posed within this study, this combination of methods did provide significant insight and improved understanding of the accessibility of this group of websites. The methodological approach used for this study proved to be sound and yielded informative findings. For future similar studies, the researcher offers some suggestions that could enhance the methodological approach. Much information could likely result from interviews with the federal agency webmasters and with the organizational leaders that have responsibility for the agency website. Such interviews could address the perceived impediments to attaining accessibility from both the managers and webmaster points of view. Their opinions could address impediments that stem from legislated policy, internal organizational guidelines, accessibility tools and resources, and training. In addition to identifying current problems, the webmaster could identify success stories in obtaining solutions that resolve or mitigate those

272 problems. Such findings could be very valuable as lessons-learned for other agencies that may be facing similar issues. The agency webmaster‘s perceptions of their website‘s accessibility could provide an interesting comparison to their actual accessibility as independently measured by an expert review. Additionally, an on-site inspection of the organization‘s policies and procedures that serve to ensure accessible websites could provide interesting findings that indicate policy related best practices for producing websites that are more accessible. It could be interesting to have webmasters review and provide comments on the finding of accessibility studies that examined their agency‘s website. Such commentary could provide valuable insight into the internal organizational issues, as well as providing a context for the findings.

Assessment of Study Assumptions and Limitations

In chapter 3 of this study, the researcher identified key assumptions and limitations inherent to this research effort. This section revisits those assumptions and limitations to understand their impact, and to assess the effectiveness of any steps taken to mitigate the effects of those the assumptions and limitations. Key assumptions included:  Using multiple methods of data collection can result in a composite picture that provides a more comprehensive understanding than any single evaluation strategy (Babbie, 2001; Miles & Huberman, 1994).  Minimal changes to websites occur during the study period.  The automated software testing correctly identifies Section 508 violations, and is a valid and reliable product. The researcher developed these assumptions prior to the start of the data collection efforts. As previously indicated, this study employed a multiple method approach that included a side-by-side policy analysis, web content review, automated testing, and an expert review. Earlier in this chapter, an assessment of these methods addressed their value for supporting the goals and objectives of the research effort. That assessment indicated the additional benefits obtained by using multiple methods in providing a more complete understanding of the accessibility of these websites. Based on that assessment, assumption 1 was correct. By using

273 multiple methods, the study provided a more complete composite picture of the accessible of this group of websites, than was possible by any single method. The second assumption presupposed minimal changes of the websites would occur during the study period. To address this concern, the researcher executed each method in a timely manner to minimize the impact of such changes. This mitigation step proved to be effective towards assuring that no, or minimal, changes took place during a given data collection effort for a given study. However, the researcher noticed that some changes occurred from method to method. The changes were not injurious to the data collection effort, nor did those changes affect the findings. In one case, a web page used in an early data collection effort was not available during the data collection for a later method. To resolve this issue, the researcher substituted a similar web page for the web page that was no longer available. That change in web page was not injurious to the data collection effort, nor did those changes impact the findings. Steps taken by the researcher to minimize changes and the impact of any changes were appropriate and effective. As such, assumption 2 is correct. Assumption 3 indicates the need for the study to use an automated testing software product that can identify Section 508 violations, and that the software product will produce valid and reliable data. To address this concern, the researcher extensively reviewed a number of software products, using selection criteria detailed in chapter 3. From that evaluation, the researcher selected an automated testing product that had a proven history of successful application in the testing of websites for violations of Section 508 criteria. Assumption 3 is correct, as the researcher was able to find and use such a proven software product. Along with the assumptions, the researcher identified several key limitations (see chapter 3). This study was exploratory in nature, and examined a relatively small number of websites. The small sample size does not provide a large enough basis to allow generalization across all federal agencies that provide health care information. Another key limitation was that the study did not examine the websites at two periods to analyze for changes in accessibility over time. Instead, this study provided a snapshot examination, in that the study provided a single look at the websites. An analysis conducted over an extended period would provide an opportunity to examine for trends the accessibility of these websites. One noted limitation was the inability to conduct user-based accessibility testing for this research effort. Studies indicate that user based accessibility testing involving persons with

274 disabilities is the best way to determine whether websites are accessible (Jaeger, 2006, 2003; Kanayama, 2003; Mueller, 2003; Witt & McDermott, 2004; Slatin & Rush, 2003; Nielsen, 2000). However, no facilities existed locally that would support the conducting of this type of testing. By including user based testing, the opportunity would exist to compare the user-based findings to those obtained from the expert review. Such a comparison could serve to indicate the accuracy of the two separate methods in identifying the set of accessibility problems. However, the lack of user based testing was likely not a severe limitation. Jaeger (2006) did conduct both user based testing and expert review, and concluded that both methods yielded similar findings. Further, the costs and resources needed to conduct such user based accessibility testing exceeded what was available for this research effort. Another limitation existed within the web content review method. The web content review searched the websites for the presence of agency policies relating to the implementation of accessibility features. The limitation is that federal agency policies as stated on the website may not reflect the agency‘s actual actions taken attain compliance with accessibility requirements. The researcher was cautious in drawing an inference regarding actual steps taken by federal agencies, based solely upon the statements made within the web accessibility pages found on agency websites. Based on the data analysis results presented in chapter 5, the study findings acknowledge and reflect this limitation.

Summary Recommendations

Prior sections of this dissertation (chapters 4-7) included discussions of findings, issues, and recommendations that evolved from the data analysis efforts related to the specific research method that the chapter addressed. This section organizes the key recommendations made in the preceding chapters by stakeholder group. The intent is to present and organize recommendations in a way that more clearly suggests a plan of action for the parties most significantly impacted by the findings of this study. The emphasis is on the presentation of recommendations and supporting activities that provide a practical means to improve the accessibility of the federal websites that provide health care information. The key stakeholder groups addressed in this

275 section are the federal policy makers; federal health care agencies that provide website content; agency website developers; and the users of those websites.

Recommendations for Federal Policy Makers

Federal policy related to web accessibility has the intent of assuring fully accessible websites, allowing all users to have fair and equal access. This study provided a better understanding of the actual state of accessibility of the federal health care websites and revealed findings that could help shape new policies that would better promote website accessibility. The findings from the automated testing and expert review indicated that web accessibility policies have fallen short of their intent. The side-by-side policy analysis indicated that there are some inconsistencies and ambiguities with the current national policies. The following presents several key recommendations that could serve to increase the effectiveness of the national policies that promote accessible websites. These recommendations address both long term policy related needs and shorter-term policy activities that would provide interim or transitional improvements in the policy environment. The recommendations include developing a new policy, providing training programs, and conducting ongoing research. New policy. As a longer-term objective, Congress should act to provide a new consolidated policy that addresses web accessibility. New legislative is necessary to resolve the conflicting and ambiguous language that exists within the two legislative acts that attempt to promote web accessibility. By developing a consolidated policy on web accessibility, the opportunity exists to address such current problems as to need to:  Clarify the scope of coverage, as in limited application to federal websites or general application to all websites.  Provide for a centralized authority, establishing an independent agency structured to oversee, and enforce compliance with national policy on web accessibility. The office would have full authority to review, audit, and enforce the statutory requirements related to web accessibility, along with supporting research aimed at improving web accessibility.

276  Resolve unclear concepts related to compliance waivers due to undue financial burdens in providing accessible websites. Current technological advances may likely indicate that this waiver clause is no longer relevant.  Assure that the statutory provisions and requirements are appropriate for the current technological environment, as well as providing coverage for emerging and emergent technologies. Formulating this new policy would be an involved and lengthy process but would be a major step towards improving the national policy environment related to web accessibility. Congress should task OMB to develop a government wide program of activities that builds awareness of the need for web accessibility. OMB would administer the oversight and enforcement of this mandatory training, as an addition to their existing responsibility for overseeing the government‘s investment in information technology (IT) resources. In this way, the OMB IT oversight role would extend to encompass the human resource aspect, towards ensuring the training/certification of web managers, and that the mandated training occurred in a timely fashion. OMB could leverage the existing infrastructure within the Access Board, to expand its current role in promoting compliance with Section 508 standards. The Access Board could administer this expanded responsibility, taking a more aggressive stance in conducting audits of web accessibility, and working to develop training and awareness building programs. These programs would include such activities as mandating training for all web managers, providing guidance on best practices, and building institutional awareness of the ethical and legal requirements. Training and awareness building. Awareness building and training activities for federal executives, technology managers, and website developers would appear to be a strong necessity, as some agencies‘ websites display an utter misunderstanding of accessibility law (Section 508) and what those laws require (Robb, 2001). Some federal executives stated that they were never clear about how or when they had to be in compliance (Tang, 2001). Such awareness building would address a deficiency that existed since the initial implementation of Section 508, as the federal sector did little to make web managers aware of the imposed accessibility requirements (Jaeger, 2004; Tang, 2001). This problem worsened as federal training centers for government information technology managers have not offered significant training in the legal

277 requirements and the technical requirements regarding web accessibility (Yukins, 2004; Tang, 2001). Conducting professional training courses is a successful method for modifying the institutional attitudes and behaviors of management (Fender, Prentice, Gorst, et al., 1999). Professional training has been determined to be a cost effective means to bring about institutional change, which in this case would be the increased adoption of Section 508 requirements (Mason, Freemantle, Nazareth, et al., 2001; Raker & Boyd, 2007). Congress should act to expand the current role of the Access Board, so that it may provide focused training on web accessibility. Training generates the greatest amount of desired residual change (over the longer term) with the least amount of time spent with the target professional (Balas, Boren, Brown, et al., 1996). Web managers would be required to complete a comprehensive training course that includes topics on the ethical requirement for accessibility, legal mandates, liability, and also provides information on best management practices. This training/certification program could provide comprehensive coverage on many of the more technical facets of web accessibility to include accessibility related resources; development and testing tools; and providing guidance on developing an integrated accessibility testing program. Research and additional guidance. The findings from this study revealed that federal agencies would benefit from additional high-level guidance that addresses management practices related to the creation of accessible websites. Congress should act to expand the Access Board‘s role in promoting accessibility, so that its mission would include conducting research and producing guidance that would assist federal agencies in addressing issues related to web accessibility. This guidance would be an adjunct resource supplementing the additional training developed for web managers and technical staff. As indicated by the findings within this web content review and the expert testing method from this study, this guidance would need to address such topics as the design and content of web accessibility statements, training requirements and awareness building, accessibility testing, and designing websites that provide accessibility features. The literature review indicated the value of providing web accessibility statements. The web content review found that the websites provided web accessibility statement that varied widely in quality. Further, these statements lacked a consistent presentation design and varied greatly in content.

278 The expert review revealed that many sites had an inconsistent representation for their navigation mechanisms. When navigation changes from pages to page, the user with a visual disability may need to spend an inordinate amount of time trying to find the navigational menus, and then need to decipher how to use that changed mechanism. The accessibility of such sites would improve significantly, if these sites used a standardized and consistent presentation of their navigation menus on every pages of the website. The expert review and the automated testing method provided findings that indicate significant errors in compliance with the standards provided within Section 508. The indication is that federal agencies are not testing for accessibility prior to making the website available for public use. A key recommendation was that agencies‘ management and web developers receive training that covers the appropriate use of accessibility. Improved testing programs would lead to websites that are more accessible and would assure compliance with Section 508. Top level federal policies and supplemental guidance is needed that serves to institutionalize accessibility testing as a key step in the development or modification of any federal website. Federal organizational leaders need to act to develop, document, and implement effective policies and procedures that will make accessibility testing a fundamental aspect of their web development programs. These recommendations for guidance training, awareness building, and policy related to institutional practices should spring from a solid foundation of research. The role of the Access Board could expand to coordinate and conduct fundamental research that will be the basis for the development of these training programs and agency best practices. Towards developing a set of standardized agency policies and practices that promote accessible websites, the Access Board could act as a consolidator of successful procedures and best practices that could be assembled into a composite instructional manual for use by all federal agencies. By developing a compilation of best practices, all agencies could share and use these lessons learned as guiding principles for each federal website development effort. Summary recommendations for federal policy makers. Several key recommendations made by this study relate to federal policy makers. Ensuring web accessibility will rely on making several significant changes that would rely on top-level government actions. The federal government should take a more aggressive stance in auditing and enforcing compliance with the guidelines set forth in Section 508. The key recommendations for policy makers include long-

279 term actions and near term activities that could greatly improve the accessibility of federal health care websites. These recommendations call for a review of existing policy in light of the technological environment that now exists, with the goal of developing a single consolidated federal accessibility policy. In the near term, several key activities could boost compliance with Section 508 and improve the accessibility of federal websites. These activities include expanding the role of the Access Board to oversee and more aggressively pursue the audit and enforcement of the current law, as it relates to providing accessible websites that are compliant with the standards provided by Section 508. In addition, the Access Board would assume new and expanded responsibilities that include conducting research that addresses training and awareness for both federal agencies‘ leadership and for website developers. The new training courses would supplement the development of guidance for managers and developers that covers best practices and technological tools, as well as identifying resources that could assist and inform about the website accessibility. These longer-term and nearer-term recommendations address activities that could provide interim or transitional improvements in the policy environment.

Recommendations for Federal Health Care Agencies

Federal health care agencies should initiate a management review of their policies, procedures, and controls that address website accessibility. The automated testing and expert review indicated that in some cases even relatively simple efforts to conduct accessibility testing could have caught errors prior to the publication of the web page. The findings indicated that agencies may not have policies and procedures that address web accessibility, or that those policies are ignored. Agencies must determine whether current policies and controls are adequate to ensure accessible websites or, if no internal policies exist, these agencies should take action to develop and implement policies and procedures that properly promote compliance with the legal requirements as stipulated in Section 508. If the management review indicates that current policies are appropriate, then a further review is necessary to assure that management controls exist, so that the web development staff follows those policies and procedures. In support of that first step of conducting a thorough management review, the agency would need to conduct training and awareness building activities that would be similar to those

280 discussed in the prior section ―Recommendations for Federal Policy Makers.‖ Agencies should assure that their staff is aware of the legal and ethical needs for providing fully accessible websites. These training related activities may also require that the agency initiate a review of the current web development tools and resources that are now available in the marketplace. Many of the newer web development tools have built-in functionality that promotes more accessible website by providing appropriate prompts to the programmer. As an example, when the web programmer adds an image to the web page, the software may provide a prompt the programmer to add descriptive text, making the image accessible to users with impaired vision. Another step that agencies should take towards assuring more accessible websites would be to conduct a review of their website to ensure that the site provides a web accessibility statement and that the statement is of high quality. Within this study, the web content review provided findings that indicated a wide range of content and quality across the accessibility statements. While Section 508 does not mandate these statements, web accessibility experts indicate the value of such statements to the users with disabilities, making these statements an important feature in improving the overall accessibility of the website. Agencies should update their organizational policies and procedures to ensure the business practice of providing a web accessibility statement.

Recommendations for Web Management Staff

The web development staff must seek out opportunities for training related to web accessibility. Training must be comprehensive so that it addresses the legal and ethical requirements for compliance with Section 508 standards; addresses the technical means to build- in accessible features during web development, and should cover the tools and resources that could assist in the development of accessible websites. The training should extend beyond just assuring compliance with the minimum accessibility standards put forth by Section 508. The training should extend to cover many features that serve to enhance the usability and accessibility of the site by individuals with disabilities. Several findings within this study point to recommendations that could greatly enhance the accessibility of websites. Many of these practices are technical in nature and would be best

281 addressed by the web development team within the federal agencies. Some of the key technical recommendations include:  Provide a high quality web accessibility statement within the website.  Standardize the location of a visibly clear link to the accessibility statement across all web pages.  Assure that all content is clearly presented and readily understandable.  Standardize navigational and search mechanisms across all web pages.  Provide contact information for additional help or feedback with accessibility issues.  Adopt a testing program that comprehensively examines for accessibility issues. These recommendations for improved practices address several of the key issues revealed in this study. Each of these recommendations would serve to increase accessibility and promote the role of the web manager as a responsible party for assuring more accessible websites. To meet that responsibility, it is important that the web staff have the requisite expertise that will help assure more accessible websites.

Recommendations for Website Users

Users of federal health care websites can play an important role in promoting more accessible websites. A prior recommendation addressed the need for websites to include a web accessibility statement, which would include a means for the user to provide feedback and to get help on issues related to the accessibility of the website. This means of communication provides the user with a channel directly to the web manager, whereby the users can offer suggests and comments as to the accessibility of the website. While the web developers may have the necessary skills, tools, and desire to provide accessible feedback, the true accessibility of a website can be most accurately determined by actual users with disabilities. The user has the opportunity to provide the web developer with the most valid and reliable testing of the website for issues related to accessibility. User feedback to the web developer is one important way for users to contribute to improving web accessibility. Another key contribution can be through informed advocacy. Users should seek out the key policies that attempt to ensure accessibility. By staying up to date on current policies, the informed users can assess the effectiveness of these policies and

282 determine if new provisions are required. Users can make policy makers aware of what additional provisions are appropriate to meet their accessibility needs in an every changing technological environment. Remaining informed extends beyond a basic knowledge of accessibility policy. Users with disabilities should stay abreast of current developments in accessibility resources, assistive tools, and emergent assistive technologies. By staying informed and being an active advocate for accessibility, the user can play a critically important role in improving the accessibility of websites for all users.

Recommendations Summary

The recommendations presented in the preceding sections addressed the issues presented in this chapter and from the literature review (chapter 2). From the study findings and the analysis of those findings, the four key stakeholders (policy makers, federal organizations, web developers, and users) can play important roles in improving web accessibility by implementing the recommendations identified within this study. The implementation of these recommendations will help improve the accessibility of federal health care websites.

Future Research

The literature review and the analysis of the data collected from this study indicate a number of opportunities for future research. These opportunities include research that would offer a better understanding of the accessibility of websites and offer insight into the factors which impede attaining fully accessible websites. These suggestions for future research also address other areas of interest related to accessibility, as methodological issues and developing the conceptual framework. The following presents a categorized listing of pertinent research questions that would help extend the body of knowledge related to website accessibility.

283 Study Approach

 What accessibility problems would users with disabilities find? User based research findings may produce differing results than those obtained by the expert review method. Do these methods offer comparable results, or does user testing provide greater insight into the cause, impact, and resolution of the problems found? The indication is that a great deal of potential value may result from including user based testing in future similar studies.  From the web manager‘s perspective, what are the key impediments to providing accessible websites? What is the relative importance of these impediments? The indication is that web managers and the organizational culture may be a key component is the resulting accessibility of an entities‘ website.  What are the best practices for mitigating those factors that impede web accessibility? This type of information would be very valuable to organizations that face similar problems, saving them time and resources in knowing what has worked for others.

Conceptual Framework

 Very basic conceptual modeling questions could address the appropriateness of the TAM framework for this type of analysis. Is there a better framework for guiding compliance based web accessibility research?  What additional factors should be included in the TAM conceptual framework (see Figure 1.4), and what are the relationships among these factors? The literature review found few studies that addressed the use of the modified TAM as a conceptual framework for the study of website accessibility. The literature review revealed no study that focused intensely on developing a conceptual framework, and then exploring the topic systematically and extensively.  How do organizations decide upon their level of compliance with the requirements of Section 508? What are the factors and processes involved? The TAM conceptual framework is virtually unexplored in the area of ―Acceptance of the value of Section 508 compliance.‖

284  Looking not solely at Section 508 accessibility but at accessibility requirements more broadly, how does the TAM conceptual framework differ across types of government entities (e.g., federal, state, local) or across organizational types (e.g. government, corporate, non-profit)? This type of comparison could even be extended to examine how the conceptual framework differs across various national governments.  What unique factors may surround the situation where federal agencies contract externally for website development and maintenance? There may be unique agency institutional and contractor organizational issues at play that affect the adoption of web accessibility standards.

Methodological Issues

 What would improve the instrument, criteria, and procedures used to ―score‖ websites, as performed in this study‘s expert review (chapter 6)? This study attempted to perform a scoring effort similar to work done in a prior study (Jaeger, 2006), in order provide a degree of comparability. However, comparability may not be appropriate as it was unclear as to the specific criteria used by Jaeger. This study took extensive efforts to better define the criteria and procedure used, so that future work could provide results that are more comparable. Much work remains in this area to develop a scoring instrument, criteria and procedures that will support directly comparable results.  What research methods would be best suited to collecting the various types of information required to refine the TAM conceptual framework? The initial indication is that surveys could provide much high quality information that would address a number of the known factors. However, obtaining high response rates for survey participation is difficult to achieve. What approaches or techniques could improve response rates and improve data quality?

Stakeholder Questions

 Policy Makers:

285 o What policy changes do users with disabilities feel are appropriate for promoting website accessibility? A partial update of Section 508 is underway, and that effort by the Access Board would benefit from research that gathered user based opinions for improvements to the accessibility standards that promote accessible websites. A survey of users‘ needs could indicate broader changes to accessibility policy. o What are other nations doing to promote accessible websites? A survey of international policies on web accessibility accompanied by an assessment of the level of accessibility of that government‘s websites could provide indications of more effective policies. The opportunity exists to learn from the successes attained by other nations‘ efforts to promote accessible websites.  Federal Agencies o What internal policies and procedures do federal agencies have and how effective are those practices in producing accessible websites? A survey of various agencies could allow the discerning of those management policies, controls and procedures that are more effective. o How should organizations integrate the various accessibility testing methods into the website life cycle, from design to development and through the operational period?  Web Developers o What are the best practices followed by web mangers in developing accessible websites? There may be specific tools, resources and technical processes that are more effective or more efficient in creating accessible websites. o What are the major impediments to providing accessible websites, and how have developers addressed or mitigated those problems? A sharing of success stories or best practices could be of great value to those agencies planning a new website.  Users o What constitutes a good web accessibility statement? The design and content of the web accessibility statement should be grounded on user based needs. o How can users be encouraged to provide feedback to federal organizations about website accessibility issues? Soliciting the user‘s experience would provide

286 valuable information as to accessibility problems and successes, as well as providing insight into the severity or cause of the accessibility problem. While the findings of this study indicated a great number of additional opportunities to shape future research, the above issues appear to be the most compelling and may provide the greatest pragmatic value toward improving web accessibility. Within this set of questions, those that deal with best technical practices, success stories, testing methodologies, and institutional policies and practices may offer the greatest practical return towards improving the accessibility of websites.

Summary Conclusion

On August 7, 1998, Congress amended Section 508 of the Rehabilitation Act (29 U.S.C. § 794d.) to expand the federal government's responsibility to provide electronic and information technology which is accessible to, and usable by, individuals with disabilities. Section 508 requires federal agencies to ensure that the electronic and information technology is accessible. Section 508 requires that individuals with disabilities seeking information or services from a federal department or agency, have access to and use of information and data comparable to that provided to individuals without disabilities. That requirement extends to apply to federal websites that provide health care information. This study examined a select group of such website and found that a significant portion of those sites did not comply with the standards of compliance found within Section 508. Moreover, a broader examination of those sites found numerous accessibility related problems that would effectively deny certain individuals with access to the information and services provided by those websites. The intent of the national policies that promote website accessibility is quite clear in stating the goal of assuring fair and equal access for individual with disabilities. The fact that agencies have fallen short in providing fully accessible websites is also clear. There is a breakdown within these federal health care agencies, as they do not have internal policies and procedures in place that would ensure compliance with the federal mandate for accessibility. At a minimum, these agencies fail to conduct appropriate testing that would ensure the identification and correction of these issues.

287 This study revealed that compliance with Section 508 is uneven across the federal agencies, and the accessibility problems identified were of a varied nature. A key conclusion of this study is that several avenues of opportunity exist that would address this failure to ensure accessible websites. This study identified several key issues may that contribute to this failure to provide accessible websites. Recommendations for addressing these problems include:  Developing new federal policy that consolidates and extends existing policy,  Creating a central federal accessibility office,  Conducting management and web developer training programs, and  Providing ongoing research to study, monitor, and assess accessibility issues. In combination, these recommendations provide a broad-spectrum approach that attacks the problems from multiple angles. This multiple front approach would be more effective that any single campaign. The social impact of inaccessible websites can be quite serious. Inaccessible websites effectively deny a large group of individuals their full and complete access to government information, products, and services. This nation has made great strides in striking down discriminatory practices that deny fair and equal access to government services, finding such practices to be ethically unacceptable and in direct conflict with the intent of the national goal of equal treatment under the law. Failing to make accessibility a reality could result in the disenfranchising of a major segment of society. Ethical concerns and a sense of social responsibility for any under-served group compel action to ensure compliance with the intent of the law. It is time for the federal government to implement key reforms that will promote more accessible websites. These reforms must reshape the national policies on web accessibility and act to implement culture changing programs that will better promote web accessibility.

288 APPENDIX A WEB ACCESSIBILITY CRITERIA FROM SECTION 508

The following is an extract from www.section508.gov, the federal government website that provides information about Section 508 issues. This extract addresses the web accessibility criteria provided within Section 508. The entire text of Section 508 can be found at

§ 1194.22 Web-based intranet and internet information and applications. (a) A text equivalent for every non-text element shall be provided (e.g., via "alt", "longdesc", or in element content). (b) Equivalent alternatives for any multimedia presentation shall be synchronized with the presentation. (c) Web pages shall be designed so that all information conveyed with color is also available without color, for example from context or markup. (d) Documents shall be organized so they are readable without requiring an associated style sheet. (e) Redundant text links shall be provided for each active region of a server-side image map. (f) Client-side image maps shall be provided instead of server-side image maps except where the regions cannot be defined with an available geometric shape. (g) Row and column headers shall be identified for data tables. (h) Markup shall be used to associate data cells and header cells for data tables that have two or more logical levels of row or column headers. (i) Frames shall be titled with text that facilitates frame identification and navigation. (j) Pages shall be designed to avoid causing the screen to flicker with a frequency greater than 2 Hz and lower than 55 Hz.

289 (k) A text-only page, with equivalent information or functionality, shall be provided to make a website comply with the provisions of this part, when compliance cannot be accomplished in any other way. The content of the text-only page shall be updated whenever the primary page changes. (l) When pages utilize scripting languages to display content, or to create interface elements, the information provided by the script shall be identified with functional text that can be read by assistive technology. (m) When a web page requires that an applet, plug-in or other application be present on the client system to interpret page content, the page must provide a link to a plug-in or applet that complies with §1194.21(a) through (l). (n) When electronic forms are designed to be completed on-line, the form shall allow people using assistive technology to access the information, field elements, and functionality required for completion and submission of the form, including all directions and cues. (o) A method shall be provided that permits users to skip repetitive navigation links. (p) When a timed response is required, the user shall be alerted and given sufficient time to indicate more time is required.

Note to §1194.22:

1. The Board interprets paragraphs (a) through (k) of this section as consistent with the following priority 1 Checkpoints of the Web Content Accessibility Guidelines 1.0 (WCAG 1.0) (May 5, 1999) published by the Web Accessibility Initiative of the World Wide Web Consortium:

2. Paragraphs (l), (m), (n), (o), and (p) of this section are different from WCAG 1.0. web pages that conform to WCAG 1.0, level A (i.e., all priority 1 checkpoints) must also meet paragraphs (l), (m), (n), (o), and (p) of this section to comply with this section. WCAG 1.0 is available athttp://www.w3.org/TR/1999/WAI-WEBCONTENT-19990505.

290 Section 1194.22 Paragraph WCAG 1.0 Checkpoint (a) 1.1 (b) 1.4 (c) 2.1 (d) 6.1 (e) 1.2 (f) 9.1 (g) 5.1 (h) 5.2 (i) 12.1 (j) 7.1 (k) 11.4

291 APPENDIX B

AUTOMATED ACCESSIBILITY TESTING PRODUCTS

The following is an extract from the JimThatcher.COM website (Thatcher, 2009), that identifies some of the more commonly used accessibility testing software. The tools listed here include a brief description. 1. InFocus from SSB Technologies. $1795. InFocus™ InFocus Desktop was the first commercial web accessibility software and remains the market leader, with over 115 accessibility tests encompassing all major accessibility standards and a high level of automation available, InFocus 2. The LIFT Machine from UsableNet $999. LIFT Machine is a server-based application that automatically scans internal and external websites for over 140 quality, accessibility, and usability issues. It then generates a variety of web-based reports for both executives and individual content creators. 3. Ramp Ascend from Deque $1499.. Ramp Ascend includes full capabilities for adding SMIL captioning to multi-media, ensures web animations are safe, and provides comprehensive table remediation to even the most complex, n-dimensional tables. Includes plug-ins for Macromedia Dreamweaver, Microsoft FrontPage, and Mercury Interactive TestDirector 8. The product without repair functions is Ramp Grade, $269, and without spidering, Ramp Personal Edition, $69. Deque also an Enterprise product. 4. WebKing from Parasoft (Contact [email protected] for prices.) WebKing allows users to record critical user click paths by following them in a browser, then it automatically configures and executes functional/regression tests that verify paths and page contents while ignoring insignificant differences. webKing‘s static analysis identifies client-side code that does not comply with Section 508 accessibility rules, and pages with broken links, XML problems, and spelling errors. 5. WebXM from Watchfire (Contact [email protected] for prices.) Watchfire provides software and services to identify, measure and prioritize accessibility, and compliance risks that exist on corporate web properties. The Web Accessibility Initiative Evaluation and Repair Tools working group has over 90 software tools listed in three categories: Evaluation, Repair and Transformation (W3C, 2009b).

292 APPENDIX C

RESOURCES ON ACCESSIBILITY

General Information on Accessibility

 WWW.SECTION508.GOV

 World Wide web Consortium (W3C), at http://www.w3.org/WAI/

 WebAIM (Accessibility In Mind), at http://www.webaim.org/

 The International Center for Disability Resources on the Internet. Available at http://www.icdri.org/  ACM SIGACCESS (Special Interest Group on Accessible Computing). Available at http://www.hcibib.org/accessibility

Designing Websites For Accessibility

 University of Washington, at http://www.washington.edu/doit/Resources/web-

design.html

 disABILITY Information and Resources, at http://www.makoa.org/web-design.htm

 University of Wisconsin System, at http://trace.wisc.edu/world/web/

Conducting In-Depth Accessibility Testing

Jaeger, P. T. (2006). Multi-method evaluation of U.S. federal electronic government websites in terms of accessibility for persons with disabilities. (Doctoral dissertation, Florida State University, 2006). Retrieved on September 17, 2007, from the Florida State University Electronic Thesis and Dissertations website: http://etd.lib.fsu.edu.proxy.lib.fsu.edu/ theses/available/etd-03062006-120905

293 APPENDIX D

WEB CONTENT REVIEW PROTOCOL

1. Using a PC/Internet connection access the agency website from the www.health.gov portal. 2. Confirm this is the home page for this website. If not, examine for home page by executing the following scan page protocol for this site in order to find the home page: 1. Beginning in upper left, scan from left to right across the top of the page. 2. Return to upper left corner and scan the left side of the page to the bottom. 3. From bottom left corner, scan from left to right across bottom of page 4. Proceed to top right corner and scan from top to bottom. 5. Proceed to upper left and scan the remainder of the web page (center area) from top to bottom. If this scan does not identify the home page Universal Resource Locator (URL) or active link, then repeat the scan examining more closely each hot link that may provide information that identifies or contains and active link to the home page. If this second scan fails to identify the home page then perform a Google search to find the home page. If these three scan/searches fail to produce a home page then discontinue the search and indicate that the search for the site was without success. Note: The Scan page protocol for a web page is the protocol used throughout this study when a web page must be searched for any specified term(s).

294 APPENDIX E

EXAMPLE OF AN AUTOMATED TESTING SOFTWARE REPORT

The following is an example of an output report received from an automated testing software program. The program used to generate this report was CynthiaSays (HiSoftware, 2010), and the website examined was the www.health.gov homepage.

295 t-'"'"'' ＮＺＮＺ Ｇ Ｂ ＺＺＺ ＢＧ ＺＺＮＧ［ＺＺＺ ＢＢ ＺＺＺ Ｇ ［ＺＺ ｕＢ ＺＺＺ Ｂｾ ＺＺＮＺ ＧｙＺＺ｣ＮＺＺ ｗ ｣ＮＮＮＮＮＮＮＺ Ｇ ＮＮＡ ｬ ＮＺＺＺＧＢＢＧ］ Ｇ Ｇ ＭＭＭＭＮＺＱ＠ D ｃｹｮｴｨ ｩ ｾ ｓｉｾ＠ Report .. C f:1 httpjjwww.contentquality.com/mynewtester/cynthia.exe Cj Customizt links CJ ｂｬ｡｣ｫｾ＼､ ｃ ｆｓｕｻ＠ ... j_ ThisAmern:anlift · .. -tt Googlt • ｅｶｯｮｹ Ｍ ｾｴｦｲｈＺｗ＠ .•. ｾ＠ lokats 'n' FunnyPic... CJ Bob and Tom * FSUEmail o No EM8ED elements found in document body.

lc 508 Standa rds Sestjon 1194 ?2 lrl Web pi!oes ｾｕ＠ be desioned so tNt <1l inform<1tion conveyed with colo!" is a lso avail3ble without colo!" fOf" ･ｾｭｶ ｬ ･＠ from context or m3r1

o Not": ｏｯ｣ｵｭ ･ｮ ｴ ｵ ＬＮＮＬ ｳＮＬ ｸｴ ･ ｭ ｡ ｬ Ｎｴｹ ｬ ＼Ａ ｾ ｨ･ ･ｴｳ Ｌ ｩ ｮ ｬ ｩ ｮ･ ｳｴ ｹ ｬ ･ ｩ ｮｦｯｲｭＦ ｴ ｩ ｯｮ Ｌ ｯ ｲｴＮＮＮ｡ ､ ＼Ａ ｲ ｳｴｹ ｬ ･ ｩ ｮ ｦｯｲｭＦｴ ｩ ｯｮ Ｎ＠

Redundant text links shanbeprovided for eadlactivere

dient-side image maps shan be provided instead of server-side image maps except where the re!,Jions cannot be defined with an available geometric shape. o Rule : <;. 1.1 - No !MG ele mentshouldcontainthe 'is map' a ttribute . o No server-side ima ge ma p IMG e le me nts fO

o Rule : 5.1.1 - Ide ntifyaii DataTABLE e le me nts. o Note: Data TABLE Ele me nt found et Line: 1<;1, Column : 1<;.

Markup shan be used t o associat e dat a cefl s and header cens for data t ables that have two Of more looical levels of row or column headers. o Rule : 5.2. 1 - Ide ntify a ll Data TABlE ele me nts. o Note : Data TABLE Element found et Line: 191, Column : 19.

Frames shan be titled with t ext that facilitates frame identification and navigation. o Document is not a FRAMESET Page.

Pages shan be designed to avoid causing the saeen to llid::er with a frequency greater than 2 Hz and lower than 55 Hz. o Rule : 7.1.1 -Docume nts a re require d not toconta in t he BLINK e leme nt. o NoBLINKelements fo.und in docume ntbody. o Rule : 7. 1.2- Docume nts a re required not to conta in the MARQUEE element. o No MARQUEE e leme nts found in document body.

toaeateint the saipt sha

o Rule : 6.3.1 - Anchor e le me nts are re<1uired not to use ja vascript for the link taroet when the NOSCRJPT ele me nt is not oeckpoint if t he NOSCRIPT e le me nt is found, ｨｯ ｷ･ ｶ･ ｲ Ｌｴ ｨ･ｹ ｷｩ ｬｬ ｾ ｩ ､･ ｮｴ ｩ ｦ ｩ ･､ Ｎ＠ o No AREA Ele me nts found in docume nt body. o Rule : 6.3. 3 - locllte ele mentsthatuse HTML eventhandle l"$. o ｎｯｴ･ Ｚｔｨ ｩ ｳｲｵ ｬ ･ ｨ｡ ｳ ｮｯｴ ｾ ･ｮ ｳ･ ｬ ･ ｣ｴ･ ､ ｴｯ ｢･ ｶ･ ｲｩｦｩ ･ ､ｦ ｯｲ ｴ ｨ ｩ ｳ ｣ｨ･ ｣ｫｰｯ ｩ ｮｴ Ｎ＠ o Rule : 6.3.4 - When SCRIPT Ele me nts a re used, the NOSCRIPT ele ment is required in the docume nt. o No SC RIPTele me nts found in docume nt.

Whenaw thatan applet, pluo-in or be present on t systemto intefl)fetpage content, ltlrouQtl (1).

Y ahoo! Search

Cj ｃｵｳｴｯｭｯｺｾ ｬ ｯｮｫｳ＠ C) Blackboa

a navigation.

"a•-<• ..... ,. •l•m•"" · "'• ｭ ｦ ｯｾ ｴｩ ｯ ｯ＠ ,,. ,., , , "'• ''""' •h•l b• <•otifi•d " "' fuoctioo• l ,.,. "'•' ｾＰ＠ b• , • • , , , ｾ ﾷﾷ＠ ｾ＠

o Rule : 6.3.1 - An.cho. Ｂ ｬ ･ ｭ ･ Ｎ ｾ Ｎ＠ ui

When a web page requires that an applet, pluo-«"l or other application be present on the dient system t o intefl)l"et page content, the page must provide a link. t o a pJuo-«"l or applet that ompijes with §1194.21(a) tnrouQtl (1). o Rule : 6.3.5 ·All OBJECT e lements a re required to contein e le me nt conte nt. o No OBJECT elements found in document body. o Rule : 6.3.6 - All APPLET e lements are re<1uired to conta in ｾ ｨ＠ ele me nt conte nt a nd the alt attribute. o NoAPPLETele me nts found in document body . o Rule : 6.3. 7 - When EMBED Ele me nts are used , the NOEMBED eleme nt is required in t he docume nt. o No EMBED e le me ntsfO

ｃｬ＾ﾫＺｫｰｯｩＡＡｴｒ ＭＮＮｉｴ ｾ Ｚ＠ 1'- • ｐ ｵ ＮＮＮｊａ＾ｉ ｾ ｴ Ｎ､ ｖｍｾ ｴｩ＼＾ｮ Ｌ＠ No • ｆ ｾ［Ｏ Ｎ､ ａ ｵｴＮＮＮＮＮＮＮｬﾫｦ ｖｍｾ ｴｩ＼＾＼ｯ Ｌ＠ W...-ning • ｆｾ ｕ ﾫ＼ａ＾ｩ ｾｉﾫ＼ｖｍｾｴｩ＼＾ｮ Ｌ ｬｯ＼ｷｩＮＬＮＬＭ Ｌ ｃｍ ｾＢＢＧ Ｇ ＢＧ Ｎｯｯ ＮＮＭ Ｍｴ ｯｦ ｾ ｩ ｬ ＨＭｴｹ ｺ ＮＬＮｊｯｮｬｹＩ Ｌ＠ NfV • ｎｯｴ ＭＧ ﾫｴＮ､ｦｵｴＢ ｾ ｍ ｾ ｴｩ＼＾＼ｯＬ＠ Nf A • ｎｯ ＮＮｩ ｾ ｬﾫｦＮｴｈｐｾｍ＾ ｾ ＮＮＮＮＮＬＮＧＭＧ＼ｊ ｩ ｮｾ ｴ Ｈｖｩ ＼ａｕ ｉ＠ only), No ｖ ｾ ｬ ｵＮ＠ • Visu.l a..dpoinl

Report generated by the HiSoftware Company Cynt hia Agent. Po wered by the AccMonitor Compliance Server HiSoftware, Cynthia Says, AccMonit or Compliance Server, Cynthia Agent are all t rademarks of HiSoftware Inc. (WWW hisoftware com 603.578.1870 or 1.888.272.2484)

ｾ ＼ ＱＰｻＱＳａｍ＠

296 APPENDIX F

TEXT OF SECTION 508 OF THE REHABILITATION ACT OF 1973

The following is an extract from www.section508.gov; the federal government website that provides information about Section 508 issues. This extract is available at

Section 508 Standards  Subpart A -- General o 1194.1 Purpose. o 1194.2 Application. o 1194.3 General exceptions. o 1194.4 Definitions. o 1194.5 Equivalent facilitation.  Subpart B -- Technical Standards o 1194.21 Software applications and operating systems. o 1194.22 web-based intranet and internet information and applications. 16 rules. o 1194.23 Telecommunications products. o 1194.24 Video and multimedia products. o 1194.25 Self contained, closed products. o 1194.26 Desktop and portable computers.  Subpart C -- Functional Performance Criteria o 1194.31 Functional performance criteria.  Subpart D -- Information, Documentation, and Support o 1194.41 Information, documentation, and support.  Figures to Part 1194 Authority: 29 U.S.C. 794d.

297 Subpart A – General

§ 1194.1 Purpose. The purpose of this part is to implement section 508 of the Rehabilitation Act of 1973, as amended (29 U.S.C. 794d). Section 508 requires that when Federal agencies develop, procure, maintain, or use electronic and information technology, Federal employees with disabilities have access to and use of information and data that is comparable to the access and use by Federal employees who are not individuals with disabilities, unless an undue burden would be imposed on the agency. Section 508 also requires that individuals with disabilities, who are members of the public seeking information or services from a Federal agency, have access to and use of information and data that is comparable to that provided to the public who are not individuals with disabilities, unless an undue burden would be imposed on the agency.

§ 1194.2 Application. (a) Products covered by this part shall comply with all applicable provisions of this part. When developing, procuring, maintaining, or using electronic and information technology, each agency shall ensure that the products comply with the applicable provisions of this part, unless an undue burden would be imposed on the agency. (1) When compliance with the provisions of this part imposes an undue burden, agencies shall provide individuals with disabilities with the information and data involved by an alternative means of access that allows the individual to use the information and data. (2) When procuring a product, if an agency determines that compliance with any provision of this part imposes an undue burden, the documentation by the agency supporting the procurement shall explain why, and to what extent, compliance with each such provision creates an undue burden. (b) When procuring a product, each agency shall procure products which comply with the provisions in this part when such products are available in the commercial marketplace or when such products are developed in response to a Government solicitation. Agencies cannot claim a product as a whole is not commercially available because no product in the marketplace meets all the standards. If products are commercially available that meet some but not all of the standards, the agency must procure the product that best meets the standards.

298 (c) Except as provided by §1194.3(b), this part applies to electronic and information technology developed, procured, maintained, or used by agencies directly or used by a contractor under a contract with an agency which requires the use of such product, or requires the use, to a significant extent, of such product in the performance of a service or the furnishing of a product.

§ 1194.3 General exceptions. (a) This part does not apply to any electronic and information technology operated by agencies, the function, operation, or use of which involves intelligence activities, cryptologic activities related to national security, command and control of military forces, equipment that is an integral part of a weapon or weapons system, or systems which are critical to the direct fulfillment of military or intelligence missions. Systems which are critical to the direct fulfillment of military or intelligence missions do not include a system that is to be used for routine administrative and business applications (including payroll, finance, logistics, and personnel management applications). (b) This part does not apply to electronic and information technology that is acquired by a contractor incidental to a contract. (c) Except as required to comply with the provisions in this part, this part does not require the installation of specific accessibility-related software or the attachment of an assistive technology device at a workstation of a Federal employee who is not an individual with a disability. (d) When agencies provide access to the public to information or data through electronic and information technology, agencies are not required to make products owned by the agency available for access and use by individuals with disabilities at a location other than that where the electronic and information technology is provided to the public, or to purchase products for access and use by individuals with disabilities at a location other than that where the electronic and information technology is provided to the public. (e) This part shall not be construed to require a fundamental alteration in the nature of a product or its components. (f) Products located in spaces frequented only by service personnel for maintenance, repair, or occasional monitoring of equipment are not required to comply with this part.

299 § 1194.4 Definitions. The following definitions apply to this part: Agency. Any Federal department or agency, including the United States Postal Service. Alternate formats. Alternate formats usable by people with disabilities may include, but are not limited to, Braille, ASCII text, large print, recorded audio, and electronic formats that comply with this part. Alternate methods. Different means of providing information, including product documentation, to people with disabilities. Alternate methods may include, but are not limited to, voice, fax, relay service, TTY, Internet posting, captioning, text-to-speech synthesis, and audio description. Assistive technology. Any item, piece of equipment, or system, whether acquired commercially, modified, or customized, that is commonly used to increase, maintain, or improve functional capabilities of individuals with disabilities. Electronic and information technology. Includes information technology and any equipment or interconnected system or subsystem of equipment, that is used in the creation, conversion, or duplication of data or information. The term electronic and information technology includes, but is not limited to, telecommunications products (such as telephones), information kiosks and transaction machines, World Wide websites, multimedia, and office equipment such as copiers and fax machines. The term does not include any equipment that contains embedded information technology that is used as an integral part of the product, but the principal function of which is not the acquisition, storage, manipulation, management, movement, control, display, switching, interchange, transmission, or reception of data or information. For example, HVAC (heating, ventilation, and air conditioning) equipment such as thermostats or temperature control devices, and medical equipment where information technology is integral to its operation, are not information technology. Information technology. Any equipment or interconnected system or subsystem of equipment, that is used in the automatic acquisition, storage, manipulation, management, movement, control, display, switching, interchange, transmission, or reception of data or information. The term information technology includes computers, ancillary equipment, software, firmware and similar procedures, services (including support services), and related resources.

300 Operable controls. A component of a product that requires physical contact for normal operation. Operable controls include, but are not limited to, mechanically operated controls, input and output trays, card slots, keyboards, or keypads. Product. Electronic and information technology. Self Contained, Closed Products. Products that generally have embedded software and are commonly designed in such a fashion that a user cannot easily attach or install assistive technology. These products include, but are not limited to, information kiosks and information transaction machines, copiers, printers, calculators, fax machines, and other similar types of products. Telecommunications. The transmission, between or among points specified by the user, of information of the user's choosing, without change in the form or content of the information as sent and received. TTY. An abbreviation for teletypewriter. Machinery or equipment that employs interactive text based communications through the transmission of coded signals across the telephone network. TTYs may include, for example, devices known as TDDs (telecommunication display devices or telecommunication devices for deaf persons) or computers with special modems. TTYs are also called text telephones. Undue burden. Undue burden means significant difficulty or expense. In determining whether an action would result in an undue burden, an agency shall consider all agency resources available to the program or component for which the product is being developed, procured, maintained, or used.

§ 1194.5 Equivalent facilitation. Nothing in this part is intended to prevent the use of designs or technologies as alternatives to those prescribed in this part provided they result in substantially equivalent or greater access to and use of a product for people with disabilities.

301 Subpart B -- Technical Standards

§ 1194.21 Software applications and operating systems. (a) When software is designed to run on a system that has a keyboard, product functions shall be executable from a keyboard where the function itself or the result of performing a function can be discerned textually. (b) Applications shall not disrupt or disable activated features of other products that are identified as accessibility features, where those features are developed and documented according to industry standards. Applications also shall not disrupt or disable activated features of any operating system that are identified as accessibility features where the application programming interface for those accessibility features has been documented by the manufacturer of the operating system and is available to the product developer. (c) A well-defined on-screen indication of the current focus shall be provided that moves among interactive interface elements as the input focus changes. The focus shall be programmatically exposed so that assistive technology can track focus and focus changes. (d) Sufficient information about a user interface element including the identity, operation and state of the element shall be available to assistive technology. When an image represents a program element, the information conveyed by the image must also be available in text. (e) When bitmap images are used to identify controls, status indicators, or other programmatic elements, the meaning assigned to those images shall be consistent throughout an application's performance. (f) Textual information shall be provided through operating system functions for displaying text. The minimum information that shall be made available is text content, text input location, and text attributes. (g) Applications shall not override user selected contrast and color selections and other individual display attributes. (h) When animation is displayed, the information shall be displayable in at least one non- animated presentation mode at the option of the user. (i) Color coding shall not be used as the only means of conveying information, indicating an action, prompting a response, or distinguishing a visual element.

302 (j) When a product permits a user to adjust color and contrast settings, a variety of color selections capable of producing a range of contrast levels shall be provided. (k) Software shall not use flashing or blinking text, objects, or other elements having a flash or blink frequency greater than 2 Hz and lower than 55 Hz. (l) When electronic forms are used, the form shall allow people using assistive technology to access the information, field elements, and functionality required for completion and submission of the form, including all directions and cues.

§ 1194.22 web-based intranet and internet information and applications. (a) A text equivalent for every non-text element shall be provided (e.g., via "alt", "longdesc", or in element content). (b) Equivalent alternatives for any multimedia presentation shall be synchronized with the presentation. (c) web pages shall be designed so that all information conveyed with color is also available without color, for example from context or markup. (d) Documents shall be organized so they are readable without requiring an associated style sheet. (e) Redundant text links shall be provided for each active region of a server-side image map. (f) Client-side image maps shall be provided instead of server-side image maps except where the regions cannot be defined with an available geometric shape. (g) Row and column headers shall be identified for data tables. (h) Markup shall be used to associate data cells and header cells for data tables that have two or more logical levels of row or column headers. (i) Frames shall be titled with text that facilitates frame identification and navigation. (j) Pages shall be designed to avoid causing the screen to flicker with a frequency greater than 2 Hz and lower than 55 Hz. (k) A text-only page, with equivalent information or functionality, shall be provided to make a website comply with the provisions of this part, when compliance cannot be accomplished in any other way. The content of the text-only page shall be updated whenever the primary page changes.

303 (l) When pages utilize scripting languages to display content, or to create interface elements, the information provided by the script shall be identified with functional text that can be read by assistive technology. (m) When a web page requires that an applet, plug-in or other application be present on the client system to interpret page content, the page must provide a link to a plug-in or applet that complies with §1194.21(a) through (l). (n) When electronic forms are designed to be completed on-line, the form shall allow people using assistive technology to access the information, field elements, and functionality required for completion and submission of the form, including all directions and cues. (o) A method shall be provided that permits users to skip repetitive navigation links. (p) When a timed response is required, the user shall be alerted and given sufficient time to indicate more time is required.

Note to §1194.22:

1. The Board interprets paragraphs (a) through (k) of this section as consistent with the following priority 1 Checkpoints of the web Content Accessibility Guidelines 1.0 (WCAG 1.0) (May 5, 1999) published by the web Accessibility Initiative of the World Wide web Consortium:

Section 1194.22 Paragraph WCAG 1.0 Checkpoint (a) 1.1 (b) 1.4 (c) 2.1 (d) 6.1 (e) 1.2 (f) 9.1 (g) 5.1 (h) 5.2 (i) 12.1 (j) 7.1 (k) 11.4

2. Paragraphs (l), (m), (n), (o), and (p) of this section are different from WCAG 1.0. web pages that conform to WCAG 1.0, level A (i.e., all priority 1 checkpoints) must also meet paragraphs

304 (l), (m), (n), (o), and (p) of this section to comply with this section. WCAG 1.0 is available at http://www.w3.org/TR/1999/WAI-WEBCONTENT-19990505.

§ 1194.23 Telecommunications products. (a) Telecommunications products or systems which provide a function allowing voice communication and which do not themselves provide a TTY functionality shall provide a standard non-acoustic connection point for TTYs. Microphones shall be capable of being turned on and off to allow the user to intermix speech with TTY use. (b) Telecommunications products which include voice communication functionality shall support all commonly used cross-manufacturer non-proprietary standard TTY signal protocols. (c) Voice mail, auto-attendant, and interactive voice response telecommunications systems shall be usable by TTY users with their TTYs. (d) Voice mail, messaging, auto-attendant, and interactive voice response telecommunications systems that require a response from a user within a time interval, shall give an alert when the time interval is about to run out, and shall provide sufficient time for the user to indicate more time is required. (e) Where provided, caller identification and similar telecommunications functions shall also be available for users of TTYs, and for users who cannot see displays. (f) For transmitted voice signals, telecommunications products shall provide a gain adjustable up to a minimum of 20 dB. For incremental volume control, at least one intermediate step of 12 dB of gain shall be provided. (g) If the telecommunications product allows a user to adjust the receive volume, a function shall be provided to automatically reset the volume to the default level after every use. (h) Where a telecommunications product delivers output by an audio transducer which is normally held up to the ear, a means for effective magnetic wireless coupling to hearing technologies shall be provided. (i) Interference to hearing technologies (including hearing aids, cochlear implants, and assistive listening devices) shall be reduced to the lowest possible level that allows a user of hearing technologies to utilize the telecommunications product. (j) Products that transmit or conduct information or communication, shall pass through cross- manufacturer, non-proprietary, industry-standard codes, translation protocols, formats or other

305 information necessary to provide the information or communication in a usable format. Technologies which use encoding, signal compression, format transformation, or similar techniques shall not remove information needed for access or shall restore it upon delivery. (k) Products which have mechanically operated controls or keys, shall comply with the following: (1) Controls and keys shall be tactilely discernible without activating the controls or keys. (2) Controls and keys shall be operable with one hand and shall not require tight grasping, pinching, or twisting of the wrist. The force required to activate controls and keys shall be 5 lbs. (22.2 N) maximum. (3) If key repeat is supported, the delay before repeat shall be adjustable to at least 2 seconds. Key repeat rate shall be adjustable to 2 seconds per . (4) The status of all locking or toggle controls or keys shall be visually discernible, and discernible either through touch or sound.

§ 1194.24 Video and multimedia products. (a) All analog television displays 13 inches and larger, and computer equipment that includes analog television receiver or display circuitry, shall be equipped with caption decoder circuitry which appropriately receives, decodes, and displays closed captions from broadcast, cable, videotape, and DVD signals. As soon as practicable, but not later than July 1, 2002, widescreen digital television (DTV) displays measuring at least 7.8 inches vertically, DTV sets with conventional displays measuring at least 13 inches vertically, and stand-alone DTV tuners, whether or not they are marketed with display screens, and computer equipment that includes DTV receiver or display circuitry, shall be equipped with caption decoder circuitry which appropriately receives, decodes, and displays closed captions from broadcast, cable, videotape, and DVD signals. (b) Television tuners, including tuner cards for use in computers, shall be equipped with secondary audio program playback circuitry. (c) All training and informational video and multimedia productions which support the agency's mission, regardless of format, that contain speech or other audio information necessary for the comprehension of the content, shall be open or closed captioned.

306 (d) All training and informational video and multimedia productions which support the agency's mission, regardless of format, that contain visual information necessary for the comprehension of the content, shall be audio described. (e) Display or presentation of alternate text presentation or audio descriptions shall be user- selectable unless permanent.

§ 1194.25 Self contained, closed products. (a) Self contained products shall be usable by people with disabilities without requiring an end- user to attach assistive technology to the product. Personal headsets for private listening are not assistive technology. (b) When a timed response is required, the user shall be alerted and given sufficient time to indicate more time is required. (c) Where a product utilizes touchscreens or contact-sensitive controls, an input method shall be provided that complies with §1194.23 (k) (1) through (4). (d) When biometric forms of user identification or control are used, an alternative form of identification or activation, which does not require the user to possess particular biological characteristics, shall also be provided. (e) When products provide auditory output, the audio signal shall be provided at a standard signal level through an industry standard connector that will allow for private listening. The product must provide the ability to interrupt, pause, and restart the audio at anytime. (f) When products deliver voice output in a public area, incremental volume control shall be provided with output amplification up to a level of at least 65 dB. Where the ambient noise level of the environment is above 45 dB, a volume gain of at least 20 dB above the ambient level shall be user selectable. A function shall be provided to automatically reset the volume to the default level after every use. (g) Color coding shall not be used as the only means of conveying information, indicating an action, prompting a response, or distinguishing a visual element. (h) When a product permits a user to adjust color and contrast settings, a range of color selections capable of producing a variety of contrast levels shall be provided. (i) Products shall be designed to avoid causing the screen to flicker with a frequency greater than 2 Hz and lower than 55 Hz.

307 (j) Products which are freestanding, non-portable, and intended to be used in one location and which have operable controls shall comply with the following: (1) The position of any operable control shall be determined with respect to a vertical plane, which is 48 inches in length, centered on the operable control, and at the maximum protrusion of the product within the 48 inch length (see Figure 1 of this part). (2) Where any operable control is 10 inches or less behind the reference plane, the height shall be 54 inches maximum and 15 inches minimum above the floor. (3) Where any operable control is more than 10 inches and not more than 24 inches behind the reference plane, the height shall be 46 inches maximum and 15 inches minimum above the floor. (4) Operable controls shall not be more than 24 inches behind the reference plane (see Figure 2 of this part).

§ 1194.26 Desktop and portable computers. (a) All mechanically operated controls and keys shall comply with §1194.23 (k) (1) through (4). (b) If a product utilizes touchscreens or touch-operated controls, an input method shall be provided that complies with §1194.23 (k) (1) through (4). (c) When biometric forms of user identification or control are used, an alternative form of identification or activation, which does not require the user to possess particular biological characteristics, shall also be provided. (d) Where provided, at least one of each type of expansion slots, ports and connectors shall comply with publicly available industry standards.

Subpart C -- Functional Performance Criteria

§ 1194.31 Functional performance criteria. (a) At least one mode of operation and information retrieval that does not require user vision shall be provided, or support for assistive technology used by people who are blind or visually impaired shall be provided. (b) At least one mode of operation and information retrieval that does not require visual acuity greater than 20/70 shall be provided in audio and enlarged print output working together or

308 independently, or support for assistive technology used by people who are visually impaired shall be provided. (c) At least one mode of operation and information retrieval that does not require user hearing shall be provided, or support for assistive technology used by people who are deaf or hard of hearing shall be provided. (d) Where audio information is important for the use of a product, at least one mode of operation and information retrieval shall be provided in an enhanced auditory fashion, or support for assistive hearing devices shall be provided. (e) At least one mode of operation and information retrieval that does not require user speech shall be provided, or support for assistive technology used by people with disabilities shall be provided. (f) At least one mode of operation and information retrieval that does not require fine motor control or simultaneous actions and that is operable with limited reach and strength shall be provided.

Subpart D -- Information, Documentation, and Support

§ 1194.41 Information, documentation, and support. (a) Product support documentation provided to end-users shall be made available in alternate formats upon request, at no additional charge. (b) End-users shall have access to a description of the accessibility and compatibility features of products in alternate formats or alternate methods upon request, at no additional charge. (c) Support services for products shall accommodate the communication needs of end-users with disabilities.

309 APPENDIX G

TRAINING PROGRAM ELEMENTS

The following information presents a proposed course content structure for the two training courses. The topics covered and the times allotted represent the commonly covered material found in federal and commercial accessibility training course (GAO, 2009; USPTO, 2009; Webucator, 2009). This information provides a beginning structure that could be refined and adapted as indicated by training feedback and other assessment methods.

Web Accessibility Training for Web Designers Course Outline (24 hours)  An Introduction to Section 508  Text Equivalents  Assessing Your Site's Accessibility  Navigation  Accessible Image Maps  Audio and Multimedia  Special Cases  Accessible Forms  Accessible Tables  Scripts and Applets  Cascading Style Sheets  Review of Section 508 Standards for web Accessibility  Best Practices and Additional Resources  Integrated Testing Programs to Promote Accessibility  Institutionalizing an Accessibility Culture

Web Accessibility and Section 508 Training for Agency Managers Class Outline (8 hours)  An Introduction to Section 508  Legal and Social Concerns for Accessibility  Assessing Your Site's Accessibility

310  Tools and Techniques for Building-In Accessibility  Review of Section 508 Standards for web Accessibility  The ―ROI‖ of Accessibility  Best Practices and Available Resources  Comprehensive Testing Programs to Promote Accessibility  Institutionalizing an Accessibility Culture

311 REFERENCES

456 Berea Street. (2011). 10 colour contrast checking tools to improve the accessibility of your design. Retrieved August 1, 2011, from http://www.456bereastreet.com/archive/200709/10_colour_contrast_checking_tools_to_i mprove_the_accessibility_of_your_design/

Access Board. (1999, May 12). Electronic and Information Technology Access Advisory Committee - Final Report. Retrieved from http://www.access-board.gov/sec508/commrept/eitaacrpt.htm

Access Board. (2009). About the U.S. Access Board. Retrieved from http://www.access- board.gov/about.htm

Access Board. (2010a). Electronic and information technology accessibility standards (Section 508). United States Access Board. Retrieved from http://www.access- board.gov/sec508/standards.htm

Accessibility Institute. (2011). How are the Flesch-Kincaid and Flesch Reading Ease scores calculated? Retrieved July 5, 2011, from http://www.utexas.edu/disability/ai/resource/readability/manual/flesch-calculate- English.html

ADA Amendments Act of 2008. (Public Law 110-325, ADAAA).

Americans with Disabilities Act of 1990, 42 U.S.C. § 12101 et seq. Retrieved March 7, 2008, from Fastcase law library: https://www.fastcase.com/Pages/StatutesSearch.aspx

AdvisorMax. (2007). The real value of professional certifications. Retrieved April 16, 2009, from http://www.advisormax.com/jforum/posts/list/30.max

Ai Squared. (2011). ZoomText. Retrieved July 11, 2011, from http://www.aisquared.com/zoomtext

Aronson, J. (1994). A pragmatic view of thematic analysis. The Qualitative Report, 2(1). Retrieved March 9, 2011, from http://www.nova.edu/ssss/QR/BackIssues/QR2- 1/aronson.html

ASH. (2009). The Americans with Disabilities Act as applicable to sensitive nonsmokers. Retrieved March, 29, 2009, from http://ash.org/papers/h211.htm

AT508.com. (2010). Internet TV for assistive technology. Retrieved from http://www.at508.com/

312 Babbie, E. (2001). The practice of social research. (9th ed.). Belmont, CA: Wadsworth/Thomson Learning.

Baker, S. M., & Kaufman-Scarborough, C. (2001). Marketing and public accommodation: A retrospective on Title III of the Americans with Disabilities Act. Journal of Public Policy & Marketing, 20(2), 297-304.

Balas, E., Boren, S., Brown, G., Ewigman, B., Mitchell, J., & Perkoff, G. (1996). The effect of physician profiling on utilization. Journal of General Internal Medicine, 11(10), 584-590.

Ball, S. (2008). Design for all - How web accessibility affects different people. In: Craven J. (ed.) Web accessibility: Practical advice for the library and information professional. London: Facet Publishing.

Barron, T. (2010). Iowa State architecture professor's book makes new ADA requirements more accessible. Retrieved January 9, 2012, from http://www.news.iastate.edu/news/2010/sep/OsterbergBook

Bartley, S. & Golek, J. (2004). Evaluating the cost effectiveness of online and face to face instruction. Educational Technology & Society, 7(4), 164-175.

Bednarz, A. (2002). Getting plugged into the e-government. Network World. Retrieved from http://www.nwfusion.com/supp/government2002/authentication.html

Berland, G. K., Elliott, M. N., Morales, L. S., Algazy, J. I., Kravitz, R. L., Broder, M. S., et al. (2001). Health information on the Internet: Accessibility, quality, and readability in English and Spanish. Journal of the American Medical Association, 285(20), 2612-2621.

Bernard, H. R., & Ryan, G. W. (2010). Analyzing qualitative data: Systematic approaches. Los Angeles, CA: SAGE.

Bertot, J. C., McClure, C. R., Thompson, K. M., Jaeger, P. T., & Langa, L. A. (2003). Florida electronic library: Pilot project functionality assessment for the Florida Division of Library Services. Tallahassee, FL: Information Use Management and Policy Institute. Retrieved from http://www.ii.fsu.edu/Research/Projects/All/Projects-from-2009-to- 1999/2003-Project-Details

Bertot, J. C., Snead, J. T., Jaeger, P. T., & McClure, C. R. (2006). Functionality, usability and accessibility: Iterative user-centered assessment strategies for digital libraries. Performance Measurement and Metrics: The International Journal for Library and Information Services, 7(1), 17-28. DOI:10.1108/14678040610654828

Bloom, B. (2005). Effects of continuing medical education on improving physician clinical care and patient health: A review of systematic reviews. International Journal of Technology Assessment in Health Care, 21(3), 380-385.

313 Braddock, D. L., & Parish, S. L. (2001). An institutional history of disability. In G. L. Albrecht, K. D. Seelman, & M. Bury (Eds.), Handbook of disability studies (pp. 11-68). Thousand Oaks, CA: Sage.

Brajnik, G. (2008). A comparative test of web accessibility evaluation methods. In Assets '08: Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility, pages 113-120, New York, NY, USA. ACM.

Bray, M., Flowers, C., & Gibson., P. (2003). Accessibility of school districts‘ web sites: A descriptive study. Information Technology in Childhood Education Annual (1), 209 –221.

Brinck, T., Gergle, D., & Wood, S. D. (2002). Usability for the web: Designing websites that work. New York: Academic Press.

Brobst, J. (2009). Evaluating the accessibility of Florida public library home pages. Libri, 59(2), 88-103.

Caldwell, R. (2006). Web-accessibility, E-learning, and academic libraries. International Journal of Public Information Systems, 2006(1), 1-9. Retrieved from http://www.ijpis.net/ index.html?http%3A//www.ijpis.net/issues/no1_2006/no1_2006_p1.htm.

Cardinali, R., & Gordon, . (2002). Technology: Making things easier for all of us - For the disabled making things possible. Equal Opportunities International, 21(1), 65-79.

Ceaparu, I., & Shneiderman, B. (2002). Improving web-based civic information access: A case study of the 50 US States. Proceedings of the IEEE International Symposium on Technology and Society, pp. 275–282.

Chilson, M. (2002) Website accessibility for the visually impaired and web policy at NCATE- accredited colleges and universities in the Mountain Region. (Doctoral dissertation, Idaho State University, 2002). Retrieved from the ProQuest dissertation and theses database: http://proquest.umi.com.proxy.lib.fsu.edu/pqdweb?index=0&did= 726482451&SrchMode=1&sid=2&Fmt=2&VInst=PROD&VType=PQD&RQT=309& VName=PQD&TS=1196632015&clientId=20174

Choudhury, A. (2010). Cohen's Kappa. Retrieved March 24, 2011 from Experiment Resources: http://www.experiment-resources.com/cohens-kappa.html

Cline, R. J., & Haynes, K. M. (2001). Consumer health information seeking on the Internet: The state of the art. Health Education Research, 16, 671–692.

Coiera, E. (1996). The Internet‘s challenge to health care provision. British Medical Journal, 312, 3-4.

Colorfilter.wickline.org. (n.d.). Colorblind Web Page Filter. Retrieved July 5, 2011, from http://colorfilter.wickline.org/

314

Creswell, J. W. (2002). Mixed method designs. In J. W. Creswell, Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research (pp. 559- 601). Upper Saddle River, NJ and Columbus, OH: Merrill Prentice Hall.

Cullen, K. (2008). Accessibility of ICT products and services to disabled and older people. Retrieved from http://www.elearningeuropa.info/main /index.php?page=search_results&qry=Accessibility&service=1&qmatch=0&area=- 1&find=&sfld=1&ad=DESC&pn=6&npp=10

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13, 319-339.

Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35, 983-1003.

Diaper, D. & Worman, L. (2002). Two falls out of three in the automated accessibility assessment of World Wide Web sites: A–Prompt v. Bobby, Retrieved on December 1, 2007, from http://dec.bmth.ac.uk/staff/ddiaper/hci2003sub.pdf

DisabilityNow. (2009). Disability rights and lefts. Retrieved March 23, 2009, from http://www.disabilitynow.org.uk/latest-news2/politics-1/disability-rights-2013-and-lefts

Disability Rights Commission (United Kingdom). (2004). The web: Access and inclusion for disabled people. London: The Stationary Office. Retrieved from http://www.drc-gb.org/PDF/2.pdf

Doubleday, A., Ryan, M., Springett, M., & Sutcliffe, A. (1997). A comparison of usability techniques for evaluating design [Electronic version]. Proceedings of the conference on Designing interactive systems: processes, practices, methods, and techniques. Amsterdam.

DRA. (2009). About disability rights advocates. Retrieved March 30, 2009, from http://www.dralegal.org/about/index.php eAccess. (2011). Accessibility statement for web accessibility. Retrieved February 23, 2011, from http://www.eaccessplus.eu/node/43

Economic & Social Data Ranking. (2006). Economic & social data ranking/ United States. Retrieved March 23, 2009, from http://dataranking.com/country.cgi?LG=e&CO=30

Ellison, J. (2004). Accessing the accessibility of fifty United States government web pages: Using Bobby to check on Uncle Sam. First Monday, 9(7). Retrieved from http://www.firstmonday.org/issues/issue9_7/ellison/index.html

315 EPA. (1993). Environmental Protection Agency. Memorandum. Subject: Formation of a Federal Advisory Committee Act Subcommittee for New Source Review (NSR) Issues. Retrieved April 19, 2009, from http://www.epa.gov/Region7/programs/artd/air/nsr/nsrmemos/ subcomm.pdf

EvenGrounds. (2010). Accessibility statement: What is it, and who uses it? Even Grounds, Accessibility Consulting Inc. Retrieved February 24, 2011, from http://www.evengrounds.com/articles/accessibility-statement

Fagan, J. & Fagan, B. (2006). An accessibility study of state legislative websites. Government Information Quarterly, 21, 65-85.

Feinstein, O. (2002). Use of evaluations and the evaluation of their use. Evaluation, 8, 433-439.

Fender, G., Prentice, A., Gorst, T., Nixon, R., Duffy, S., Day, N., & Smith, S. (1999). Randomized controlled trial of educational package on management of menorraghia in primary care: The Anglia mennorhagia education study. British Medical Journal, 318, 1246-1250.

Fleischer, D., & Zames, F. (2001). The disability rights movement: From charity to confrontation. Philadelphia: Temple University Press.

Florin, D., & Coulter, A. (2001). A partnership in primary care consultation. In: Gillam S. (ed.) New Beginnings: Towards Patient and Public Involvement in Primary Care. London: King‘s Fund.

Fox, S. (2005). Health information online. Pew Internet & American Life Project. Retrieved from http://www.pewinternet.org/PPF/r/156/report_display.asp

Fox, S. (2006). Online health search 2006. Washington, DC: Pew Internet and American Life Project. Retrieved from http://www.pewinternet.org/~ /media//Files/Reports/2006/PIP_Online_Health_2006.pdf.pdf

Fox, S. & Fallows, D. (2003). Internet health resources: Health searches and email have become more commonplace, but there is room for improvement in searches and overall Internet access. Technical Report from the Pew Internet and American Life Project. Washington, DC: Pew Research Center.

Fox, S., & Rainie, L. (2000). The online health care revolution. Pew Internet & American Life Project. Retrieved from http://www.pewinternet.org/PPF/r/26/report_display.asp

Fox, S., & Rainie, L. (2002). Vital decisions: How Internet users decide what information to trust when they or their loved ones are sick. Pew Internet & American Life Project. Retrieved from http://www.pewinternet.org/PPF/r/59/report_display.asp

316 Fridsma. D. B., Ford, P., & Altman R. (1994). A survey of patient access to electronic mail: Attitudes, barriers and opportunities. Proceedings of the Symposium on Computer Applications in Medicine. Journal of the American Medical lnformatics Association (symposium supplement). 15-19.

FWC. (1997). U.S. Government Internet Domain Names. Federal Networking Council. May 1997. Retrieved from http://tools.ietf.org/html/rfc2146

Gallagher, T. (2003). Is big government back? The political economy of activist government policy. Business Economics Retrieved March 30, 2009, from http://findarticles.com/p/articles/mi_m1094/is_3_38/ai_107422982/

GAO. (2009). About GAO. General Accountability Office. Retrieved April 19, 2009, from http://www.gao.gov/about/index.html

Gemoets, D., Rosemblat, G., Tse, T., & Logan, R. (2004). Assessing readability of consumer health information: An exploratory study. Medinfo, 11(Pt. 2), 869-73.

Georgia Tech research Institute. (2007). Accessibility assistant. (Source: accessIT's "Accessibility Guidelines for Application Software"). Retrieved July 7, 2011, from http://accessibility.gtri.gatech.edu/assistant/definitions.php

Gibson, Dunn, & Crutcher, LLP. (2009). Year-end update on class actions: Explosive growth in class actions continues despite mounting obstacles to certification. Retrieved from http://www.gibsondunn.com/publications/Pages/Year-EndUpdateOnClassActions.aspx

Golden, D., & Buck, D. (2003). State IT accessibility policy: The landscape of today. Information Technology and Disabilities, 9(1). Retrieved from http://www.rit.edu/~easi/itd/itdv09n1/golden.htm.

Gore, A. (1993). Reengineering through information technology. Retrieved April 17, 2009, from http://govinfo.library.unt.edu/npr/library/reports/it01.html

Grandon, E. E., & Pearson, J. M. (2004). Electronic commerce adoption: An empirical study of small and medium U.S. businesses. Information & Management, 42, 197-216.

Grant Thorton, LLP. (2009). Elevating our federal workforce. Retrieved March 30, 2009, from www.ourpublicservice.org/OPS/publications/download.php?id=130

Gray, W. D., & Salzman, M. C. (1998). Damaged merchandise? A review of experiments that compare usability evaluation methods. Human-Computer Interaction, 13(3), 203-261.

GSA. (2009). Government domain registration and services. General Services Administration. Retrieved from http://www.dotgov.gov/

317 Guadagnoli, E., & Ward, P. (1998). Patient participation in decision making. Social Science and Medicine. 47, 329–339.

Guest, G., Bunce, A., & Johnson, L. (2006). How Many Interviews Are Enough?: An Experiment with Data Saturation and Variability. Field Methods (1) 18. 59-82. DOI: 10.1177/1525822X05279903

Gunderson, J. (2009). ADA and Section 508 disability and access requirements. Retrieved March 30, 2009 from http://cita.disability.uiuc.edu/presentations/2001-10-16- CIC/JPGFrame1.html

Hackett, S., Parmanto, B., & Zeng, X. (2005). A retrospective look at website accessibility over time. Behaviour & Information Technology, 24(6), 407-417.

Hardy, M. (1999). Doctor in the house: The Internet as a source of lay health knowledge and the challenge to expertise. Sociology of Health & Illness. 21, 820–835.

Harris Interactive. (2006). Number of Cyberchrondriacs: Adults who have ever gone online for health information – increases to an estimated 136 million nationwide. Retrieved from http://www.harrisinteractive.com/harris_poll/index.asp?PID=686

Hart, J. (2007). Internet law: A field guide. Washington, DC: BNA Publishers.

Hernon, P., Reylea, H. C., Dugan, R. E., & Cheverie, J. F. (2002). United States government information: Policies and sources. Westport, CT: Libraries Unlimited.

Hesse, B.W., Nelson, D. E., Kreps, G. L., Croyle, R. T., Arora, N. K., Rimer, B. K. & Viswanath, K. (2005). Trust and sources of health information: The impact of the Internet and its implications for health care providers: Findings from the first Health Information National Trends Survey. Archives of Internal Medicine, 165 (12), 2618-2624.

Hignite, K. (2000). The accessible association. Association Management, 52(13), 36-43.

HiSoftware. (2010). Welcome to the HiSoftware CynthiaSays portal. Retrieved from http://www.contentquality.com/

Howard, M. (2001). E-government across the globe: How will "e" change government? Government Finance Review, 17(4), 6-9.

Howell, J. (2008). Accessibility Advice and Guidance. in Craven, J. (ed.) Web accessibility: Practical advice for the library and information professionals. pp. 57-73. London: Facet Publishing.

HTCTU. (2010). High Tech Center Training Unit of the California Community Colleges. Retrieved from http://www.htctu.fhda.edu/publications/articles/cahe/section508.pdf

318 Huberman, A. M., & Miles, M. B. (2002). The qualitative researcher's companion. Thousand Oaks, CA: Sage Publications.

Hudson, L. (2002). A new age of accessibility. Library Journal, 127(1), 19-21.

Illinois. (2009). Illinois Information Technology Accessibility Act Standards 1.0. Retrieved July 11, 2011, from http://www.dhs.state.il.us/IITAA/IITAAStandards.html

IRS, Internal Revenue Service. (2009). The agency, its mission and statutory authority. Retrieved March 17, 2009, from http://www.irs.gov/irs/article/0,,id=98141,00.html

Ishikawa, J. (2001). Accessibility check of websites. Retrieved from http://fuji.shizuoka- ken.ac.jp/˜ishikawa/web check.htm

IWA. (2010). IWA: Global Leader for Advancing the Web Professional. International Webmaster Association. Retrieved from http://www.iwanet.org/

Jackson-Sanborn, E., Odess-Hamish, K., & Warren, N. (2002). Website accessibility: A study of six genres. Library Hi Tech, 20(3), 308-317.

Jacko, J. A., & Hanson, V. L. (2002). Universal access and inclusion in design. Universal Access in the Information Society, 2, 1-2.

Jaeger, P. T. (2002). Section 508 goes to the library: Complying with federal legal standards to produce accessible electronic and information technology in libraries. Information Technology and Disabilities, 8(2). Retrieved from http://www.rit.edu.proxy.lib.fsu.edu/~easi/itd/itdv08n2/jaeger.htm

Jaeger, P. T. (2003). The importance of measuring the accessibility of the federal e-government: What studies are missing and how these issues can be addressed. Information Technology and Disabilities, 9(1). Retrieved from http://www.rit.edu/~easi/itd/itdv09n1/jaeger.htm

Jaeger, P. T. (2004). The social impact of an accessible E-democracy. Journal of Disability Policy Studies. 15(1), 19-27.

Jaeger, P. T. (2006). Multi-method evaluation of U.S. federal electronic government websites in terms of accessibility for persons with disabilities. (Doctoral dissertation, Florida State University, 2006). Retrieved from the Florida State University Electronic Thesis and Dissertations website: http://etd.lib.fsu.edu.proxy.lib.fsu.edu/ theses/available/etd-03062006-120905

Jaeger, P. T., & Bowman, C. (2005). Understanding disability: Inclusion, access, diversity, and civil rights. Westport, CT: Praeger Publishers.

Jellinik, D., Lambden, P., & Lilley R. (2000). Your health and the Internet. Norwich (UK): Stationery Office.

319 Johnston, S., & Helms, L. (2008). Policy development for advocates: Lessons from the job accommodation network (JAN). Journal of Disability Policy Studies, 19, 172-181.

JuicyStudio (2009). Readability test. Retrieved March 10, 2009, from http://juicystudio.com/services/readability.php#interpret

Kanayama, T. (2003). Leaving it all up to industry: People with disabilities and the Telecommunications Act of 1996. Information Society, 19, 185-194.

Kane, S.K., Shulman, J.A., Shockley, T.J., & Ladner, R.E. (2007). A Web accessibility report card for top international university Web sites. Proceedings of the 2007 International Cross-Disciplinary Conference on Web Accessibility (W4A), New York: Association for Computing Machinery Press, 148-156.

Kelly, B., Phipps, L., & Swift, E. (2004). Developing a holistic approach for e-learning accessibility. Canadian Journal of Learning and Technology, 30(3). Retrieved from http://www.cjlt.ca/index.php/cjlt/article/view/138/131

Killam, B. & Holland, B. (2001). Position paper on the suitability to task of automated utilities for testing web accessibility compliance. The Usability SIG Newsletter: Usability Interface Accessibility and Usability: Partners in Effective Design (dated April 2003).

King, M., Thatcher, J.W., Bronstad, P.M., & Easton, R. (2005). Managing usability for people with disabilities in a large web presence. IBM Systems Journal, 44(3), 519-535.

Klein, K. (2007). Is your website handicap-accessible? Smart answers. Business Week On-line. Retrieved from http://www.businessweek.com/smallbiz /content/dec2007/sb20071217_774630.htm

Kruse, D., & Hale, T. (2003). Disability and employment: Symposium introduction. Industrial Relations, 42, 1-10.

Kruse, D., & Schur, L. (2003). Employment of people with disabilities following the ADA. Industrial Relations, 42, 31-66.

Lang, T. (2003). Comparing website accessibility evaluation methods and learnings from usability evaluation methods. Peak Usability. Retrieved from http:// www.peakusability.com.au/pdf/

LaPlant, W. P., Laskowski, S. J., & Stimson, M. J. (2001). Report on UPA Workshop 6 Exploring Measurement and Evaluation Methods for Accessibility [Electronic version]. Workshop at Tenth Annual Usability Professionals’ Association Conference.

LaPorte, R. E. (1994). Global public health and the information superhighway. British Medical Journal, 308. 1651-1652.

320 Law, L., & Hvannberg, E. (2002). Complementarity and convergence of heuristic evaluation and usability test: a case study of universal brokerage platform [Electronic version]. Proceedings of the Second Nordic Conference on Human-computer Interaction, October 19-23, 2002. Denmark: Aarhus.

Lazar, J., Beere, P., Greenidge, K., & Nagappa, Y. (2003). Web accessibility in the mid-Atlantic United States: A study of 50 websites. Universal Access in the Information Society, 2(4), 1–11.

Lazar, J., Dudley-Sponaugle, A. & Greenidge, K. (2004). Improving web accessibility: A study of webmaster perceptions. Computers and Human Behavior. 20(2), 269-288.

Lazar, J., & Greenidge, K. (2006). A year older, but not necessarily wiser: web accessibility trends over time. Universal Access in the Information Society Journal, 4(4), 285-291.

Lederer, A. L., Maupin, D. J., Sena, M. P., & Zhuang, Y. (2000). The technology acceptance model and the World Wide Web. Decision Support Systems, 29, 269-282.

Lemon, G. (2005). How to write an accessibility statement. Retrieved March 3, 2011, from http://juicystudio.com/

Leuchovius, D. & Parker, R. (2008). ADA & A: The Rehabilitation Act and ADA connection. Pacer Center, Retrieved March 15, 2009, from http://www.pacer.org/publications/adaqa/ adaqa.asp

Leung, P., Owens, J., Lamb, G., Smith, K., Shaw, J. & Hauff, R. (1999). Assistive technology: Meeting the technology needs of students with disabilities in postsecondary education. Retrieved from http://www.dest.gov.au/archive/ highered/eippubs/eip99-6/eip99_6.pdf

Lilly, E., & Van Fleet, C. (2000). Measuring the accessibility of public library home pages. Reference & User Services Quarterly, 40(4), 156-163.

Lu, J., Yu, C. S.. Liu, C., & Yao, J. (2003). Technology acceptance model for wireless Internet. Internet Research: Electronic Networking Applications and Policy,13(3), 206-222.

Luongo, T. (2008). The ADA Amendments Act of 2008: Another effort to eliminate discrimination against the disabled. HRTutor.Com. Retrieved March 18, 2009, from http://www.hrtutor.com/en/news_rss/articles/2008/10-15-ADA-Amendments-Act -of-2008-Another-Effort-to-Eliminate-Discrimination-Against-the-Disabled.aspx

Maclure, M., Allen, M., Bacovsky, R., Bugden, S., Lopatka, H., MacNair, K., Morrow, R., et al.. (2006). Show me the evidence: Best practices for using educational visits to promote evidence-based prescribing. Canadian Academic Detailing Collaboration and Drug Policy Futures. Retrieved on October 21, 2007, from http://pdfserve.pharmacy.uq.edu.au/ qumdatabase/PDFs/ID565_Report_1.4MB.pdf

321 Malone, M., Harris, R., Hooker, R., Tucker, T., Tanna, N., & Honnor S. (2004). Health and the Internet - Changing boundaries in primary care. Family Practice, 21, 189–191.

Marcis, J., & Bland, E. (2006). The value of professional certifications in academe. Atlantic Economic Journal, 26(1), 116.

Marincu, C., & McMullin, B. (2004). A comparative analysis of web accessibility and technical standards conformance in four EU states. First Monday, 9(7). Retrieved from http://www.firstmonday.org/issues/issue9_7/marincu/index.html

Mason, J., Freemantle, N., Nazareth, I., Eccles, M., Haines, A., & Drummond, M. (2001). When is it cost-effective to change the behavior of health professionals? Journal of the American Medical Association, 286(23), 2988-2992.

Mather, D. (2009). Training – The backbone of cultural change. Reliabilityweb.Com. Retrieved March 26, 2009, from http://reliabilityweb.com/index.php/articles/training_- _the_backbone_of_cultural_change

McClure, C. R. (1999). Information policy. In A. Kent (Ed.), The encyclopedia of library and information science (pp. 306−314). New York: Marcel Dekker.

McClure, C. R., Moen, W. E., & Bertot, J.C. (1999). Descriptive assessment of information policy initiatives: GILS as an example, Journal of the American Society for Information Science, 50(4), 314–330.

McClure, C. R. (2002). Information policy based indicators to assess US federal websites: Methods and issues. Proceedings of the 4th North Umbria International Conference on Performance Measures in Library and Information Services pp.145-154.

McCord, S., Frederiksen, L., & Campbell, N. (2002). An accessibility assessment of selected web-based health information resources, Library Hi Tech, 20 (2), 188 – 198. DOI: 10.1108/07378830210432552

McLaughlin, M. W. (1987). Learning from experience: Lessons from policy implementation. Educational Evaluation and Policy Analysis, 9(2), 171-178.

McKean, C. (2007). Section 508: Reconstruction in progress. 508Portal.Com. Retrieved March 14, 2009, from http://www.508portal.com/?q=node/251

Michael, S. (2004). Making government accessible—online. Federal Computer Week, 4, 24–30.

Microsoft Corporation. (2011). Test your document's readability. Retrieved March 26, 2011 from http://office.microsoft.com/en-us/word-help/test-your-document-s-readability- HP010148506.aspx#BM1

322 Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. (2nd ed.). Thousand Oaks, CA: Sage Publications.

Morahan-Martin, J. M. (2004). How Internet users find, evaluate, and use online health information: A cross-cultural review. CyberPsychology & Behavior, 7, 497–510.

Moss, T. (2007). The problem with automated accessibility testing tools. Retrieved April 14, 2007, from http://www.webcredible.co.uk/user-friendly-resources/web-accessibility /automated-tools.shtml

Mueller, J. P. (2003). Accessibility for everybody: Understanding the Section 508 accessibility requirements. New York: Springer-Verlag.

Muir, A., & Oppenheim, C. (2002). National information policy developments worldwide II: Universal access - addressing the digital divide. Journal of Information Science, 28(4), 263-273.

National Council on Disability. (2001). The accessible future. Retrieved February 20, 2009, from http://www.ncd.gov

NCAM. (2009). National Center for Accessible Media. Making educational software and web sites accessible. Retrieved March 16, 2009, from http://ncam.wgbh.org/cdrom/ guideline2003/policy.html

NDU. (2009). National Defense University. Retrieved April 16, 2009, from http://www.ndu.edu/

NetApplications. (2010). Browser Market Share. Retrieved from http://www.netmarketshare.com/browser-market-share.aspx?qprid=0&qpaf=- 000%09101%09US%0D&qptimeframe=M&qpsp=136&sample=28

Nielsen, J. (2000). Designing web usability. Indianapolis: New Riders.

Nugent, J. D. (2001). If e-democracy is the answer, what's the question? National Civic Review, 90, 221-233.

OMB, Office of Management and Budget. (2009). About OMB. Retrieved March 17, 2009, from http://www.whitehouse.gov/omb/organization_role/

OPM, Office of Personnel Management (2009). Total government employment since 1962. Retrieved April 17, 2009, from http://www.opm.gov/feddata/HistoricalTables/TotalGovernmentSince1962.asp

Paasche-Orlow, M., Taylor, H., & Brancati, F. (2003). Reliability standards for informed consent forms as compared with actual readability. The New England Journal of Medicine, 348, 721-726.

323 Paciello, M. (2000). Web accessibility for people with disabilities. Lawrence, : CMP Books.

Paciello, M. (2007). One reason why Section 508 isn‘t working. The Paciello Group Blog. Retrieved Mar 14, 2009, from http://www.paciellogroup.com/blog/?p=6

Pappin, G. (2002). Blind Justice. The Harvard Salient. Retrieved March 23, 2009, from http://www.hcs.harvard.edu/~salient/issues/09272001/09272001_feature_pappin.html

Patterson, D. (2002). Along the disability divide. Government Technology’s Electronic Government, 3(1), 8-13.

Patton, C.V. (2010). Steps for a Successful Policy Analysis. Retrieved from http://www.socialresearchmethods.net/tutorial/Barrien/barrien.htm

Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd ed.). Newbury Park, CA: Sage Publications.

Patton, M. Q. (1997). Utilization-focused evaluation: A new century text (3rd ed.). Thousand Oaks, CA: Sage.

Pells, M. (1998). The webmaster demystified. Association Management, 50(9). 46-52.

Peters, I. (2009). Future of the Internet. Net History. Retrieved from http://www.nethistory.info/History%20of%20the%20Internet/future.html

Peterson, W. (1998). Public policy affecting universal design. Assistive Technology, 10(1), 13- 20.

Pew Internet and American Life Project. (2003). The ever-shifting Internet population. Washington, DC. Retrieved from http://www.pewinternet.org/ pdfs/PIP_Shifting_Net_Pop_Report.pdf

Pew Internet and American Life Project. (2004). People with disabilities. Washington, DC. Retrieved from http://www.pewinternet.org/reports/

Pew Internet and American Life Project. (2007). Internet Activities. Washington, DC: Pew Internet & American Life Project.

Politico. (2008). The White House Political Office will remain. Retrieved March 23, 2009, from http://www.mlive.com/us- politics/index.ssf/2008/11/white_house_political_office_w.html

Powell, J. & Clarke, A. (2002). The WWW of the World Wide Web: Who, what, and why? Journal of Medical Internet Research, 4(1). Retrieved from http://www.jmir.org/2002/1/e4/

324 Rebell, M. (1986). Structural discrimination and the rights of the disabled. Georgetown Law Journal, 74, 1435-1489.

Quinn, R. (1991). The Americans with Disabilities Act: Time for amendments. Retrieved March 29, 2009, from http://www.cato.org/pub_display.php?pub_id=1018

Raker, K., & Boyd, S. (2007). Utilizing academic detailing. Business and Company Resource Center. 161(16), 62-71.

Rehabilitation Act of 1973, 29 U.S.C.A. § 701 et seq. Retrieved January 7, 2008, from Fastcase law library: https://www.fastcase.com/Pages/StatutesSearch.aspx

Ritchie, H., & Blanck, P. (2003). The promise of the Internet for disability: A study of online services and website accessibility at centers for independent living. Behavioral Sciences and the Law, 21, 5–26.

Robb, K. (2001). Confusion reigns over accessibility compliance. Cited in: McLawhorn, L. (2001). Leveling the accessibility playing field: Section 508 of the Rehabilitation Act. North Carolina Journal of Law and Technology, 3, 63-100.

Rogerson, S., & Fairweather, N. (2001). Healthcare websites. IMIS Journal, 11(5). Retrieved from http://www.ccsr.cse.dmu.ac.uk/resources /general/ethicol/Ecv11no5.html

Schein, E. (2004). Organizational culture and leadership (3rd ed.). San Francisco: Jossey-Bass.

Schmeckle, J. (2003). Online training: An evaluation of the effectiveness and efficiency of training law enforcement personnel over the Internet. Journal of Science Education and technology, 12(3), 205-240.

Schmetzke, A. (2001). Web accessibility at university libraries and library schools. Library Hi Tech, 19(1), 35-49.

Schutt, R. (2006). Investigating the social world: The process and practice of research (5th ed.). Thousand Oaks, CA: Pine Forge Press (A Sage Company).

Scotch, R. (2001). From good will to civil rights: Transforming federal disability policy (2nd ed.). Philadelphia, PA: Temple University Press.

Seale, J. (2006). A contextualized model of accessible e-learning practice in higher education institutions. Australasian Journal of Educational Technology, 22(2), 268-288.

Section508. (2009). Section 508 Training. Retrieved April 16, 2009, from http://www.section508.gov/index.cfm?fuseaction=content&id=5

325 Section508.gov. (2009). 508 Law. Retrieved from http://www.section508.gov/index.cfm?FuseAction=Content&ID=3

Section508.gov. (2010). Summary of Section 508 standards. Retrieved from http://www.section508.gov/index.cfm?FuseAction=Content&ID=11

Sellu, D. (1996). Clinical encounters in cyberspace. British Medical Journal. 312, 49.

Shapiro, J. P. (1993). No pity: People with disabilities forging a new civil rights movement. New York: Times Books.

Shneiderman, B. (2000). Universal usability: pushing human–computer interaction research to empower every citizen. Communications of the ACM, 43 (5), 84–91.

Silagy, C., & May, F. (1997). An overview of current practices of academic detailing and opinion leader education in Australia and internationally. Retrieved on October 21, 2007, from http://pdfserve.pharmacy.uq.edu.au/qumdatabase/PDFs/ID565_Report_1.4MB.pdf

Slatin, J., & Rush, S. (2003). Maximum accessibility. Addison-Wesley, New York.

Sloan, D. (2008). The importance of web accessibility. In: Craven J. (ed.) Web accessibility: Practical advice for the library and information professional. London: Facet Publishing.

Sloan, D., Gregor, P., Booth, P. & Gibson, L. (2002). Auditing Accessibility of UK Higher Education websites. Interacting with Computers. 12, 313-325.

Sloan, D., Dickson, A., McIlroy, N. and Gibson, L. (2006). Accessibility statements are inaccessible to older web users. In HCI, the Web and the Older Population Workshop at British HCI, 12 September, London, UK.

Smith, J. (2009). Developing web accessibility: Section 508 compliance of post secondary educational website home pages. Dissertation. Capella University.

StatCounter. (2011). StatCounter global stats: Top 5 browsers in the United States as of March 2011. Retrieved on July 7, 2011, from http://gs.statcounter.com/#browser-US-monthly- 201103-201103-bar

Stein, M. (2002). Web accessibility for the visually impaired and web policy at NCATE accredited colleges and universities in the pacific region (Doctoral Dissertation, Idaho State University, 2002). Proquest UMI.

Stephanidis, C., & Savidis, A. (2001). Universal access in the information society: Methods, tools, and interactive technologies. Universal Access in the Information Society, 1, 40-55.

Stowers, G. (2002). The state of federal websites: The pursuit of excellence. Retrieved from http://whitepapers.zdnet.co.uk/0,1000000651,260115771p,00.htm

326

Sullivan, T., & Matson, R. (2000). Barriers to use: Usability and content accessibility on the web‘s most popular sites. Proceedings of the ACM Conference on Universal Usability, 139–144.

Switzer, J. (2003). Disabled rights: American disability policy and the fight for equality. Washington, DC: Georgetown University Press.

Takata, Y., Nakamura, T., & Seki, H. (2004, January). Accessibility verification of WWW documents by an automatic guideline verification tool. Proceedings of the 37th Annual Hawaii International Conference on System Sciences, Big Island, Hawaii, 1-10.

Tang, B. A. (2001). Section 508 tips from the trenches. Federal Computer Week. Retrieved February 20, 2004, from http://www.fcw.com.proxy.lib.fsu.edu

Taylor, R. (2002). Pros and cons of online learning – A faculty perspective. Journal of European Industrial Training, 26(1), 24-37.

TecAccess. (2009). Section 508 training and awareness classes. Retrieved April 18, 2009, from http://www.tecaccess.net/content/trainaware.shtml

TEITAC (Telecommunications and Electronic and Information Technology Advisory Committee). (2008). Report to the Access Board: Refreshed accessibility standards and guidelines in telecommunications and electronic and information technology. (April 2008). Retrieved from http://www.access-board.gov/sec508/refresh/report/#3

Texas Center for Digital Knowledge. (2002). Assessment of ZLOT resource discovery service proof of concept prototypes. Denton, TX: University of North Texas.

Thatcher, J. (2006). Accessibility testing. In Thatcher, et al. (eds.), Web accessibility: Web standards and regulatory compliance, Berkeley, CA: Friends of Ed. (distributed by Springer-Verlag)

Thatcher, J. (2009). Accessibility checking software. Retrieved from http://jimthatcher.com/testing2.htm

Thatcher, J. (2010). Side-by-side WCAG vs. 508. Retrieved from http://jimthatcher.com/sidebyside.htm

Thatcher, J., Kirkpatrick, A., Rutter, R., Heilamnn, C. Waddell, C. Burks, M., Henry, S. L., Lawson, B. Urban, M., & Lauke, P. (2006). Web accessibility: Web standards and regulatory compliance, NewYork: Friends of Ed.

327 Thompson, K. M., McClure, C. R., & Jaeger, P. T. (2003). Evaluating federal websites: Improving e-government for the people", in George, J.F. (Eds.), Computers in society: Privacy, ethics, and the Internet, Prentice-Hall, Upper Saddle River, NJ, 400-412.

Tomlinson, L. (2008). Are accessibility statements useful? Digital Web Magazine, Retrieved February 25, 2011, from http://www.digital-web.com/articles/are_accessibility _statements_useful

United States Access Board. (2010). About the Architectural Barriers Act and other disability rights laws. Retrieved from http://www.access-board.gov/about/ aboutABA/brochure.htm

United States Access Board. (2008). Advisory committee presents report on updated 508 standards. (dated April 3, 2008). Retrieved from http://www.access-board.gov/news/teitac-report.htm

U.S. Commission on Civil Rights (2002). Funding federal civil rights enforcement: 2000–2003. Retrieved March 26, 2009, from http://www.usccr.gov/pubs/crfund02/report.htm

USDOJ (2005). A guide to disability rights laws. U.S Department of Justice, Civil Rights Division, Disability Rights Section, September 2005, Retrieved from http://www.ada.gov/cguide.htm

USPTO. (2009). Section 508 Spring course schedule. United State patent office, Retrieved April 16, 2009, from http://www.uspto.gov/go/cio/s508/courses.htm

Vischeck. (2011). Try Vischeck on a webpage. Retrieved July 31, 2011, from http://www.vischeck.com/vischeck/vischeckURL.php

Virzi, R. A. (1992). Refining the test phase of usability evaluation: How many subjects is enough? Human Factors, 34, 457-468.

Waddell, C. (1999). The growing digital divide in access for people with disabilities: Overcoming barriers to participation in the digital economy. Retrieved from http:www.icdri.org/CynthiaW/the_digital_divide.htm

Waddell, C. (2006). Worldwide accessibility laws and policies. In Thatcher, et al. (eds.), Web Accessibility: Web standards and regulatory compliance, Berkeley, CA: Friends of Ed. (distributed by Springer-Verlag)

Wade, M., & Parent, M. (2002). Relationships between job skills and performance: A study of webmasters. Journal of Management Information Systems, 18(3), 71-26.

W3C. (2008). Introduction to web accessibility. What is web accessibility? World Wide Web Consortium (W3C). Retrieved from http://www.w3.org/WAI/

328 W3C. (2009). Web Accessibility Initiative (WAI). World Wide Web Consortium (W3C). Retrieved from http://www.w3.org/WAI/

W3C. (2010a). About W3C. World Wide Web Consortium. Retrieved from http://www.w3.org/Consortium/

W3C. (2010b). Complete list of web accessibility evaluation tools. Retrieved from http://www.w3.org/WAI/RC/tools/complete

W3C. (2011). Improving the accessibility of your website. Retrieved from http://www.w3.org/

Waddell, C. (1999). Applying the ADA to the Internet: A web accessibility standard. Retrieved from http://www.icdri.org/CynthiaW/applying_the_ada_to_the_internet.htm

Watson, L. (2009). Writing an accessibility statement. Humanising Technology. Retrieved February 23, 2011, from http://www.nomensa.com/blog/2009/writing-an-accessibility- statement/

WebAim. (2010). United States Laws. Retrieved from http://www.webaim.org/articles/ laws/usa/rehab.php

WebAIM. (2011). Introduction to web accessibility. Retrieved March 9, 2011, from http://webaim.org/intro/

Wells, S. (2001). Is the ADA working? HR Magazine, 46(4), 38-46.

Wells, J., & Barron, A. (2006). School web sites: Are they accessible to all? Journal of Special Education Technology, 21 (3), 23-29 .

West, D. (2001). WMRC global E-government survey. Retrieved February 20, 2004, from http://www.insidepolitics.org/egovt01.html

West, D. (2002). State and federal e-government in the United States. Providence, RI: Brown University. Retrieved from http://www.insidepolitics.org/ egovt02city.html

West, D. (2003). State and Federal E-Government in the United States. Retrieved from http://www.insidepolitics.org/egovt03us.html

West, D. (2004). E-government and the transformation of service delivery and citizen attitudes. Public Administration Review, 64(1), 15-27.

West, D. (2006). State and federal e-government in the United States. Providence, RI: Brown University. Retrieved from http://www.insidepolitics.org/ egovt06int.pdf

329 Winzer, M. A. (1993). The history of special education: From isolation to integration. Washington, DC: Gallaudet University Press.

Witt, N., & McDermott, A. (2004). Website accessibility: What logo will we use today? British Journal of Educational Technology, 35, 45–56. www.health.gov. (2009). Accessibility. Retrieved March 27, 2011, from http://www.hhs.gov/Accessibility.html

Xiong, L. (2008). Web accessibility study of U.S. based hospitality websites. Thesis, University of Delaware. Retrieved from http://gradworks.umi.com/14/57/1457442.html

Yukins, C. (2004). Making federal information technology accessible: A case study in social policy and procurement. Public Contract Law Journal, 34.

Zeng, X., & Parmanto B. (2003). Evaluation of web accessibility of consumer health information websites. Proceedings of the AMIA Symposium, 743-747.

330 BIOGRAPHICAL SKETCH

JOHN L. BROBST

Mr. John L. Brobst has attained the following degrees: Doctor of Philosophy in Information Studies (Florida State University, Tallahassee, Florida, 2012), Master of Science in Information Management (Syracuse University, Syracuse, New York; 2004), Master of Business Administration in Finance (Kent State University, Kent, Ohio; 1976) and Bachelor of Science in Mathematics (Kent State University, Kent, Ohio; 1974). Prior to beginning his doctoral studies in 2007, Mr. Brobst held various leadership positions in information technology while working for the federal government. In 2007, Mr. Brobst began pursuing a doctorate degree at the Florida State University‘s College of Communication & Information. His areas of interest include federal information policy and the impact of technology on society. His dissertation research relates to the usability of federal health care websites for individuals with disabilities. His academic research interests include health care information systems, disability rights, and trends in library management. While pursuing his PhD, Mr. Brobst worked as a Research Associate at the Information Institute staff member from 2007 - 2011. During that time, he primarily performed grant research related to library management, information policy, website usability, and electronic government. Several of his research efforts have focused on issues and trends that currently affect our nation‘s libraries, to include service roles related to disaster preparedness. These efforts resulted in enhancing collaboration and communication using websites and web based technologies. Mr. Brobst‘s published works include articles, book chapters, and abstracts covering topics related to website usability and accessibility, federal disability policy, and library management.

331