Exploring Improvements for Software Developers who are Blind

Guarionex Salivia* Flint Million Minnesota State University-Mankato Minnesota State University-Mankato

Megan Bening Minnesota State University-Mankato

Abstract: Software developers who are blind and interact with the computer non-visually face unique challenges with information retrieval. We explore the use of speech and Braille combined with software to provide an improved interface to aid with challenges associated with information retrieval. We motivate our design on common tasks performed by students in a software development course using a Microprocessor without Interlocked Pipeline Stages (MIPS) architecture simulation tool. We test our interface via a single-subject longitudinal study, and we measure and show improvement in both the user’s performance and the user experience.

Keywords: ; Blindness; Visual Impairments; Braille; Screen reader technology; Software development

*Corresponding Author, Guarionex Salivia [email protected] Submitted October, 2019 Accepted February 4, 2020 Published online July 23, 2020

1

Vol. 23, No. 1 - 2020; Journal of Science Education for Students with Disabilities

1. INTRODUCTION problem, because these interfaces provide information contextually via visual cues, People who are blind face unique challenges as which are clearly not specifically designed for software developers. Screen reading individuals who are blind and may not work technology is often unable to adequately with existing accessibility tools such as screen represent the complex and dynamic nature of readers. Therefore, those who are blind or modern application interfaces often used for visually impaired often find IDEs to be software development. Complex interfaces challenging to work with, and in many make the task of information seeking and situations completely unusable even with the retrieval particularly challenging to developers use of screen reading technology (Lazar, Allen, who are blind. Our main goal is to help Kleinman, & Malarkey, 2007). individuals who are blind with the problem of information retrieval in a non-visual manner In our work, we explore the use of Braille as a from a . We are possible option to improve accessibility for particularly interested with graphical user software developers who are blind. One reason interfaces related to software development we concentrate on Braille as an option is based such as text editors and integrated on reports by the American Printing House for development environments (IDEs). the Blind. Out of their 27,212 student members Information retrieval has been shown to be a currently enrolled in elementary through post- prevalent problem amongst individuals who secondary education, 3,781 considered Braille are blind interacting with technology, to be their primary reading medium (American especially software developers (Mealin & Printing House for the Blind, 2013). When we Murphy-Hill, 2012). There are an estimated 19 look at the intersection between software million software developers worldwide, with developers and users of Braille, we see that an expected increase to at least 25 million by there is a big potential for Braille-based user the year 2020 (Evans Data Corporation, 2014). interfaces to aid with information retrieval. Out of the plethora of software developers, we Half of the participants in a study by Mealin do not know exactly how many of them are and Murphy-Hill (2012) reported using a blind or visually impaired. However, Braille display for software development. according to a survey conducted by Stack Furthermore, though Braille is introduced Overflow, a prominent software development early in education, we have not seen recent resource website, more than 600 respondents work on developing more effective use of identified as blind (Stack Overflow, 2017). Braille when interacting with computers.

Modern software development tools are quite In this work, we develop a software add-on to different from those used in years past. Today, an educational programming tool used in a IDEs provide highly dynamic, rich interfaces university course to add functionality that for developers. These interfaces facilitate provides a Braille user with enhanced ability to productivity through their extensive locate information and interact with the user visualization capabilities and organizational interface. A developer is usually aware of the display of content. This augments the information they need to find. For developers complexity of the information retrieval who are not blind, IDEs provide information

2

Exploring User Interface Improvements for Software Developers who are Blind

contextually via visual cues and layout access and use computers at roughly the same allowing the developer to visually locate performance level as their sighted peers desired information while simultaneously (Alexander, 1998). ignoring less important content. On the other technology provided auditory access to the hand, users who are blind make use of proxies computer’s output, and the linear nature of such as the to navigate the information commands lent itself well to the linear nature that is displayed. A contextual layout of of spoken word access. information does not help a user who is blind because he or she would have to navigate away As graphical user interfaces (GUIs) became from the current position to find the relevant more prevalent, people who are blind quickly information, and then relocate to their previous found themselves less and less able to work focus manually. Our software allows the user with the computer with the same efficiency, to specify contextual information from the speed and capability as their sighted peers visual user interface (UI) that will be displayed (Lazar et al., 2007; Alexander, 1998). While in Braille regardless of where the user’s cursor screen-reading and Braille display access is positioned, thus facilitating information technology has continuously been adapted and retrieval. Additionally, the software assists the improved, many such efforts at improving developer with parsing complex data structures accessibility of GUIs require software represented in code that are very difficult to developers to follow prescribed standards, interpret with speech alone. often via specialized Application Programming Interfaces (), to allow their We perform a user experience study to measure software to be used effectively and efficiently the subjective and objective change in our by people who are blind. Initiatives such as student participant’s performance while Microsoft’s Active Accessibility were an early interacting with an IDE which is part of a attempt to standardize access to graphical learning tool that utilizes the Microprocessor elements by users with disabilities (Lazar et al., without Interlocked Pipeline Stages CPU 2007). However, no standard to implement architecture (MIPS). By assisting with such APIs was in place at the time. In fact, information retrieval, we expect that our implementing accessibility required additional method will improve student developers’ efforts on the part of the software publisher. abilities to complete tasks in a reasonable time Due to the time and effort required, these and improve their experience while working in efforts were focused on mainstream development environments. and not on software development tools. In a CNN interview, Curtis 2. RELATED WORK Chong, former president of the National Federation of the Blind’s Science Division, Historically, computer interfaces were largely reported his own personal struggles with the character-based. When display devices were advancing GUI technology as a software developed, the interface presented on screen developer (Alexander, 1998). He specifically was linear, textual, and command based. When stated that his performance decreased as a computer interfaces were text-based, users result of the graphical tool kit requiring you to who are blind found that they were able to drag and drop items on the screen.

3

Vol. 23, No. 1 - 2020; Journal of Science Education for Students with Disabilities

The primary method of accessing GUIs for Braille terminal for computers available in the computer users who are blind is the screen United States (Van Gerven & Taylor, 2009), reader. Screen readers have become extremely based on a patent received in 1975 by the prevalent among users who are blind. A survey founder of Handy Tech of Germany (Handy taken by WebAIM ( in Mind, Tech, n.d.). Refreshable Braille displays are a non-profit organization) found that 93% of devices that display the transmitted text in an respondents reporting a significant visual array of electronic Braille cells. These cells use impairment were regular users of screen mechanical actuators, often using piezoelectric reading technology (WebAIM, 2013). technology, to allow them to instantly produce any Braille character, and just as quickly Screen readers were designed to read and "refresh" themselves to display a different interpret common application interfaces. The character. This allows the user to access GUIB (Textual and Graphical User Interfaces computer information via tactile output, using for Blind People) and Mercator projects are the Braille written language. Additionally, early efforts to provide accessibility of many contemporary Braille displays can also graphical user interfaces (GUIs) to visually function as input devices, allowing the user to impaired users (Mynatt & Weber, 1994; input information into the computer using the Edwards, Mynatt, & Stockton, 1994). Braille language. Research efforts to provide non-visual GUI accessibility have shifted to web-based Also, during the 1980s, Braille "notetaker" interfaces as web-application development devices were introduced, providing the user became more prevalent. (Takagi, Saito, Fukuda, with the ability to both input text using the & Asakawa, 2007; Lazar et al., 2007; K. & C., Braille code as well as read content using both 2014). synthetic speech and refreshable Braille. A popular early device was known as the Braille Screen readers extend the command set used to n’Speak and was developed by Blazie interact with the computer by adding Engineering. Notetakers used preinstalled commands which are specific to the needs of custom software designed and written speech output. Such commands may include specifically for their use case. These devices announcing the text on which the cursor is allowed a Braille user to perform functions currently located, announcing the position of similar to a PDA device such as maintaining an the cursor, and changing the rate or volume of address book and calendar and taking notes but the synthesized speech. did not provide access to mainstream computing interfaces on their own. The use of Braille to enhance accessibility to computers for individuals who are blind has As screen reader software was developed and been explored since the 1970s. In 1971, the improved, the ability to represent content in first Braille embossing device was released. Braille as well as via speech was developed. Braille embossers allow a person to produce Popular screen readers such as Job Access hard-copy Braille on paper from a computer. In With Speech (JAWS), Non-Visual Desktop 1982, Telesensory Inc. released the Access (NVDA) and Apple’s VoiceOver have VersaBraille, the first electronic refreshable Braille output support. The simplest mode of

4

Exploring User Interface Improvements for Software Developers who are Blind

operation is to simply display in Braille the developers in the study reported that the only same text which is being spoken aloud. way these developers have to alleviate this However, because a Braille display is simply a challenge is to take notes on another device or row of independently addressable Braille cells, in another program. This method, however, is there exists significant potential to improve not interactive and does not allow the accessibility by directly interacting with the developer to view changes to screen content Braille cells in novel ways. Screen readers do without again navigating to the content. Efforts take advantage of this for some use cases, such have been made to improve the accessibility of as "status cells" which allow a small number of complex graphical user interfaces by analyzing cells to be designated such that they display users’ abilities. Automatically generated various codes contextually relevant to the interfaces were developed specifically for current operation (Freedom Scientific, 2014). users with motor impairments (Wobbrock, For example, a status cell might indicate Kane, Gajos, Harada, & Froehlich, 2011). We whether the user is currently focused on a have not seen any such developments aimed at check box, button, or text field. users with blindness or visual impairments. We explored the possibility of a tactile method Despite many years of innovation and further of assisting developers with locating and developments in screen reader technology, a keeping up with screen content. Tactile software developer who is blind still faces graphics have been used in many educational unique challenges today. Access to information contexts, including geography and on a computer by a person who is blind is often mathematics. They have also been used as an linear (Edwards, Mynatt, & Stockton, 1994; orientation aid (Chen & Takagi, 2014). Most WebAIM, 2013). Using a screen reader, the implementations of tactile graphics involve a individual will access information either hard-copy printout onto paper or plastic of an audibly, as a stream of spoken words and/or image, with its lines or patterns represented as sound effects, or through a Braille display raised bumps or textures. While a printout of device. Modern integrated development an IDE’s user interface may assist a developer environments (IDEs) typically expose many with learning where on the screen important types of contextually sensitive information at content is displayed, we did not find that this the same time on the screen. These developers approach would be beneficial, because simply may find it challenging, frustrating, or even knowing the on-screen location of the content impossible to keep up with the flood of in two-dimensional space will not assist the information while seeking out relevant bits of developer using a screen reader with accessing content and ignoring repetitive, unrelated and that information quickly on demand. A printout unimportant information (Lazar et al., 2007). is also not dynamic and cannot contextually An exploratory study by Mealin and Murphy- change in real time as refreshable Braille can. Hill (2012) found that one of the most significant challenges software developers 3. METHODOLOGY who are blind face is that of information For our initial examination into the potential seeking—being able to locate relevant improvement in the efficiency and user information quickly without needing to experience of visually impaired users when navigate away from the current focus. The

5

Vol. 23, No. 1 - 2020; Journal of Science Education for Students with Disabilities

using Braille as an addition to auditory screen only the speech output offered by NVDA, the readers, we performed a single-subject screen reader used in the test. Each test was longitudinal study. Single-subject longitudinal selected at random from the pool of 24 tests studies produce statistically significant results until all 24 tests were completed, thus abiding in studies that compare alternative treatments by an AB testing design. The participant was (Edgington, 1967; Onghena, 1992). We not aware prior to the beginning of each test designed an ABAB test with a single-subject which task they were to be performing or we recruited, after obtaining approval from the whether they would be using the Braille IRB at Minnesota State University, Mankato. display or not. Our participant is a student who is blind, is a proficient Braille reader and has taken several The experiment took place over the course of software development courses. The participant three months. One test was performed every is female, in the range of 18-24 years of age, other day during the week. Each test lasted and was a senior at Minnesota State University, between 15 and 20 minutes. During each test, Mankato at the time of testing. She uses Braille the participant was directly observed while in her daily life as a primary means of performing the prescribed software accessing written information. The participant development task, and notes were taken on makes use of a Braille display device on a these observations. Following each session, the regular basis as well as a screen reader to participant responded to a survey meant to access the computer. gauge the subjective feeling of frustration or confusion that the participant felt while 3.1 Experiment performing the tests. The participant answered The study consisted of asking the participant to these questions after each session through a perform four different common software Qualtrics survey: development tasks. We performed an • How familiar were you with the exploration of introductory textbooks on concept(s) involved with performing software development. Our inquiry revealed a the task? set of tasks which are taught in many languages (Likert scale, 1 = least familiar, 4 = and development environments. The tasks we most familiar) selected to be performed by the participant • How difficult did you find the task to were derived from these common tasks. These be? (Likert scale, 1 = least difficult, 4 = are: most difficult) 1. reading the standard output of a • Rate your frustration level while program, performing the task on the following 2. determining the value of a variable, scale (Likert scale, 1 = least frustrated, 3. reviewing code to locate a bug, and 5 = most frustrated). 4. single-stepping through a program and • Do you feel you were able to complete monitoring its actions (including the the task in a reasonable time period? use of variable watches). (Yes/No) Each task was performed six times: three times • Did you use Braille during this session? with the use of a Braille display and our (Yes/No) software enhancements, and three times using

6

Exploring User Interface Improvements for Software Developers who are Blind

• (If Braille was used) Do you feel that changing at each iteration of the loop. your frustration was influenced by the The participant iterates through the use of the Braille display? (Yes/No) loop five times and records the initial • (If Braille was not used) Do you feel value plus the five values collected that your experience would have during the iterations into a document. improved had you used the Braille display? (Yes/No) For each task, the development environment We chose a four-point Likert scale for was prepared with pre-written code. In each questions related to the participant’s case, certain aspects of the code are knowledge and difficulty because we randomized before each test: the output of the did not consider a neutral answer (3) program, the computations performed on the relevant to these questions. variable, the value assigned to the variable, and the computations performed within the loop, 3.2 Tasks respectively. The participant was timed The following tasks were performed during the beginning when they press their first keystroke test by our participant: and ending when they complete the final step • Task 1: The participant was asked to in each task, which is to return focus to the run a program and copy the output of main application window after entering their the program to the Clipboard, and results into a document. subsequently paste that output into a document. When the Braille display was in use, each use of a specific user interface enhancement • Task 2: The participant was asked to run a program and input a number. The functionality as provided by our add-on was program then performed computations recorded as listed in Section 3.3. on that number and stored the result in a variable. The participant was then 3.3 Software asked to locate that variable in the The software selected for testing is used in the IDE’s "Registers" view and write its course IT 320 at Minnesota State University, value into a document. Mankato. We used a program emulator • Task 3: The participant was asked to package for the MIPS architecture, curiously locate a line of code in the IDE which titled "PC-SPIM” (MIPS backwards). The assigns a value to a variable. The software provides an accompanying integrated participant was asked to record the development environment (IDE) for value that was assigned to the variable programming the simulation in assembly in a document. language. (See Figure 1) • Task 4: The participant was presented with a debugging scenario in which an When accessed with a traditional screen reader infinite loop occurs. The participant without any additional tools, the entire content was asked to stop the program and and controls of PC-SPIM can be accessed. As single-step through it, reporting the is visible in Figure 1, all content is textual, and value of a variable at each step, and thus can be presented verbally. However, the indicating how that variable is design of the user interface has not been

7

Vol. 23, No. 1 - 2020; Journal of Science Education for Students with Disabilities

Figure 1. The PC-SPIM user interface optimized in any way to assist those using some of the shortcomings we are attempting to . This results in potential address with our solution. significant difficulty for the student in navigating the application and accessing For the screen reader, Non-Visual Desktop information being presented by the user Access (NVDA) was chosen. Its open source interface. Because screen readers and speech nature and use of modern accessibility synthesizers are generally optimized for standards made it an excellent choice for reading English text, many aspects of the pairing the additional Braille technologies with interface and code are read in a confusing or PC-SPIM. cumbersome manner. For example, the string “0($29)” seen in the code window above is For Braille-enhanced tests, we designed a recited as “0, left parenthesis, twenty-nine custom software interface, known to NVDA as dollars, right parenthesis”. Similarly, the an “add-on”, which interacts with both the hexadecimal addresses are read out character IDE’s user interface and the Braille display by by character, for example “zero x zero zero using the Python scripting component of the four zero zero zero zero eight”, and the NVDA screen reader. This software provides assembly language mnemonics are read as if access to various components of the IDE in they are words; the “addiu” instruction is real-time, allowing the software developer to recited similar to the word “ado”. These are quickly access information without navigating through the IDE’s numerous displays. This

8

Exploring User Interface Improvements for Software Developers who are Blind

Figure 2. An example of how Braille is used to represent source code and status indicators in the IDE. Text below the Braille cells indicates the displayed characters technique is related to the “status cells” information is organized and presented functionality that is offered by screen readers, on a line-by-line basis. but in this case it is specifically designed to • Provide quick, direct navigation to the address the needs of a software developer various displays of the IDE. using the PC-SPIM software. In addition, the • Copy output of any display to the custom software provides quick access to Clipboard. This feature is not available many IDE features directly from the Braille in the standard IDE without using the display. mouse and visually selecting and copying the text. The interface provides features specific to All of the above features are accessed via Braille as follows: additional keystrokes and/or Braille • Display the value of a variable or commands. Each time the participant executes register in a static, non-changing area of any of the above functions during a task, the the Braille display, allowing the user to incident is recorded. The total number of times navigate away from the content while all features are uses is recorded. Figure 2 keeping the value displayed on the illustrates how we display both source code Braille display. and internal register information in Braille. • Display the value of a variable or register in real time in an area of the 3.4 Hardware Braille display, allowing the user to step The Braille display device used for the study is through a program and observe the the Freedom Scientific Focus 80 Blue. It has variable’s changes over time. This is 80 display cells as well as Braille keyboard analogous to the Watch feature of input functions. It interacts with the NVDA modern IDEs. screen reader and the add-on we have written • Parse the source code in the editor and to provide access to the PC-SPIM software. present it in a format well suited for (See Figure 3) reading in Braille. Disorganized

Figure 3. A Freedom Scientific Focus-80 Braille display, as used in the experiment 9

Vol. 23, No. 1 - 2020; Journal of Science Education for Students with Disabilities

The screen reader and Braille device were used Table 2. Usage counters for Braille helper functions. with an iMac desktop computer running Function Task Task Task Task in a virtualized 1 2 3 4 environment. The keyboard used was the Copy To Clipboard 4 standard wired Apple keyboard. The mouse is not used in the experiment. Set Focus 9

4. FINDINGS Show Register Values 3 3

4.1 Time Freeze Register Values 3 1

We have observed an average improvement in Parse Code 2 speed of 2.75x faster when the participant was using the Braille display. The average time to completion for all tasks with the use of the 4.3 User Experience additional Braille hardware and software was 50.1 seconds, while the average completion without was 2 minutes and 17.8 seconds. The Kolmogorov-Smirnov normality test indicated that trials both with and without Braille had p = .2, verifying that the data is normally distributed. Thus, we performed an independent-sample T-test of use of Braille against time. Braille usage had a significant effect in favor of reduced time of completion with p < .001. Our data is illustrated in Table 1.

Table 1. Average time taken in seconds for the participant to complete the given task. Figure 4. Task completion times in seconds for Braille and non-Braille trials.

The reported frustration level by the participant was significantly higher when not using the Braille display. On a Likert scale of 1 to 5, with 5 being the most frustrated and 1 being the least, the participant provided a response of 1 (least frustrated) for every task 4.2 Braille Add-on Functionality Usage which included use of the Braille display. The We measured the usage of the functions we average frustration level without Braille was provided in the Braille add-on to determine 4.2. A Chi-Square test of the reported how they were used. Task 4 used the Braille frustration level showed a significant effect of functions the most of any task, and also used Braille in reducing the frustration level with p the most diverse set of these functions. < .001.

10

Exploring User Interface Improvements for Software Developers who are Blind

The participant found that using the Braille presented in a monospace font with spacing display made a difference in the perceived used to separate them. This caused the screen difficulty of the tasks. On a Likert scale of 1 to reader to repeatedly read irrelevant 4, with 1 being least difficult and 4 being most information, thus confusing the participant difficult, the reported average difficulty of the when she was trying to locate the requested tasks without using the Braille display was information. 3.25. With Braille, the average difficulty was 1.1, with the participant reporting mild With the additional control afforded by the difficulty on only one task. A Chi-Square test Braille display, the participant was visibly of the reported difficulty level showed a more relaxed and enthusiastic about the tests. significant effect of Braille in reducing the The participant quickly and easily located key perceived difficulty of the tasks with p < .001. pieces of information. When using Braille, the participant felt they completed the tasks in a timely manner in all 4.5 Limitations cases. Without Braille, the participant felt that completion time was too long in all but one case. A Chi-Square test showed that Braille had a significant effect in favor of the perceived reasonable time to completion with p < .001.

In all cases, the participant indicated that she felt the use of Braille was beneficial and would choose to use it if it were available. These results are illustrated in Figure 4 and Figure 5.

4.4 Observations From the notes taken during each test, we observed that when not using the Braille display, the participant exhibited many outward signs of frustration such as the use of uncouth language and facial expressions of aggression. In many cases, the participant needed to back up and repeat parts of the task due to losing her current place in the user interface, yet another sign of undesirable user experience outcome.

The screen reader also did not present the content to the participant in an effective way. For example, the variables were presented as a column list in a text window, but without Figure 5. Result averages from Qualtrics survey. Lower bars column separation. The columns were instead mean better performance and experience.

11

Vol. 23, No. 1 - 2020; Journal of Science Education for Students with Disabilities

The PC-SPIM software is of limited practical 5.1 Future Work value and is mostly used as a teaching tool for To further the practical applications of this computer science courses. While adapting the research, we will conduct a study with a larger PC-SPIM interface is a useful endeavor for the number of participants to evaluate desirable academic sector, this study could be expanded and undesirable user experience outcomes to account for more powerful IDEs such as from the use of software development GUIs Microsoft’s Visual Studio or Apple’s Xcode. with and without Braille. This study will utilize a mainstream IDE such as Visual Studio. While a single-subject longitudinal study may provide generalizable results for quantifiable 6. REFERENCES variables, it is not adequate for explaining the user experience of the highly diverse Alexander, S. (1998, November). Blind population of individuals with visual programmers face an uncertain future. impairments. CNN.com. Retrieved from http://www.cnn.com/TECH/computing/9 Finally, the cost of assistive technology is often 811/06/blindprog.idg/ prohibitive; the cost of the Focus 80 Blue American Printing House for the Blind. product used during the study is $7,795.00 at (2013). Annual report October 1, 2012- the time of this writing. The high cost of this September 30, 2013. Retrieved from equipment can limit its availability; therefore, http://www.aph.org/files/annual- even though this study has demonstrated reports/APH-Annual-Report-FY13. significant benefits from using the Braille display, the cost may limit its application in a Chen, J., & Takagi, N. (2014). Development real-world setting. of a system to assist automatic translation of hand-drawn maps into tactile graphics 5. CONCLUSIONS and its usability evaluation. Advances in This study serves as a demonstration of the Fuzzy Systems, 2014, 1–11. doi: need for more intense and detailed evaluation 10.1155/2014/357380 of the advantages that can be provided to Edgington, E. S. (1967). Statistical inference software developers when using Braille and from n = 1 experiments. The Journal of accompanying software to aid in information Psychology, 65(2), 195-199. doi: retrieval. 10.1080/00223980.1967.10544864

The GUI is not going away, but technologies Edwards, W. K., Mynatt, E. D., & Stockton, such as Braille displays and the accompanying K. (1994). Providing access to graphical software modules, which provide customized user interfaces—not graphical screens. In interfaces for them, can help give individuals Proceedings of the first annual acm who are blind the best chance of remaining conference on assistive technologies (p. competitive in the ever-evolving technology 47–54). New York, NY, USA: industry. Association for Computing Machinery. doi: 10.1145/191028.191041

12

Exploring User Interface Improvements for Software Developers who are Blind

Evans Data Corporation. (2014). Global Mynatt, E. D., & Weber, G. (1994). Nonvisual developer population and demographic presentation of graphical user interfaces: study vol. 2. Retrieved from Contrasting two approaches. In https://evansdata.com/reports/viewReleas Conference companion on human factors e.php?reportID=9 in computing systems (p. 211). New York, NY, USA: Association for Computing Freedom Scientific. (2014). Focus 40 Blue Machinery. doi: 10.1145/259963.260338 refreshable braille display user’s guide. Retrieved from National Federation of the Blind. (2014). https://www.freedomscientific.com/Conte Blindness statistics: Statistical facts nt/Documents/Manuals/Focus about blindness in the United States. /Focus40Blue/Focus-40-Blue-Online- National Federation of the Blind. Users-Guide.htm Retrieved from https://nfb.org/blindness- statistics Handy Tech. (n.d.). Company history. Retrieved from Onghena, P. (1992). Randomization tests for https://handytech.de/en/info/handy- extensions and variations of abab single- tech/history case experimental designs: A rejoinder. Behavioral Assessment, 14(2), 153-171. K., M. H., & C., S. (2014, Sep). Use of information and communication Stack Overflow. (2017). Developer survey technology by visually-impaired students: results 2017. Retrieved from A study in University of Calicut, Kerala. https://insights.stackoverflow.com/survey DESIDOC Journal of Library & /2017 Information Technology, 34(4), 342–348. doi: 10.14429/djlit.34.6586 Takagi, H., Saito, S., Fukuda, K., & Asakawa, C. (2007, September). Analysis of Lazar, J., Allen, A., Kleinman, J., & Malarkey, navigability of web applications for C. (2007). What frustrates screen reader improving blind usability. ACM Trans. users on the web: A study of 100 blind Comput.-Hum. Interact., 14(3), 13–es. users. International Journal of Human– doi: 10.1145/1279700.1279703 Computer Interaction, 22(3), 247-269. doi: 10.1080/10447310709336964 Van Gerven, C., & Taylor, A. (2009). The information age braille technology Mealin, S., & Murphy-Hill, E. (2012). An timeline. Future Reflections, 28(1). exploratory study of blind software Retrieved from https://www.nfb.org/sites/ developers. In 2012 IEEE symposium on www.nfb.org/files/images/nfb/publication visual languages and human-centric s/fr/fr28/fr280109.htm computing (VL/HCC) (p. 71-74). WebAIM. (2013). Visual disabilities: How Microsoft Corporation. (2018, May). blind people use the web. Author. Microsoft active accessibility technical Retrieved from overview. Retrieved from https://webaim.org/articles/visual/blind https://msdn.microsoft.com/en- us/library/dd373660.aspx

13

Vol. 23, No. 1 - 2020; Journal of Science Education for Students with Disabilities

WebAIM. (2015). Screen reader user survey #6 results. Author. Retrieved from https://webaim.org/articles/visual/blind Wobbrock, J. O., Kane, S. K., Gajos, K. Z., Harada, S., & Froehlich, J. (2011, April). Ability-based design: Concept, principles and examples. ACM Trans. Access. Comput., 3(3). doi: 10.1145/1952383.1952384 APPENDIX The source code used in this experiment is available at the following repository: https://www.github.com/fmillion/pcspim-nvda

The source code for this project is licensed under the GNU General Public License (GPLv3). Source code has been updated to function with the current version of NVDA, which is 2019.1 at the time of this writing.

14