Expanding Information Access Through Data-Driven Design
Total Page:16
File Type:pdf, Size:1020Kb
©Copyright 2018 Danielle Bragg Expanding Information Access through Data-Driven Design Danielle Bragg A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy University of Washington 2018 Reading Committee: Richard Ladner, Chair Alan Borning Katharina Reinecke Program Authorized to Offer Degree: Computer Science & Engineering University of Washington Abstract Expanding Information Access through Data-Driven Design Danielle Bragg Chair of the Supervisory Committee: Professor Richard Ladner Computer Science & Engineering Computer scientists have made progress on many problems in information access: curating large datasets, developing machine learning and computer vision, building extensive networks, and designing powerful interfaces and graphics. However, we sometimes fail to fully leverage these modern techniques, especially when building systems inclusive of people with disabilities (who total a billion worldwide [168], and nearly one in five in the U.S. [26]). For example, visual graphics and small text may exclude people with visual impairments, and text-based resources like search engines and text editors may not fully support people using unwritten sign languages. In this dissertation, I argue that if we are willing to break with traditional modes of information access, we can leverage modern computing and design techniques from computer graphics, crowdsourcing, topic modeling, and participatory design to greatly improve and enrich access. This dissertation demonstrates this potential for expanded access through four systems I built as a Ph.D. student: 1) Smartfonts (Chapter 3), scripts that reimagine the alphabet’s appearance to improve legibility and enrich text displays, leveraging modern screens’ ability to render letter- forms not easily hand-drawn, 2) Livefonts (Chapter 4), a subset of Smartfonts that use animation to further differentiate letterforms, leveraging modern screens’ animation capability to redefine limitations on letterform design, 3) Animated Si5s, the first animated character system prototype for American Sign Language (ASL) (Chapter 5), which uses animation to increase text resem blance to live signing and in turn improve understandability without training, and 4) ASL-Search, an ASL dictionary trained on crowdsourced data collected from volunteer ASL students through ASL-Flash, a novel crowdsourcing platform and educational tool (Chapters 6-7). These systems employ quantitative methods, using large-scale data collected through existing and novel crowd- sourcing platforms, to explore design spaces and solve data scarcity problems. They also use human-centered approaches to better understand and address usability problems. TABLE OF CONTENTS Page List of Figures .......................................... iii Glossary ............................................. vii Chapter 1: Introduction ................................... 1 Chapter 2: Related Work .................................. 7 2.1 The Psychophysics of Reading ............................ 7 2.2 English Character Design ............................... 10 2.3 ASL Displays ..................................... 13 2.4 ASL Lookup, Recognition, and Translation . 17 2.5 Crowdsourced Perception Studies .......................... 20 2.6 Participatory Design ................................. 22 Chapter 3: Smartfonts ................................... 23 3.1 Our Smartfonts .................................... 25 3.2 Optimizations ..................................... 29 3.3 Legibility Evaluation ................................. 34 3.4 Learnability Evaluation ................................ 40 3.5 Discussion and Future Work ............................. 44 Chapter 4: Livefonts .................................... 47 4.1 Livefont Designs ................................... 49 4.2 Legibility Study .................................... 53 4.3 Learnability Study .................................. 62 4.4 Discussion and Future Work ............................. 68 i Chapter 5: Animated Si5s ................................. 70 5.1 Opportunity Evaluation Study ............................ 71 5.2 Animation Design ................................... 79 5.3 Design Probe Workshop ............................... 83 5.4 Discussion and Future Work ............................. 89 Chapter 6: ASL-Search and ASL-Flash .......................... 93 6.1 Survey of Search Methods .............................. 95 6.2 ASL-Search ...................................... 97 6.3 ASL-Flash ....................................... 101 6.4 Proof of Concept for ASL-Search Algorithm . 104 6.5 Discussion and Future Work ............................. 113 Chapter 7: ASL-Search and ASL-Flash Scalability Exploration . 115 7.1 Scaled ASL-Flash Design ............................... 116 7.2 Preliminary Scalability Study ............................. 118 7.3 Preliminary Results: ASL-Flash at Scale ....................... 120 7.4 Preliminary Results: ASL-Search at Scale . 122 7.5 Discussion and Future Work ............................. 126 Chapter 8: Conclusion ................................... 128 8.1 Summary of Contributions .............................. 128 8.2 Future Vision ..................................... 130 8.3 Takeaways ....................................... 132 Bibliography .......................................... 134 Appendix A: Supplementary Materials ............................ 149 A.1 Proof: Selecting k nodes of a graph with minimal sum of edge weights is NP-hard . 149 A.2 Proof: Partitioning a graph into three sets of nodes with minimal sum of edge weights within sets is NP-hard ............................ 149 ii LIST OF FIGURES Figure Number Page 3.1 The words message and massage clear (left) and blurred (right) in our Smartfonts: (a) Tricolor (b) Logobet (c) Polkabet (on black). Words are sized to have equal areas. 23 3.2 A smartwatch displaying an SMS. The sender is oblivious to the fact that the SMS is read in a Smartfont.................................. 24 3.3 Our Smartfonts designed to improve legibility of small, blurry text. 26 3.4 Mnemonics for Polkabet’s small square characters. 28 3.5 Example of aggressive kerning with the Logobet alphabet. 29 3.6 Our confusion matrix was generated by testing a series of rows of random charac ters, like a Snellen chart. ............................... 30 3.7 Our confusion matrix of 2x3 characters - a matrix showing the frequency with which each displayed character was identified as every other character. 31 3.8 The 26 selected shapes (black) of the 42 considered. Although the confusion ma trix is high-dimensional (as measured by eigenvalues), D3’s force-directed graph layout [21] displays many confusable pairs near one another. Edges denote con fusable pairs, which repel each other in the physic simulator creating the force directed layout. .................................... 32 3.9 A D3 [21] force-directed layout of Tricolor attempts to place similar pairs of letters near one another, and also illustrates the color assignments from our optimization algorithm. Edges denote commonly confused pairs, which repel each other in the force-directed layout. ................................. 33 3.10 Sample legibility task with Smartfont Visibraille. 36 3.11 The (smoothed) empirical distribution function of log-MRA (at blur of radius 3.5 pixels) for the fonts. y is the fraction of participants whose log-MRA was larger than x.......................................... 38 3.12 Histogram of log-scores for Tricolor with blur radius 3.5 pixels. 39 3.13 Sample practice question for Smartfont Logobet. (Do fish wear clothing? Yes/No) . 41 3.14 Smartfont response time, normalized by individual average Latin time. Each point is the median of a sliding window of averages across participants to remove outliers. 43 iii 4.1 The word “livefonts” rendered in: a standard font (Verdana, top), a font designed for low vision readers (Matilda, second), a Smartfont from the previous chapter, and our Livefont (bottom). Note: figures should appear animated when this docu ment is viewed with Adobe Reader. 48 4.2 Our Livefont versions (top: Version 1, bottom: Version 2), with descriptions of character color and animation. Characters are colored, animated blocks, chosen to be maximally distinguishable by our color/animation perception study. 49 4.3 Animation/color perception study task. A target block is shown. The participant is asked to identify the target color (red selected) and animation (quick jump selected). 51 4.4 The static Smartfonts (letters A-E) to which we compared Livefont Version 1. 53 4.5 Transcription task with Livefont Version 1 – a target string (and partial guess), with a visual keyboard of Livefont Version 1 characters for transcription. (See Figure 4.2 for the mapping of Version 1 characters to the Latin A-Z.) . 56 4.6 Scanning task with Latin (Verdana Bold). The target string (top) is a random 5 character string. The selected matching string in the random pseudo-sentence is outlined in white (bottom). 57 4.7 Transcription results. Box-plots of participants’ smallest legible size for that script, normalized by their smallest legible Latin size. Lower means more legible. 59 4.8 Distribution of errors in identifying a) colors and b) animations of Livefont Version 1 across all participants in the transcription task. Each row shows the breakdown of mistakes in identifying that color or animation. (See Figure 4.2 for Version 1 alphabet.) . 60 4.9 Scanning results. Box-plots of participants’