Virtual Display of Tactile Graphics and Braille by Lateral Skin Deformation
Total Page：16
File Type：pdf, Size：1020Kb
Virtual Display of Tactile Graphics and Braille by Lateral Skin Deformation Vincent Lévesque Department of Electrical & Computer Engineering McGill University Montreal, Canada August 2009 A thesis submitted to McGill University in partial fulfillment of the requirements for the degree of Doctor of Philosophy. © 2009 Vincent Lévesque i Abstract Graphical content is increasingly pervasive in digital interfaces and documents yet it remains accessible to visually impaired persons almost exclusively on media with limited flexibility such as embossed paper. Textual content is more accessible but nevertheless limited by the cost and functionality of refreshable Braille displays and voice synthesis. This thesis explores the use of a novel tactile stimulation approach that relies on lateral skin deformation for the computerized display of virtual Braille and tactile graphics. Tactile synthesis by lateral skin deformation is initially explored in the context of Braille. The feasibility of producing virtual Braille by laterotactile stimulation is first demonstrated by creating the illusion of brushing against a line of Braille dots through the synchronization of a travelling wave of skin deformation with the displacement of a tactile array of eight actuators. This principle is then extended to complete 6- dot Braille cells by distributing lines of virtual dots onto the rows of actuators of a general-purpose STReSS2 tactile array. Reading the resulting virtual Braille is shown to generally be feasible but demanding, suggesting that a specialized laterotactile Braille display should be devised or that dots be rendered for contrast rather than realism. Tactile rendering by lateral skin deformation is then further explored with the grad- ual development of a virtual tactile graphics framework that emulates conventional features such as raised lines and areal textures through a coherent set of patterns that includes grating textures, stroked and dotted shapes, bitmap-based masks, and com- posite patterns. Dynamic rendering is also exploited to produce novel effects such as tactile flow, reactive textures dependent on the exploration behaviour, and interactive content with alternate views. The usability of the framework is informally evaluated with visually impaired volunteers and early tactile patterns studied through formal ex- periments. The tactile patterns are presented on the Tactograph, a haptic interface redesigned specifically for the display of tactile graphics that combines a STReSS2 dis- play with an instrumented planar carrier. This thesis demonstrates the potential of lateral skin deformation for the display of Braille and tactile graphics, and explores in the process the ways in which this novel approach to tactile stimulation can be applied to produce meaningful tactile sensations. ii Sommaire Le graphisme gagne en présence dans les interfaces et documents numériques mais est accessible aux personnes non-voyantes quasi-exclusivement sur media à flexibilité limitée tel que le papier embossé. Le texte est plus accessible mais néanmoins lim- ité par le coût et les fonctionnalités des afficheurs Braille et de la synthèse vocale. Cette thèse explore l’utilisation d’une nouvelle approche pour la stimulation tactile opérant par déformation latérale de la peau pour l’affichage informatisé de Braille et de graphiques tactiles virtuels. La synthèse tactile par déformation latérale de la peau est initialement explorée dans le contexte du Braille. La faisabilité de produire du Braille virtuel par stimu- lation latérotactile est d’abord démontrée en créant une illusion de frottement contre une ligne de points par la synchronisation d’une onde progressive de déformation de la peau avec le déplacement d’une plage tactile à 8 actionneurs. Ce principe est en- suite étendu au Braille à 6 points par la distribution de lignes de points virtuels sur les rangées d’actionneurs d’un afficheur tactile STReSS2. La lecture de ce Braille virtuel est généralement faisable mais exigeante, suggérant qu’un afficheur Braille latérotac- tile soit conçu ou que le rendu des points vise le contraste plutôt que le réalisme. Le rendu tactile par déformation latérale de la peau est ensuite exploré plus à fond avec le développement graduel d’un système de graphique tactile virtuel qui émule des éléments conventionnels tels que les traits surélevés et les surfaces texturées à travers un ensemble cohérent de motifs incluant des textures à rayure, des formes tracées ou pointillées, des masques par image, et des motifs composites. Le rendu dynamique est ensuite exploité pour produire de nouveaux effets tel que le flot tactile, les textures réactives répondant au comportement d’exploration, et le contenu interac- tif avec vues alternatives. L’ergonomie du système est évaluée informellement avec des volontaires non-voyants et les premiers motifs tactiles étudiés à travers des expéri- ences formelles. Les motifs tactiles sont présentés sur le Tactograph, une interface haptique reconçue spécifiquement pour l’affichage de graphiques tactiles qui combine un afficheur STReSS2 à un charriot planaire instrumenté. Cette thèse démontre le potentiel de la déformation latérale de la peau pour l’affichage de Braille et de graphiques tactiles, et explore ainsi comment cette approche à la stim- ulation tactile peut être appliquée à la production de sensations tactiles signifiantes. iii Acknowledgments Foremost, I would like to thank my supervisor, Prof. Vincent Hayward, without whom this work would not have been possible. His support, encouragement and creativity have made this challenging experience both pleasant and rewarding. This thesis also rests on an innovative technology that was developed under his supervision by Christine Desmarais, Juan Manuel Cruz-Hernandez, Jérôme Pasquero, and Qi Wang. I am indebted to them all for their dedication and hard work, and wish to particularly thank Qi Wang for his help and support with the latest prototype. I have had the chance to work alongside many inspiring and helpful colleagues during my years in the Haptics Laboratory. Jérôme Pasquero has been a valuable col- laborator and friend, and has influenced this work through innumerable discussions. Andrew Gosline patiently shared his mechanical engineering skills and helped me with the design of the Tactograph. Mounia Ziat’s expertise was invaluable for the supplementary statistical analysis presented in Chapters 4 and 6. I would also like to thank other past and present members of the Haptics Lab and Center for Intelligent Machines for their support and friendship (in alphabetical order): Omar Ayoub, Gi- anni Campion, Hanifa Dostmohamed, Diana Garroway, Prasun Lala, Akihiro Sato, Qi Wang, and Hsin-Yun Yao. I’m especially grateful to Sandra Skaff with whom I’ve shared much of this experience and who has proved a supportive and valuable friend. I would also like to thank Jean-Samuel Chenard for sharing his electronics ex- pertise and saving in extremis a demonstration of the tactile graphics system, and Don Pavlasek of the Mechanical Workshop (ECE dept.) for his advice and meticu- lous work. This work also benefited from a collaboration with Gregory Petit (UdeM), Prof. Aude Dufresne (UdeM), Nicole Trudeau and Pierre Ferland (INLB), with whom I’ve had numerous interesting discussions about tactile graphics. I would also like to thank Maryse Legault (HumanWare), Chantal Nicole (INLB) and Marie-Chantal Wanet-Defalque (INLB) for their assistance. VisuAide, now HumanWare Canada, also provided the initial impetus for this research and donated experimental hardware used for the early work on Braille. Last but not least, I wish to thank my parents, Suzanne and Viateur Lévesque, my sister, Danny Lévesque, as well as my family and friends for their support and encouragement throughout these years. iv Contents 1 Introduction 1 2 Assistive Technologies for Visually Impaired Persons 8 2.1 Introduction . .8 2.2 Haptics . .9 2.3 Blindness and Visual Impairment . 10 2.3.1 Definition, Prevalence and Causes . 10 2.3.2 Neurology and Psychology . 11 2.4 Technological Aids . 13 2.4.1 Orientation and Mobility . 13 2.4.2 Reading and Writing . 17 2.4.3 Graphics . 20 2.4.4 Human-Computer Interaction . 23 2.4.5 Vision Substitution . 25 2.4.6 Sight Recovery . 26 2.5 Discussion . 27 2.5.1 Assistive Technologies Market . 27 2.5.2 Recommendations . 28 2.5.3 Trends and Future Developments . 29 2.5.4 Applications of Haptics . 30 2.6 Conclusion . 32 3 Display of Virtual Braille Dots by Lateral Skin Deformation: Feasibility Study 34 3.1 Introduction . 37 Contents v 3.1.1 Braille Displays . 37 3.1.2 Alternative Technologies . 38 3.1.3 Overview . 38 3.2 Virtual Braille Display . 39 3.2.1 Device . 39 3.2.2 Skin Deformation Patterns . 43 3.3 Parameter Tuning . 46 3.3.1 Method . 47 3.3.2 Procedure . 48 3.3.3 Results . 48 3.3.4 Discussion . 49 3.4 Virtual Braille Legibility . 50 3.4.1 Method . 50 3.4.2 Results and Discussion . 51 3.5 Control Experiment . 55 3.5.1 Method . 55 3.5.2 Results and Discussion . 57 3.6 Conclusion and Future Work . 58 4 Braille Display by Lateral Skin Deformation with the STReSS2 Tactile Transducer 61 4.1 Introduction . 64 4.2 Virtual Braille Rendering . 65 4.2.1 Hardware . 65 4.2.2 Skin Deformation Patterns . 66 4.3 Virtual Braille Legibility . 69 4.3.1 Method . 69 4.3.2 Results . 71 4.4 Discussion . 75 4.5 Conclusion . 76 5 A Haptic Memory Game using the STReSS2 Tactile Display 78 5.1 Introduction . 81 5.2 Technology . 81 Contents vi 5.3 Tactile Rendering . 82 5.4 Haptic Memory Game . 83 5.5 Conclusion . 84 6 Tactile Graphics Rendering Using Three Laterotactile Drawing Primitives 86 6.1 Introduction . 89 6.2 Tactile Display Prototype . 91 6.3 Tactile Rendering . 92 6.3.1 Dot Rendering . 94 6.3.2 Vibration Rendering . 94 6.3.3 Grating Rendering . 95 6.3.4 Composite Rendering . 95 6.4 Experiments . 96 6.4.1 Shape . 96 6.4.2 Grating Spatial Frequency .