Tactile Line Drawings for Improved Shape Understanding in Blind and Visually Impaired Users
Total Page：16
File Type：pdf, Size：1020Kb
Tactile Line Drawings for Improved Shape Understanding in Blind and Visually Impaired Users ATHINA PANOTOPOULOU, Dartmouth College and Boston University, USA XIAOTING ZHANG and TAMMY QIU, Boston University, USA XING-DONG YANG, Dartmouth College, USA EMILY WHITING, Boston University, USA Fig. 1. We present a novel approach for generating tactile illustrations to improve shape understanding in blind individuals. (a) Physical 3D object (3D printed). (b) The input to our pipeline is a pre-partitioned object, colors indicate segmented parts. (c) A local camera is assigned to each part, with a master camera to combine the resulting multi-projection image. (d) Resulting stylized illustration. Cross-sections are used for texturing the interior of each part to communicate surface geometry. (e) We evaluated the technique in a user study with 20 blind participants. Tactile illustrations were fabricated using microcapsule paper. Members of the blind and visually impaired community rely heavily on Additional Key Words and Phrases: fabrication, design, accessibility, tactile tactile illustrations – raised line graphics on paper that are felt by hand – to shape perception, tactile images, non-photorealistic rendering understand geometric ideas in school textbooks, depict a story in children’s books, or conceptualize exhibits in museums. However, these illustrations ACM Reference Format: often fail to achieve their goals, in large part due to the lack of understanding Athina Panotopoulou, Xiaoting Zhang, Tammy Qiu, Xing-Dong Yang, and Emily in how 3D shapes can be represented in 2D projections. This paper describes Whiting. 2020. Tactile Line Drawings for Improved Shape Understanding a new technique to design tactile illustrations considering the needs of blind in Blind and Visually Impaired Users. ACM Trans. Graph. 39, 4, Article 89 individuals. Successful illustration design of 3D objects presupposes identif- (July 2020), 13 pages. https://doi.org/10.1145/3386569.3392388 cation and combination of important information in topology and geometry. We propose a twofold approach to improve shape understanding. First, we 1 INTRODUCTION introduce a part-based multi-projection rendering strategy to display geo- metric information of 3D shapes, making use of canonical viewpoints and Understanding the 3D geometry of everyday objects via 2D media is removing reliance on traditional perspective projections. Second, curvature essential to live, learn, and work. Designers communicate the shape information is extracted from cross sections and embedded as textures in of a product (e.g., a chair) via sketches and renderings. Consumers our illustrations. understand the design by browsing images of the product in a cata- logue. Students learn physics, such as why a plane is shaped in a CCS Concepts: • Human-centered computing ! Accessibility systems and tools; • Computing methodologies ! Shape analysis; Perception; certain way for aerodynamics, from fgures in a textbook. Visual Non-photorealistic rendering. perception of complex 3D geometry from a 2D projection is taken for granted by people with healthy vision. However, for people with Authors’ addresses: Athina Panotopoulou, [email protected], Dart- near or total blindness, understanding 3D geometry of daily objects mouth College and Boston University, USA; Xiaoting Zhang, [email protected]; Tammy Qiu, [email protected]; Emily Whiting, [email protected], Boston University, USA; Xing- from current media is extremely challenging. Dong Yang, [email protected], Dartmouth College, USA. In this paper we introduce a novel approach to generating tactile that aid 3D shape understanding in users with near or Permission to make digital or hard copies of all or part of this work for personal or line drawings classroom use is granted without fee provided that copies are not made or distributed total blindness, enabling the blind community to perceive complex for proft or commercial advantage and that copies bear this notice and the full citation 3D objects from 2D media. Members of the blind community rely on the frst page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, heavily on tactile illustrations, defned as raised graphics (e.g. on to post on servers or to redistribute to lists, requires prior specifc permission and/or a paper) that are felt by hand. Common uses of tactile illustrations fee. Request permissions from [email protected]. are to make visual information accessible in textbooks, maps [Brock © 2020 Association for Computing Machinery. 0730-0301/2020/7-ART89 $15.00 et al. 2015], scientifc diagrams [Brown and Hurst 2012], children’s https://doi.org/10.1145/3386569.3392388 literature [Claudet et al. 2008; Stangl et al. 2014; Stangl 2019], or ACM Trans. Graph., Vol. 39, No. 4, Article 89. Publication date: July 2020. 89:2 • Panotopoulou et al. • We present results of a formative study involving sculpting 3D objects, aiming to discover salient information for blind users in haptic shape understanding (Sec. 3.1). • We present results of a second formative study on the suc- cess of existing 2D tactile illustration guidelines for shape understanding (Sec. 3.2). • We introduce a new design methodology for tactile illustra- tions based on the outcome of our formative studies, apply- ing techniques in multi-projection rendering and geometry- Fig. 2. Tactile illustrations (a,c) displayed alongside exhibits in the Museum aware textures (Sec. 4). of Fine Arts Boston (b,d) for accessibility purposes (Photographs ©Museum • We design and implement a user study evaluating the ability of Fine Arts, Boston, www.mfa.org). Black areas in illustrations indicate for blind users to understand 3D shape with our new illustra- raised regions on the paper surface that can be perceived by touch. tion style (Sec. 5). 2 RELATED WORK museum exhibits. The tactile illustration in Figure 2 shows two rep- resentative samples, both used to give access to artifacts displayed Our approach to tactile graphics is motivated by cognitive princi- in the Museum of Fine Arts Boston. ples in tactile perception and design principles of stylized graphic To design these types of tactile illustrations, designers rely on illustration. guidelines provided by a number of associations, such as the Braille Authority of North America [2010]. Hundreds of pages of rules Tactile Perception. Mental representations in shape understand- make manual design time consuming. For example, rules related to ing are an active area of research in psychology and neurobiology, size, placement, variety, and form of diferent elements (e.g. lines, particularly concerning similarities in how humans perceive shapes textures, points, labels, titles, and captions) permit easier explo- visually and through touch [Farley Norman et al. 2004; James et al. ration. Rules related to simplifcation, separation, elimination, and 2002; Lakatos and Marks 1999]. A prominent theory on shape un- distortion of the illustration avoid clutter. Rules related to semantic derstanding is “recognition by components" [Biederman 1987; Blake segmentation, keying of the design, and selection of viewpoints and Sekuler 2006], suggesting that our visual system processes shape permit easier understanding. information by representing an object as a spatial arrangement of Unfortunately, studies have shown that shape understanding is 3D parts called geons (generalized cones). Our visual system extracts fawed in traditional tactile diagrams, which can be seen from low geons and their arrangement from an image to recognize the object. object identifcation scores [Klatzky et al. 1993]. We aim to improve Visual translation theory hypothesizes that haptic perception is shape understanding by re-evaluating graphic design considerations translated to the same visual representations [Cooke et al. 2007; and accounting for the unique needs of blind individuals. For exam- Erdogan et al. 2014; Handjaras et al. 2016; Lederman and Klatzky ple, while current tactile illustrations are created from a standard 2009], while other work suggests vision and haptics are functionally single viewpoint, a blind individual who has never had access to overlapping but do not necessarily have equivalent representations visual information might fnd it unnatural to explore a visual image of 3D shapes [Farley Norman et al. 2004; James et al. 2002; Lakatos that includes perspective distortions. This is supported by drawings and Marks 1999]. Our proposed work on illustration style abides created by blind individuals [Kennedy 1993], in which illustrations by visual shape recognition theory, where pictorial representations of 3D objects appear to use a “fold-out” method to arrange faces should clearly show subcomponents and their arrangement in space onto a fat surface [Kurze 1997]. [Thompson et al. 2006]. In this paper we frst present two formative studies that moti- vated our approach to tactile illustration. We investigate salient Tactile illustrations. Tactile illustrations are typically made by ex- shape properties in tactile perception, and assess the strengths and pert designers in a time consuming process, involving identifcation weaknesses of existing tactile illustration styles. Next, we propose a and application of principles from existing guidelines (e.g. Braille novel method to render tactile graphics with a focus on improving Authority of North America [2010]). Rules relate to use of points, shape understanding