DOCSLIB.ORG

Automatic Speech Recognition for the Hearing Impaired in an Augmented Reality Application

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Automatic Speech Recognition for the Hearing Impaired in an Augmented Reality Application Automatic Speech Recognition for the Hearing Impaired in an Augmented Reality Application Anja Virkkunen School of Science Thesis submitted for examination for the degree of Master of Science in Technology. Espoo 19.11.2018 Supervisor Prof. Mikko Kurimo Advisors D.Sc. Kalle Palom¨aki M.Sc. Juri Lukkarila Copyright © 2018 Anja Virkkunen Aalto University, P.O. BOX 11000, 00076 AALTO www.aalto.fi Abstract of the master’s thesis Author Anja Virkkunen Title Automatic Speech Recognition for the Hearing Impaired in an Augmented Reality Application Degree programme Computer, Communication and Information Sciences Major Machine Learning, Data Science and Artificial Code of major SCI3044 Intelligence Supervisor Prof. Mikko Kurimo Advisors D.Sc. Kalle Palom¨aki,M.Sc. Juri Lukkarila Date 19.11.2018 Number of pages 90+11 Language English Abstract People with hearing loss experience considerable difficulties in participating and understanding spoken communication, which has negative effects on many aspects of their life. In many proposed solutions to the problem, the deaf or hard of hearing person has to take their attention away from the speaker. As a consequence, the hearing impaired miss for instance gestures and expressions of the speaker. This thesis studied the use of augmented reality and automatic speech recognition technologies in an assistive mobile application for the hearing impaired. The appli- cation uses mobile augmented reality with video-based augmentations. Automatic speech recognition is done using modern neural network models. In the implementa- tion, automatic speech recogniser transcriptions were placed in speech bubbles on top of an augmented reality view of the conversation partner. This minimised the distance between the speaker and the transcriptions which help the hearing impaired follow the conversation. To validate the usefulness of the approach, user tests were organised with hearing impaired participants. The results show that the deaf and hard of hearing found the augmented reality view and the application helpful for following conversations. The most requested improvements by the user testers were support for visual separation and identification of speakers in group conversations and higher speech recognition accuracy. Keywords augmented reality, speech recognition, mobile development, user testing, hearing impairment , assistive technology Aalto-yliopisto, PL 11000, 00076 AALTO www.aalto.fi Diplomityön tiivistelmä Tekijä Anja Virkkunen Työn nimi Automaattinen puheentunnistus kuuroille ja huonokuuloisille lis¨atyn todellisuuden sovelluksessa Koulutusohjelma Computer, Communication and Information Sciences Pääaine Machine Learning and Data Mining Pääaineen koodi SCI3044 Työn valvoja Prof. Mikko Kurimo Työn ohjaajat TkT Kalle Palom¨aki,DI Juri Lukkarila Päivämäärä 19.11.2018 Sivumäärä 90+11 Kieli Englanti Tiivistelmä Huonokuuloisilla ja kuuroilla ihmisill¨a on huomattavia vaikeuksia keskusteluihin osal- listumisessa ja niiden ymm¨art¨amisess¨a, joka laskee heid¨an el¨am¨anlaatuaan monella tavalla. Suuressa osassa ongelmaan tarjotuista ratkaisuista kuurot ja huonokuuloi- set joutuvat siirt¨am¨a¨an huomionsa pois puhujasta. T¨all¨oin kuulovammainen ei n¨ae esimerkiksi puhujan eleit¨a ja ilmeit¨a. T¨ass¨a ty¨oss¨a tutkittiin lis¨atyn todellisuuden ja automaattisen puheentunnistuksen hy¨odynt¨amist¨a huonokuuloisille ja kuuroille tarkoitetussa avustavassa sovelluksessa. Sovellus k¨aytt¨a¨a video- ja mobiilipohjaista lis¨atty¨a todellisuutta. Puheentunnistuk- sessa hy¨odynnet¨a¨an moderneja neuroverkkomalleja. Toteutuksessa automaattisen puheentunnistuksen tulokset sijoitettiin puhekupliin videokuvassa n¨akyv¨an puhujan kasvojen l¨ahelle. N¨ain kuuro tai huonokuuloinen k¨aytt¨aj¨a pystyi helposti seuramaan sek¨a puhujaa ett¨a puheentunnistustuloksia. Sovelluksen hy¨odyllisyytt¨a arvioitiin j¨arjest¨am¨all¨a k¨aytt¨aj¨atestej¨a kuuroille ja huonokuuloisille. Tulosten perusteella huonokuuloiset ja kuurot kokivat lis¨atyn todellisuuden ja sovelluksen auttavan keskustelujen seuraamisessa. Testik¨aytt¨ajien eniten toivomia pa- rannuksia olivat eri puhujien puheentunnistustulosten visuaalinen erottelu toisistaan ja parempi puheentunnistustarkkuus. Avainsanat lis¨atty todellisuus, puheentunnistus, mobiilikehitys, k¨aytt¨aj¨atestaus, kuulovauriot, apuv¨alineteknologia 5 Preface First, I would like to thank supervisor Prof. Mikko Kurimo for his feedback, patience and understanding during the whole process. I would like to thank advisors D.Sc. Kalle Palom¨akiand M.Sc. Juri Lukkarila for letting me work on this project and for all they have done to help me with this work. I am extremely lucky to have had two people to guide me and give me new perspectives. I am grateful to all the deaf and hard of hearing who participated and gave valuable feedback in the user tests and to people from Kuuloliitto ry for their help in recruiting people to the user tests. I would like to thank Katri Leino, Juho Leinonen, Peter Smit, Tuomas Kaseva, Zhicun Xu, Mittul Singh, Aku Rouhe and Reima Karhila from the Speech Recognition group for general help, ideas, conversations and good company. Additionally, I would like to thank Tarmo Simonen and Aleksi Oyry¨ from the Aalto University, for providing me all the necessary equipment. I am thankful to the Speech Recognition research group of the Department of Signal Processing and Acoustics at the Aalto University School of Electrical Engi- neering and the Academy of Finland for supporting and enabling this thesis as part of the project Conversation Assistant for the Hearing Impaired. Last but not least, I would like to express my gratitude to Jarkko and my family for believing in me, cheering me forward and supporting me in the moments of despair until the very end. Otaniemi, 19.11.2018 Anja Virkkunen 6 Contents Abstract3 Abstract (in Finnish)4 Preface5 Contents6 Symbols and abbreviations8 1 Introduction9 1.1 Augmented reality............................9 1.2 Conversation assistant.......................... 10 1.3 Research goals............................... 11 1.4 Thesis structure.............................. 11 2 Background 12 2.1 Augmented reality............................ 12 2.1.1 Definition............................. 13 2.1.2 Human factors.......................... 15 2.1.3 Technology............................ 20 2.1.4 Past and present AR systems.................. 24 2.1.5 Challenges for AR usage and adoption............. 25 2.2 Automatic speech recognition...................... 26 2.2.1 The structure of an ASR system................. 27 2.2.2 Conversational speech...................... 31 2.3 Hearing impairment............................ 32 2.3.1 Hearing loss types......................... 33 2.3.2 Diagnosis and treatment..................... 35 2.3.3 Societal effects.......................... 35 2.4 Visual dispersion............................. 36 2.5 Previous work............................... 38 3 Augmented Reality Conversation Assistant 41 3.1 Description................................ 41 3.2 System structure............................. 42 3.3 Software.................................. 45 3.3.1 iOS application.......................... 46 3.3.2 ASR model and server...................... 49 7 4 User testing 51 4.1 Objectives................................. 51 4.2 Test plan................................. 52 4.2.1 Introduction............................ 53 4.2.2 Getting to know the application................. 53 4.2.3 Section 1: Word explanation................... 53 4.2.4 Section 2: Conversation..................... 54 4.2.5 Conclusion............................. 54 4.3 Questionnaire............................... 54 4.4 Arrangements............................... 55 4.4.1 Test environment setup...................... 58 4.4.2 Participants............................ 60 5 Results 62 5.1 Word explanation and conversation tasks................ 63 5.2 Final ratings................................ 67 5.3 Written feedback............................. 71 5.4 Discussion on results........................... 73 6 Conclusions 75 References 77 A Questionnaire form 91 B Questionnaire answers 101 8 Symbols and abbreviations Abbreviations AR Augmented reality ASR Automatic speech recognition AV Augmented virtuality AVSR Audiovisual speech recognition CE Central executive FOV Field-of-view GMM Gaussian Mixture Model GPS Global positioning system HMD Head-mounted display HMM Hidden Markov Model IT Information technology LSTM Long short-term memory MFCC Mel frequency cepstral coefficient MR Mixed reality MVC Model-view-controller OST Optical see-through RSD Retinal scanning display TDNN Time-delay neural network TLS Transport Layer Security TTS Text-to-Speech UI User interface VR Virtual reality VST Video see-through WM Working memory 9 1 Introduction Hearing loss in all of its forms can significantly limit the access to auditory information and the ability to communicate. Participating in conversations and social life becomes a struggle, especially in noisy environments, causing exhaustion, withdrawal and a poorer quality of life [1,2,3]. Working life and education are affected as well, with hearing impaired often quitting schools and working life earlier than their hearing peers [4, 5, 6, 7]. Different studies estimate 10 to 20 percent of the population to have hearing loss [8, 9], which means it is a considerable societal issue in addition to complicating individual lives. Moreover, hearing loss has the highest prevalence in elderly population, so the number of hearing impaired is
Load more

Recommended publications
  • Liquid Crystal Display Module
    LIQUID CRYSTAL DISPLAY MODULE Product Specification DENSITRON STANDARD LCD MODULE PRODUCT LM/TS 3234 – LM/TS 4234 – LM/TS 6234 NUMBER DEFINITION Date Display 240*128 dots 19/04/04 INTERNAL APPROVALS Quality Mgr Product Mgr Project Leader Mech. Eng Electr. Eng Date: Date: Date: Date: Date: DENSITRON TECHNOLOGIES plc. – PROPRIETARY DATA – ALL RIGHTS RESERVED TABLE OF CONTENTS 1 PART NUMBER DESCRIPTION FOR AVAILABLE OPTIONS....................................................................................4 2 MAIN FEATURES..................................................................................................................................................................5 3 MECHANICAL SPECIFICATION ......................................................................................................................................6 3.1 MECHANICAL CHARACTERISTICS..........................................................................................................................6 3.2 MECHANICAL DRAWING...........................................................................................................................................7 4 ELECTRICAL SPECIFICATION........................................................................................................................................8 4.1 ABSOLUTE MAXIMUM RATINGS.............................................................................................................................8 4.2 ELECTRICAL CHARACTERISTICS............................................................................................................................8
    [Show full text]
  • Lcd Product Specification
    LCD PRODUCT SPECIFICATION PART NUMBER: USMPG-TQ240128D-SZWBI 240x128 Graphic LCD; STN Blue Display DESCRIPTION: Mode; Transmissive, Negative, with LED Sidelight and 6 O'Clock Viewing Direction. ISSUE DATE APPROVED BY CHECKED BY PREPARED BY (Customer Use Only) PROPRIETARY THIS SPECIFICATION IS THE PROPERTY OF US MICRO PRODUCTS AND SHALL NOT BE REPRODUCED OR NOTE: COPIED WITHOUT THE WRITTEN PERMISSION OF US MICRO PRODUCTS AND MUST BE RETURNED TO US MICRO PRODUCTS UPON ITS REQUEST. www.usmicroproducts.com CONFIDENTIAL (800) 741-7755 USMPG-TQ240128D-SZWBI 1.Features a) +5V power supply b) Built-in controller (RA6963L2NA) c) 240x128 dots graphic LCD module d) STN Blue mode, transmissive; neagative display e) Viewing direction: 6:00 O’clock f) 1/128 duty cycle g) (white)LED backlight 2.Outline dimension CONFIDENTIAL 3.Absolute maximum ratings Item Symbol Standard Unit Power voltage VDD-VSS 0 - 7.0 V Input voltage VIN VSS - VDD Operating temperature range VOP -20 - +70 ℃ Storage temperature range VST -30 - +80 www.usmicroproducts.com 1 (800) 741-7755 USMPG-TQ240128D-SZWBI 4.Block diagram /WR COM /RD LCD PANEL /CE 240X128 Dots C/D COM /RST DB0~DB7 RA6963 FS 13 COL COL COL VDD 8 RAM V0 VSS 2 FG VEE DC/DC LEDA LED SIDE(WHITE) 5.Interface pin description Pin External CONFIDENTIAL Symbol Function No. connection 1 VSS Signal ground for LCM (GND) 2 VDD Power supply Power supply for logic (+5V) for LCM 3 V0 Operating voltage for LCD 4 C/D MPU H: Instruction L: Data 5 /RD MPU Read enable signal 6 /WR MPU Write enable signal 7~14 DB0~DB7 MPU Data bus line 15 /CE MPU Chip enable signal 16 /RST MPU Reset signal 17 VEE Negative voltage output 18 MD2 Selection of number of columns:H-32,L-40 19 FS MPU Font selection: H=6x8 dot matrix, L=8x8 dot matrix 20 LEDA BKL power supply Power supply for BKL(+5.0V) Contrast adjust DC/DC build in VDD~V0: LCD Driving voltage VR: 10k~20k www.usmicroproducts.com 2 (800) 741-7755 USMPG-TQ240128D-SZWBI 6.Optical characteristics θ φ 1 φ1 2 θ2 12:00 9:00 3:00 6:00 STN type display module (Ta=25℃, VDD=5.0V) Item Symbol Condition Min.
    [Show full text]
  • 4957 User Manual
    4957D/E/F Microwave Analyzer User Manual China Electronics Technology Instruments Co., Ltd. Contents Foreword Thanks very much for purchasing and using the 4957D/E/F microwave analyzer produced by China Electronics Technology Instruments Co., Ltd. The Company follows the ISO9000 standard always during production and adheres to the customer-oriented and quality-first principles. Please read carefully the manual, so as to facilitate the operation. We will spare no efforts to meet your requirements, providing you with operating devices with the higher cost performance and better after-sales services. We always adhere to the principle of“Good Quality, Satisfied Service”, providing customers with satisfactory products and services as well as work convenience and shortcut. The hotline is shown below, and we are looking forward to hear from you: Qingdao Tel: +86-0532-86896691 Website: www.ceyear.com E-mail: [email protected] Address: No. 98, Xiangjiang Road, Qingdao City, China Postal code: 266555 The manual introduces the application, performance and features, basic principles, usage, maintenance and precautions of the 4957D/E/F microwave analyzer, facilitating your quick understanding the usage and key points of the tester. To operate the product well and provide you with higher economic effectiveness, please read the manual carefully. Mistakes and omissions can hardly be avoided due to time urgency and writer’s knowledge limit. Your valuable comments and suggestions are highly welcome! We are sorry for any inconvenience caused due to our errors. The manual is the third version of the 4957D/E/F microwave analyzer Manual, with version number of C.2. The contents hereof are subject to change without notice.
    [Show full text]
  • Prepared by Dr.P.Sumathi
    COMPUTER GRAPHICS 18BIT53C UNIT I: Overview of Graphics System – Display Devices – CRT – Random Scan and Raster Scan Monitors – Techniques for Producing Colour Display – Beam – Penetration and Shadow – Mask Methods – DVST – Plasma – Panel Displays – Hardcopy Devices – Printers and Plotters – Display Processors – Output Primitives – DDA and Bresenham’s line drawing algorithms – Antialiasing lines – Bresenham’s Circle Algorithm – Character Generation. UNIT II: Two-dimensional Transformations – Scaling, Translation and Rotation – Matrix Representations – Composite Transformations – Reflection – Shearing – Other Transformations. Windowing and Clipping – Concepts – Cohen and Sutherland Line Clipping Algorithm – Midpoint Subdivision. UNIT III: Three dimensional Concept- Three-Dimensional object representations – polygon surfaces – polygon tables- plane equations - Three-Dimensional geometric transformations – translation – rotation – scaling – other transformations. UNIT IV: Three-Dimensional viewing – viewing pipeline - Display Techniques – Parallel Projection – Perspective Projection – Hidden-Surface and Hidden-Line removal – Back face removal – Depth Buffer Method – Scan Line Method – BSP Tree Methods – Depth-Sorting Method – Area-subdivision Method – Octree Methods – Comparison of Hidden-Surface Methods. UNIT V: Colour models and colour applications – properties of light – standard primaries and the chromaticity diagram – xyz colour model – CIE chromaticity diagram – RGB colour model – YIQ, CMY, HSV colour models, conversion between HSV and RGB models, HLS colour model, colour selection and applications. TEXT BOOK 1. Donald Hearn and Pauline Baker, “Computer Graphics”, Prentice Hall of India, 2001. Prepared by Dr.P.Sumathi 1 COMPUTER GRAPHICS Computer graphics is an art of drawing pictures on computer screens with the help of programming. It involves computations, creation, and manipulation of data. In other words, we can say that computer graphics is a rendering tool for the generation and manipulation of images.
    [Show full text]
  • Se-7034-19 Htv-Ru710seriesdsht
    RU710 Series Hospitality TVs 43" HG43RU710NFXZA 55" HG55RU710NFXZA 50" HG50RU710NFXZA 65" HG65RU710NFXZA Make every guest RU710 Series Hospitality TVs 4K UHD Edge-Lit LED TVs with outstanding detail and color. experience an Make sure your guest rooms remain the showcases of elegance you’ve designed them to be. The Samsung RU710 hospitality TVs look stunning before they’re even turned on, with unsurpassed one. stylish stands that look good from every angle, and a clean cable management system. Once your guests do power them on, the 4K UHD screens display content with exceptional detail and amazingly rich, vibrant color. And Samsung’s LYNK™ DRM technology protects content, while offering guests a seamless viewing experience. Install a premium experience in every room. Install the Samsung RU710 series TVs. Key Features 4K UHD with PurColor for Breathtaking Color Deliver the unmatched, luxury experience your guests expect with 3840 x 2160 4K UHD resolution. Offering four times the pixels of Full HD, the RU710 series showcases content in stunning, lifelike detail. PurColor technology concentrates on red, green, and blue color adjustment points, but unlike traditional LED, PurColor also adjusts magenta, yellow and cyan. The result is fine-tuned color of incredible nuance and realism. Easier, Faster Control with Home App and Eden UI The Home app installed on the RU710 allows you to easily incorporate a welcome message, add logos/branding, and provide other useful information such as phone extensions to various guest services. Displayed on the Eden user interface, it’s a sleek, fast, simple solution. HDR10+ Picture Technology for Incredible Depth 4K UHD delivers more pixels, and HDR10+ gets the most out of every one.
    [Show full text]
  • LCD Component Data Sheet Model Number: 12864-12
    PACIFIC DISPLAY DEVICES LCD Component Data Sheet Model Number: 12864-12 128 x 64 Dot STN Graphic LCD Assembly With T-6963C Graphics Controller LED / EL Panel Backlight CONTENTS 1. GENERAL INFORMATION 1.1 Product Overview 2 1.2 Part Options and Numbering System 2 1.3 Absolute Maximum Ratings 3 1.4 Circuit Block Diagram 3 1.5 Mechanical Characteristics 3 1.6 Input Signal Function 4 1.7 LCM Power, Contrast Control and Bias 4 1.8 LCD Dimensions 5 2. ELECTRICAL / OPTICAL CHARACTERISTICS 2.1 DC Electrical Characteristics 6 2.2 AC Electrical Characteristics 7 2.3 Optical Characteristics 9 2.4 LED Backlight Characteristics 11 2.5 EL Panel Backlight Characteristics 11 3. LCD CONTROLLER CHARACTERISTICS 3.1 LCD Controller Display and Control Functions (T6963) 12 3.2 LCD Controller Command List 25 3.3 LCD Controller Character Code map 26 3.4 Application Circuits 27 4. RELIABILITY 28 5. PRECAUTIONS FOR USING LCD MODULES 29 139 Avenida Victoria – Suite 100 – San Clemente, CA – 92672 Tel: 949-361-8957 – Fax: 949-361-9158 – Web: www.pacificdisplay.com SPECIFICATIONS FOR LIQUID CRYSTAL DISPLAY MODULE MODEL NO: 12864-12 1. GENERAL INFORMATION 1.1 Product Overview • 128 x 64 dot matrix LCD • STN (Super Twisted Nematic) or FSTN (Film compensated Super Twisted Nematic) Technology • T-6963 (or equivalent) Graphics Controller IC w/ 8K SRAM. • Multiplex drive : 1/64 duty, 1/9 bias • LCD Module Service Life: 100,000 hours minimum 1.2 Part Options and Numbering System 12864 -12 -SL -F -ST -ELED -GY -6 ¾ Custom Option Designator: • (-12) T6963 Graphic Controller
    [Show full text]
  • Information Display • the Journal of Data Display Technology
    whatever your computer says ... Volume9 Number6 November/December 1972 Information Display • The journal of Data Display Technology PANEL DISPLAYS I") I ') I.J \ll z ,-. "t V l -J SELF-SCAN Panel Display Subsystem, 256"Ch.aracter . 7 0 t M t ix Characters 0.25 Htgh. ~:?c~~;~~~e~ lor~at, !~ailable with all drive ele_ctronics including memory, character generator, and ttmmg. Whether it's a police officer. mapa. 1 ~o 1ca r requestingd. t s informa-for h·t t speclfymg coor ma e lion on a suspect, an arc I ~clding or a type setter verifying type plumbing fixtures m a ne;E~~-SCAN panel displays guarantee for a mall-order catalogf f data between the operator and the most accurate trans er o uter dis lay and demand the computer. If yo~ ne~d a ~o~tw the le:d of Kustom Electronics error-free commun l cat~~n~, oce Accessories Corporation (1) in Chanute, Kansas, Cle ~ i Ke boards (3) in (2) in Southport, .connectlcdu:;,:~~~ ~th:r SELF-SCAN panel Belleview, Washmgton, an salesman demonstrate his displa.y a why the SELF-SCAN panel us~rs. H~ve Bu.~r~u~'~lssee Circle Reader Service Ca rd No. 25 a bnefcaffse .t. oman/machine interface device d"teisprmmlay ~ISlm th e most e ec lve available today. Burroughs Corpora t1.on, Electronic Components Division, Burroughs Plainfield, New Jersey 07061 . (201) 757-3400. ~~ A Technology Publ ishing Corp. Publicat ion Volume9 Number6 November/December 1972 Information Display The journal of Data Display Technology The Sykes 2220 Cassette Tape Controller receives, stores and transmits data between termi­ nals and communications lines all day long.
    [Show full text]
  • Final Report
    Color Dot Matrix Proof of Concept A Major Qualifying Project Report Submitted to the faculty Of the Worcester Polytechnic Institute Worcester, MA In partial fulfillment of the requirements for the Degree of Bachelor of Science By ________________________ Stephen Hansen Electrical and Computer Engineering ʻ12 ________________________ Brandon Rubadou Electrical and Computer Engineering ʻ12 Project Advisor: Professor Stephen Bitar Electrical and Computer Engineering Table of Contents Acknowledgements!.........................................................1 Abstract!............................................................................1 Introduction!......................................................................1 1. Background!..................................................................2 1.1 Defining Goals..................................................................................! 2 1.2 Background Research!.....................................................................3 1.2.1 Motion Sensing LED Panels!......................................................................3 1.2.2 Dot Matrices!................................................................................................4 1.2.3 Peggy!...........................................................................................................6 2. Design Details!............................................................10 2.1 Choosing the Design!.....................................................................10 2.1.1 High Resolution LED Display
    [Show full text]
  • Electronic Displays Technical Report
    California Energy Commission DOCKETED 12-AAER-2A TN 72475 Electronic DisplaysJAN 08 2014 Technical Report - Engineering and Cost Analysis In Support of the Codes and Standards Enhancement (CASE) Initiative For PY 2013: Title 20 Standards Development Supplemental to CASE Report submitted on July 29, 2013 Docket #12-AAER-2A January 8, 2014 Prepared for: PACIFIC GAS & ELECTRIC SOUTHERN CALIFORNIA SAN DIEGO GAS AND SOUTHERN CALIFORNIA COMPANY EDISON ELECTRIC GAS COMPANY Prepared by: Clancy Donnelly, Katherine Dayem, and Brendan Trimboli, Ecova This report was prepared by the California Statewide Utility Codes and Standards Program and funded by the California utility customers under the auspices of the California Public Utilities Commission. Copyright 2014 Pacific Gas and Electric Company, Southern California Edison, Southern California Gas Company, San Diego Gas & Electric. All rights reserved, except that this document may be used, copied, and distributed without modification. Neither PG&E, SCE, SoCalGas, SDG&E, nor any of its employees makes any warranty, express of implied; or assumes any legal liability or responsibility for the accuracy, completeness or usefulness of any data, information, method, product, policy or process disclosed in this document; or represents that its use will not infringe any privately-owned rights including, but not limited to, patents, trademarks or copyrights. Table of Contents 1 EXECUTIVE SUMMARY ............................................................................................................ 1
    [Show full text]
  • Head-Mounted Displays and Dynamic Text Presentation to Aid Reading in Macular Disease
    HEAD-MOUNTED DISPLAYS AND DYNAMIC TEXT PRESENTATION TO AID READING IN MACULAR DISEASE Howard Farnoush Freemantle Moshtael-Oskui Submitted for the degree of Doctor of Engineering Heriot-Watt University School of Engineering & Physical Sciences January 2017 The copyright in this thesis is owned by the author. Any quotation from the thesis or use of any of the information contained in it must acknowledge this thesis as the source of the quotation or information. ABSTRACT The majority of individuals living with significant sight loss have residual vision which can be enhanced using low vision aids. Smart glasses and smartphone-based headsets, both increasing in prevalence, are proposed as a low vision aid platform. Three novel tests for measuring the visibility of displays to partially sighted users are described, along with a questionnaire for assessing subjective preference. Most individuals tested, save those with the weakest vision, were able to see and read from both a smart glasses screen and a smartphone screen mounted in a headset. The scheme for biomimetic scrolling, a text presentation strategy which translates natural eye movement into text movement, is described. It is found to enable the normally sighted to read at a rate five times that of continuous scrolling and is faster than rapid serial visual presentation for individuals with macular disease. With text presentation on the smart glasses optimised to the user, individuals with macular disease read on average 65% faster than when using their habitual optical aid. It is concluded that this aid demonstrates clear benefit over the commonly used devices and is thus recommended for further development towards widespread availability.
    [Show full text]
  • Microcontroller-Based Moving Message Display Powered by Photovoltaic Energy
    International Conference on Renewable Energies and Power Quality European Association for the Development of Renewable (ICREPQ’10) Energies, Environment and Power Quality (EA4EPQ) Granada (Spain), 23rd to 25th March, 2010 Microcontroller-Based Moving Message Display Powered by Photovoltaic Energy F. H. Fahmy, S. M. Sadek, N. M. Ahamed, M. B. Zahran, and Abd El-Shafy A. Nafeh Electronics Research Institute, PV Dept., NRC Building, El-Tahrir St., Dokki, 12311-Giza, Egypt Tel./Fax+202- 33310512/33351631 [email protected], [email protected], [email protected] Abstract. energy (e.g. solar energy, wind, hydrogen geothermal The aim of this paper is to design a textual display ….etc.) [1]. system, based on a light emitting diode (LED) dot matrix Solar energy is a renewable energy source that is array powered by solar energy. The paper involves taking environmentally friendly, unexhausted and unlike fossil the device from an initial concept, through a design phase, fuels, solar energy is available just about everywhere on to constructing a prototype of the product. The system earth and this source of energy is free. Stand-alone consists of the display unit, which is powered from a photovoltaic (PV) systems are designed to operate, photovoltaic ( PV) module and a solar sealed lead acid independent of electric utility grid. They are excellent for battery.. The self-contained nature of the intended design remote applications where utility grid is inaccessible and will allow the display to be mounted almost anywhere it is in locations where significant connection cost makes grid needed. Therefore, the main purpose of this paper is to power prohibitively expensive.
    [Show full text]
  • Osaka University Knowledge Archive : OUKA
    Adaptive Display of Virtual Content for Title Improving Usability and Safety in Mixed and Augmented Reality Author(s) Orlosky, Jason Citation Issue Date Text Version ETD URL https://doi.org/10.18910/55854 DOI 10.18910/55854 rights Note Osaka University Knowledge Archive : OUKA https://ir.library.osaka-u.ac.jp/ Osaka University Adaptive Display of Virtual Content for Improving Usability and Safety in Mixed and Augmented Reality January 2016 Jason Edward ORLOSKY Adaptive Display of Virtual Content for Improving Usability and Safety in Mixed and Augmented Reality Submitted to the Graduate School of Information Science and Technology Osaka University January 2016 Jason Edward ORLOSKY Thesis Committee Prof. Haruo Takemura (Osaka University) Prof. Takao Onoye (Osaka University) Assoc. Prof. Yuichi Itoh (Osaka University) Assoc. Prof. Kiyoshi Kiyokawa (Osaka University) I List of Publications Journals 1) Orlosky, J., Toyama, T., Kiyokawa, K., and Sonntag, D. ModulAR: Eye-controlled Vision Augmentations for Head Mounted Displays. In IEEE Transactions on Visualization and Computer Graphics (Proc. ISMAR), Vol. 21, No. 11. pp. 1259–1268, 2015. (Section 5.3) 2) Orlosky, J., Toyama, T., Sonntag, D., and Kiyokawa, K. The Role of Focus in Advanced Visual Interfaces. In KI-Künstliche Intelligenz. pp. 1–10, 2015. (Chapter 4) 3) Orlosky, J., Shigeno, T. Kiyokawa, K. and Takemura, H. Text Input Evaluation with a Torso-mounted QWERTY Keyboard in Wearable Computing. In Transaction of the Virtual Reality Society of Japan. Vol.19, No. 2, pp. 117–120, 2014. (Section 6.3) 4) Kishishita, N., Orlosky, J., Kiyokawa, K., Mashita, T., and Takemura, H. Investigation on the Peripheral Visual Field for Information Display with Wide-view See-through HMDs.
    [Show full text]