Eye Tracking in Virtual Reality
Total Page：16
File Type：pdf, Size：1020Kb
Charles University in Prague Faculty of Mathematics and Physics BACHELOR THESIS Jozef Jir´asek Sledov´an´ıoˇcn´ıch pohyb˚u ve virtu´aln´ırealitˇe Eye Tracking in Virtual Reality Department of Software and Computer Science Education Supervisor of the bachelor thesis: Mgr. Cyril Brom Ph.D. Study programme: Informatics Specialization: Programming Prague 2011 I would like to thank my advisor Mgr. Cyril Brom, Ph.D. for providing guidance in my work, and for allowing me to utilize an office at the Department of Software and Computer Science Education to develop and test the software. I would also like to thank Ing. Vratislav Fabi´an, Ing. Marcela Fejtov´a, and Ing. Jiˇr´ıMouˇcka from Medicton Group Ltd. for their assistance in solving technical problems with the I4Control device. Further thanks go to Mgr. Iveta Fajnerov´aand Mgr. Kamil Vlˇcek, Ph.D. from the Institute of Physiology of the Academy of Sciences of the Czech Republic for providing me with details about spatial navigation experiments and for help with testing my software during the experiments. Last, I would like to thank Mgr. Cyril Brom, Ph.D. and RNDr. Galina Jir´askov´a, CSc. for proofreading my work. I declare that I carried out this bachelor thesis independently, and only with the cited sources, literature, and other professional sources. I understand that my work relates to the rights and obligations under the Act No. 121/2000 Coll., the Copyright Act, as amended, in particular the fact that the Charles University in Prague has the right to conclude a license agreement on the use of this work as a school work pursuant to Section 60 paragraph 1 of the Copyright Act. In Prague date ............ N´azov pr´ace: Sledov´an´ıoˇcn´ıch pohyb˚uve virtu´aln´ı realitˇe Autor: Jozef Jir´asek Katedra: Kabinet software a v´yuky informatiky Ved´uci bakal´arskej pr´ace: Mgr. Cyril Brom Ph.D. Abstrakt: V tejto pr´aci prezentujeme aplik´aciu pre sledovanie a ukladanie d´at o pohybe l’udsk´eho oka pri sledovan´ıobrazovky poˇc´ıtaˇca. Pouˇz´ıvame dve komerˇcne dos- tupn´ezariadenia: I4Control od firmy Medicton Group s.r.o. a TrackIR 4 od firmy NaturalPoint. N´aˇssoftware komunikuje so syst´emom SpaNav pouˇz´ıvan´ym na v´yskum priestorovej orient´acie. Tieˇzposkytujeme moˇznost’n´aˇssoftware jednodu- cho rozˇs´ırit’a pouˇzit’pri v´yvoji in´ych aplik´aci´ıktor´esleduj´uoˇcn´epohyby. Kl’´uˇcov´eslov´a: Sledovanie pohl’adu, priestorov´anavig´acia, interakcia ˇcloveka s poˇc´ıtaˇcom. Title: Eye Tracking in Virtual Reality Author: Jozef Jir´asek Department: Department of Software and Computer Science Education Supervisor: Mgr. Cyril Brom Ph.D. Abstract: In this work we present an application for observing and recording data about movements of a human eye when looking at a computer screen. We use two com- mercially available devices: I4Control by Medicton Group Ltd. and TrackIR 4 by NaturalPoint. We build a software package which interfaces with the SpaNav sys- tem for cognitive research. We also provide an extensible framework for creating other eye tracking applications. Keywords: Eye tracking, head tracking, spatial navigation, human-computer interaction. Contents Introduction 2 1 Eye Tracking Overview 3 2 Our Work 5 3 Technologies Used 7 3.1 SpaNav ................................ 7 3.2 EyelinkII ............................... 8 3.3 I4Control ............................... 9 3.4 TrackIR4 ............................... 10 4 Design Considerations 12 4.1 Designgoals.............................. 12 4.2 Softwarearchitectureoverview . 13 5 Implementation 15 5.1 Communication ............................ 15 5.2 Tracking................................ 15 5.2.1 Headtrackingandthecenterpoint . 16 5.2.2 Eyetrackingandthedeltafunction . 17 5.3 Calibration .............................. 19 5.3.1 Calibratingtheheadtracker . 19 5.3.2 Calibratingtheeyetracker . 20 5.4 Datarecording ............................ 20 5.5 EyeTrackingclients . 22 5.5.1 CalibrationWorker . 23 5.5.2 TestWorker .......................... 24 5.5.3 UTWorker........................... 24 5.6 UserInterface ............................. 25 6 Testing 26 7 Conclusions 29 Bibliography 31 1 Introduction Eye tracking is the technology of observing the movements of a human eye. This is usually achieved by recording the eye using video or infrared cameras and analyzing the recorded data. Eye tracking has applications in cognitive stud- ies, assistive technologies, design, advertisement, entertainment, and other areas. Basic information about eye tracking can be found in [Duchowski(2007)]. In this work, we build upon the thesis of Ivana Supalov´a[ˇ Supalov´a(2009)].ˇ In her work, she built a virtual environment SpaNav for running spatial navigation experiments. SpaNav presents a person with a simple virtual reality scenario and observes and records their eye movements as they try to solve this scenario. Virtual reality is often used in such experiments, as a virtual scenario it is usually less expensive to create, maintain, and modify than a real physical environment. Virtual reality has its limits, but using current technology it is usable for the purpose of such an experiment. The SpaNav software works with the device Eyelink II developed by SR Re- search Ltd. [SR Research Ltd.(2010)] This is a professional apparatus which comes with hardware and software performing all stages of the eye tracking pro- cess. Unfortunately, the high cost of this device is prohibitive to small research teams. Therefore, we have been tasked with developing a solution which would perform eye tracking measurements using less expensive commercially available hardware and software. We use an eye tracker I4Control by Medicton [Medicton group(2008)], which can gather eye movement data. Since the subject’s head can move, we also need to collect data about the head’s movement. We use a head tracker TrackIR 4 by NaturalPoint [NaturalPoint(2008)]. Our application combines data from these two devices and uses them to calculate a position on the computer screen where the observed subject is looking at. This information is then sent to the SpaNav system, which is used to perform the actual experiments. In addition to interfacing with SpaNav, we provide an extensible framework for applications which use eye tracking data. Programmers can easily extend our software to develop their own applications and use our eye tracking measurements. 2 1. Eye Tracking Overview Eye tracking is the process of measuring movements of the human eye. There are two main subtasks involved. First, we need to determine the gaze direction of the subject. This is the vector in three-dimensional space in the direction the subject is looking, relative to the position and orientation of the subject’s head. The second step is computing the intersection of this vector with the observed scene, this is called the gaze point. Methods of determining the gaze direction can be divided into two groups: intrusive and non-intrusive. An example of an intrusive method is attaching special contact lenses to the subject’s eye and tracking movement of these lenses [Huey(1908)]. Another intrusive method called electrooculography uses electrodes placed around the eye to measure eye muscle movement [Bulling et al.(2011)]. Non-intrusive methods use a video camera (typically in the infrared spectrum) to capture movements of the eye. Images from this camera are then analyzed to determine the gaze direction. In general, non-intrusive methods are cheaper and easier to set up and use, but their results are less accurate. Both styles of tracking can be further subdivided into online and offline meth- ods. Offline methods only gather data during the experiment and analyze it later. Online methods analyze data and compute the gaze point while the measurement is being performed. Due to computational complexity, online methods are usually limited to lower frame rates and accuracy than offline methods. However, online methods give valuable feedback to both the experimenter and the subject in real time, such as whether the eye tracking device is working properly. Eye tracking has many applications in different areas of science and indus- try. Psychologists study the way we look at images or scenes when asked to perform various tasks [Yarbus(1967)]. Advertisers and web page designers are interested in knowing which parts of their product attract the most attention [Goldberg et al.(2002)]. Movements of the eye can also be used in assistive tech- nologies to allow people who can not use their hands to control a computer or another device [ICT Results(2009)]. Recently, as eye tracking systems are becom- ing more affordable, eye tracking has also found its way into the entertainment 3 and video gaming industry [Sundstedt(2010)]. In the remainder of this work, we will focus on eye tracking for physiological experiments, in particular experiments concerned with spatial navigation. This has several important implications. First, the measurement has to be sufficiently accurate. In an experiment with SpaNav, we want to know which object the subject is looking at during their decision process. The objects’ size usually varies between 100 and 300 pixels on the computer screen, which translates to about 4 to 10 degrees required accuracy when computing the gaze vector. Another important fact is that we do not receive any immediate feedback about accuracy of the measurement. For instance, in assistive technologies, when the subject controls movement of the mouse cursor using eye movement, we can immediately see when the measurements are inaccurate - the cursor does not point to the area the subject is looking at. In our applications, we have no such feedback during an experiment. 4 2. Our Work In this work we present a software package EyeTracking which performs eye track- ing using commercially available hardware. This software was developed in col- laboration with Institute of Physiology of the Academy of Sciences of the Czech Republic. One of the research areas of the Institute is spatial navigation. This research can help for instance in prediction of Alzheimer’s disease in test subjects. The experiments are performed in a virtual environment created on a personal computer.