Improving Eye-Gaze Tracking Accuracy Through Personalized Calibration of a User's Aspherical Corneal Model by Isabella Taba B.Sc., Simon Fraser University, 2008 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE in The Faculty of Graduate Studies (Electrical and Computer Engineering) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) January 2012 c Isabella Taba 2012 Abstract The eyes present us with a window through which we view the world and gather information. Eye-gaze tracking systems are the means by which a user's point of gaze (POG) can be measured and recorded. Despite the active research in gaze tracking systems and major advances in this field, calibration remains one of the primary challenges in the development of eye tracking systems. In order to facilitate gaze measurement and tracking, eye-gaze trackers utilize simplifications in modeling the human eye. These simplifications include using a spherical corneal model and using population averages for eye parameters in place of individual measurements, but use of these simplifications in modeling contribute to system errors and impose inaccuracies on the process of point of gaze estimation. This research introduces a new one-time per-user calibration method for gaze estimation systems. The purpose of the calibration method developed in this thesis is to calculate and estimate different individual eye parameters based on an aspherical corneal model. Replacing average measurements with individual measurements promises to improve the accuracy and reliability of the system. The approach presented in this thesis involves estimating eye parameters by statistical modeling through least squares curve fitting. Compared to a current approach referred to here as the Hennessey's calibra- tion method, this approach offers significant advantages, including improved, individual calibration. Through analysis and comparison of this new cali- bration method with the Hennessey calibration method, the research data presented in this thesis shows an improvement in gaze estimation accuracy of approximately 27%. Research has shown that the average accuracy for the Hennessey calibration method is about 1:5 cm on an LCD screen at a distance of 60 cm, while the new system, as tested on eight different sub- jects, achieved an average accuracy of 1:1 cm. A statistical analysis (T-test) of the comparative accuracy of the new calibration method versus the Hen- nessey calibration method has demonstrated that the new system represents a statistically significant improvement. ii Table of Contents Abstract ................................. ii Table of Contents ............................ iii List of Tables .............................. vi List of Figures . viii Acknowledgements . xii Dedication . xiii 1 Introduction ............................. 1 1.1 Overview . 1 1.2 Research Goal . 2 1.3 Contributions of This Thesis . 3 2 The Eye and Eye Tracking Background ............ 5 2.1 Eye Detection Methods . 5 2.2 Review of Gaze Estimation Methods . 7 2.2.1 Electrooculography(EOG) . 8 2.2.2 Scleral Contact Lens . 9 2.2.3 Recent Eye-Tracking Systems . 9 2.3 Basic Optics of the Eye . 19 2.3.1 The Cornea . 21 2.3.2 Axes and Angle of the Eye . 25 2.3.3 Light and the Eye . 28 2.3.4 Ocular Biometry . 28 3 Hennessey's POG Tracking Method Used Here . 30 3.1 Background . 30 3.1.1 Eye Model . 31 3.1.2 Corneal Model . 33 iii Table of Contents 3.2 Coordinate Systems . 37 3.2.1 Camera Coordinate System(CCS) . 38 3.2.2 Light Source Model . 39 3.3 Eye, Camera, Light Sources, in One System . 40 3.3.1 One Camera and One Light Source . 40 3.3.2 One Camera and Two Light Sources . 41 3.3.3 Two Cameras and One or More Light Sources . 41 3.4 The Method . 43 3.4.1 Feature Extraction . 43 3.4.2 Pattern Matching . 44 3.4.3 Center of Corneal Curvature Estimation . 45 3.4.4 Pupil Center Estimation . 48 3.4.5 Optical Axis Estimation . 50 3.4.6 Calibration Phase . 50 3.4.7 Summary of Point of Gaze (POG) Estimation . 53 4 Proposed New Calibration Method . 54 4.1 Introduction . 54 4.2 Calibration Method . 55 4.2.1 System Calibration . 57 4.2.2 User Calibration . 59 4.3 Calibration Method Based on Personalized Eye Parameters and Aspherical Corneal Surface . 61 4.3.1 Calibration Phase I . 63 4.3.2 Calibration Phase II . 65 5 Experimental Method and Results . 71 5.1 Introduction . 71 5.2 Experimental Methods . 72 5.2.1 System Design . 72 5.2.2 Accuracy Metrics . 73 5.2.3 Methods . 73 5.3 Results . 76 5.3.1 Radius of the Corneal Curvature for Different Users . 81 5.4 Statistical Analysis . 84 6 Discussion and Conclusions ................... 90 6.1 Summary of Contribution . 90 6.2 Discussion . 90 6.3 Conclusions and Future Work . 92 iv Table of Contents 6.3.1 Future Work . 93 References ................................ 95 Appendices A Appendix ...............................101 A.1 Purkinje Images . 101 A.2 Optics of a Keratometer . 101 A.3 Keratoscopy . 104 B Appendix ...............................105 B.1 Stereo Camera . 105 v List of Tables 2.1 Path of light ray using Gullstrand's eye model based on a spherical eye and corneal model [1] [2] . 13 3.1 Different conic curves that are defined by Baker's equation according to their p-values . 35 5.1 During the calibration phase, a grid of 3x3 points was shown to each subject. The above table shows these calibration points in the world coordinate system in centimeters. 76 5.2 During the trial phase, a different grid of 3x3 points was shown to each subject. The above table shows these points in centimeters. 77 5.3 The above table summarizes an average error on POG estima- tions. These data were collected by Hennessey [3]. In total, 7 test subjects were tested on the system. An average error for each subject by using new proposed calibration method and the old calibration method (i.e. Hennessey's method [4]) was estimated. 79 5.4 A new set of data was collected during this experiment. In total, 8 test subjects were tested on the system. An aver- age gaze error for each subject by using the new proposed calibration method and old calibration method was estimated. 79 5.5 Radius of the corneal variation between different test sub- jects. The radius of corneal curvature is estimated at each calibration point for each test subject, (the new data set). 82 5.6 Estimated values for rp (i.e. distance between the center of the corneal curvature and pupil center) for each test subject. 84 5.7 Calculated mean, standard deviation, and variance for two data groups collected by Hennessey [3]. The data was cal- ibrated using the old calibration method (group 1) and the new calibration method (group 2). The two groups were com- pared using the T-test statistical analysis . 86 vi List of Tables 5.8 T-test statistical analysis for data collected by Hennessey [3]. 86 5.9 Calculated mean, standard deviation, and variance for the newly recorded data groups used in the T-test statistical anal- ysis . 87 5.10 T-test statistical analysis for the new collected data set . 89 A.1 Location of the Purkinje images for Gullstrand schematic of the eye . 101 vii List of Figures 1.1 Overview of a remote eye-gaze tracking system based on 3D modeling. The optical axis of the eye is a vector from the center of corneal curvature to the center of the pupil. The visual axis of the eye is a vector from the fovea to the point of the regard on the screen. 3 1.2 The new calibration method provides personalized eye pa- rameters (α and β : angular offsets between the visual and optical axes, Rc: radii of the corneal curvature, and Rp: the distance between the center of corneal curvature and pupil center) as inputs for Hennessey's gaze tracking system. 4 2.1 The appearance of the eye as is projected into the camera image may vary as the user changes the view angle or moves the head. 5 2.2 Image of EyeSeeCam, a head mounted VOG[5] . 11 2.3 The Cornea is not spherical. The radius of the corneal cur- vature is the shortest at the apex of the cornea. 13 2.4 The left image is dark pupil image and the right image is bright pupil image. As shown, in dark pupil image the light reflections from the surface of the cornea (glints) are visible. 14 2.5 The P-CR vector is a vector from corneal reflection to the cen- ter of pupil. Through 2-D mapping in the calibration phase, the P-CR vector is related to POG on the screen [3] . 15 2.6 Human eye cross sectional view . 20 2.7 The cornea is modeled as an ellipsoid whose outer limit corre- sponds to the limbus. The approximate eccentricity, and the radius of curvature at the apex varies between individuals. q 2 2 Rc is the radius of corneal curvature, where Rc = Rx + Ry. 23 2.8 The sagittal depth is the distance between the flat (bottom) plane at given diameter to the apex of the cornea. 24 2.9 Shape factor for aspheric surface modified from [6]. 25 viii List of Figures 2.10 Family of conic sections with different shape factor (P), values modified from [6] . 26 3.1 Nodal points of the eye, N1 = first nodal point, N2= second nodal point. Nodal points are points on the optical axis of the eye. If a ray is directed toward one of them it gets refracted by the eye's lens in a direction toward the other one as if the ray comes from the other nodal point.