IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 58, NO. 1, JANUARY 2011 103 An Automated Hirschberg Test for Infants Dmitri Model*, Student Member, IEEE, and Moshe Eizenman Abstract—A novel automated method to measure eye misalign- displacement of the deviating eye by a proportionality constant ment in infants is presented. The method uses estimates of the known as the Hirschberg ratio (HR) [2]. The HR represents the Hirschberg ratio (HR) and angle Kappa (the angle between the amount of horizontal ocular rotation (in degrees) per millimeter visual and optical axis) for each infant to calculate the angle of eye misalignment. The HR and angle Kappa are estimated automati- of horizontal displacement of the corneal reflex from the center cally from measurements of the direction of the optical axis and the of the entrance pupil. ◦ coordinates of the center of the entrance pupil and corneal reflexes The mean value of the HR is approximately 12.5 /mm with an in each eye when infants look at a set of images that are presented intersubject variability of more than ±20% of the mean value sequentially on a computer monitor. The HR is determined by the [3]–[6]. Angle kappa may have both horizontal and vertical slope of the line that describes the direction of the optical axis ± ◦ as a function of the distance between the center of the entrance components of up to 5 [7]. Both angle Kappa and the HR are pupil and the corneal reflexes. The peak of the distribution of pos- subject-dependent and have to be estimated for each subject to sible angles Kappa during the image presentation determines the achieve accurate measurements of eye misalignment. This paper value of angle Kappa. Experiments with five infants showed that describes a novel automated HT (AHT) that uses estimates of the 95% limits of agreement between repeated measurements of the HR and angle Kappa to calculate eye misalignment. angle Kappa are ±0.61◦. The maximum error in the estimation of eye alignment in orthotropic infants was 0.9◦ with 95% limits of In contrast to earlier methods to automatically measure eye agreement between repeated measurements of 0.75◦. misalignment [3], [4], [8]–[10], the AHT allows free head move- ments and does not assume accurate fixation on any specific Index Terms—Angle kappa, Eye tracking, Hirschberg ratio (HR), Hirschberg test (HT), optical axis, strabismus, visual axis. target, and is therefore, more suitable for use with infants. The paper is organized as follows. The novel AHT is de- scribed in Section II. Experiments with infants are described in Section III. The discussion and conclusions are presented in I. INTRODUCTION Section IV. HE Hirschberg test (HT) to measure binocular ocular mis- T alignment was introduced more than 120 years ago [1]. It is still being used as the primary clinical method to measure II. AUTOMATED HIRSCHBERG TEST ocular misalignment prior to strabismus surgery in patients for The AHT is based on measurements from a remote two- which the alternate prism and cover test cannot be used reli- camera gaze-estimation system that does not require a user- ably (i.e., infants and young children). The test is performed calibration procedure [11]. The system calculates the position by estimating/measuring the displacement of the virtual image and orientation of each eye in space using the coordinates of of a light source, which is created by the front surface of the the pupil centers and corneal reflexes (see Fig. 1, inset). The cornea (corneal reflex), from the center of the entrance pupil, pupil centers and corneal reflexes are detected and tracked au- when subjects fixate on the light source. During the test, the dis- tomatically in pairs of images (see Fig. 1) that are captured placement in one eye (the fixating eye or the nondeviating eye in simultaneously by an eye-tracking system. patients with strabismus) is estimated first and the displacement In the following analysis, all points are represented as 3-D in the other eye is adjusted [2]. The adjustment compensates column vectors (bold font) in a right-handed Cartesian world for angle Kappa of the deviating eye under the assumption that coordinate system (WCS). The origin of the WCS is at the angles Kappa of the two eyes exhibit mirror symmetry [2]. center of a computer screen that is positioned in front of the Ocular misalignment is calculated by multiplying the adjusted subject (Fig. 2, inset). The Xw -axis is horizontal, the Yw -axis is vertical pointing up, and the Zw -axis perpendicular to the screen, pointing out of the screen. The analysis is based on the model that is described in Fig. 2, in which the light sources Manuscript received May 12, 2010; revised August 10, 2010; accepted September 11, 2010. Date of publication October 7, 2010; date of current are modeled as point sources, the video cameras are modeled as version December 17, 2010. This work was supported in part by a grant from pinhole cameras and the front surface of the cornea is modeled as the Natural Sciences and Engineering Research Council of Canada (NSERC), a spherical section. The line connecting the center of curvature and in part by the scholarships from NSERC and the Vision Science Research Program Award (Toronto Western Research Institute, University Health Net- of the cornea c, and the pupil center p, defines the optical axis of work, Toronto, ON, Canada). Asterisk indicates corresponding author. the eye. Only one of the M (M ≥ 2) light sources of the system *D. Model is with the Department of Electrical and Computer Engi- is shown in the Figure. neering, University of Toronto, Toronto, ON M5S 3G4, Canada (e-mail: [email protected]). M. Eizenman is with the Department of Electrical and Computer Engineer- ing, the Department of Ophthalmology and Vision Sciences, and the Institute of A. Estimation of the Optical Axis of the Eye Biomaterials and Biomedical Engineering, University of Toronto, Toronto, ON M5S 3G49, Canada (e-mail: [email protected]). Some of the equations in this section were presented earlier Digital Object Identifier 10.1109/TBME.2010.2085000 [11]–[14] and they are repeated here for completeness. 0018-9294/$26.00 © 2011 IEEE 104 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 58, NO. 1, JANUARY 2011 Fig. 1. Pair of images of a six-month-old baby obtained during the experiments using a remote binocular gaze-tracking system with two video cameras andthree infrared light sources. Pupil center and three corneal reflexes (marked by a cross and squares, respectively) are identified and tracked automatically by the system (see inset). Fig. 2. Ray-tracing diagram (not to scale in order to be able to show all the elements of interest), showing schematic representations of the eye, two cameras and one of the light sources of the system. Inset: remote gaze-estimation system and WCS. First, consider a ray that originates at a light source i, li ,travels Equations (1)–(3) suggest that for each combination of a cam- to a point qij on the corneal surface such that the reflected ray era j =1, 2 and a light source i =1,...,M, c can be expressed goes through the nodal point of camera j, oj , and intersects the as a function of two parameters camera image plane at a point uij . Then, qij can be expressed as cij = cij (kq,ij,R) . (4) oj − uij Since all the cij should be equal to each other, the unknown qij = oj + kq,ij − (1) oj uij parameters can be estimated by solving the following minimiza- where k represents the distance between the point of reflec- tion problem q,ij tion q , and the nodal point of the camera o . ij j ˆ ˆ According to the law of reflection, the incident ray, the re- kq,ij, R flected ray, and the normal at the point of reflection n , are i=1,...,M ;j=1,2 q ij coplanar, and the normal at the point of reflection nq ,ij is a 2 = arg min cij (kq,ij,R) − ckl (kq,kl,R) (5) bisector of an angle li − qij − oj . Thus (i,j)=( k,l) li − qij oj − qij nq,ij = + . (2) where the summation is over all possible distinct combinations li − qij oj − qij of cij and ckl. Since any normal to the spherical surface goes through the Finally, c is obtained as an average of all the cij center of curvature of the cornea c, then 1 ˆ nq,ij c = cij (kq,ij, Rˆ) (6) c = qij − R (3) N nq,ij i,j where R is the radius of the cornea. where N =2M in the case of two cameras and M light sources. MODEL* AND EIZENMAN: AN AUTOMATED HIRSCHBERG TEST FOR INFANTS 105 Then, consider an imaginary ray that originates at the pupil In strict terms, the spatial location of crij depends on the center p, travels through the aqueous humor and cornea (effec- position of the nodal point of the camera oj relative to the tive index of refraction ≈ 1.3375), and refracts at a point rj on eye. Therefore, in general, the spatial location of crij will be the corneal surface as it travels into the air (index of refraction slightly different for each of the two cameras. Despite this, ≈ 1) such that the refracted ray passes through the nodal point an approximate virtual image of the light source cri can be of camera j, oj , and intersects the camera image plane at a point found as the midpoint of the shortest segment defined by a point up,j. This refraction results in the formation of an image of the belonging to each of the lines given by (12), j =1, 2, i.e., center of the entrance pupil pv,j located on the extension of the −1 1 a1 · a1 −a1 · a2 refracted ray, i.e., cri = [ a1 a2 ] 2 −a1 · a2 a2 · a2 p o o − u v,j = j + kp,j ( j p,j), for some kp,j .