UC Berkeley UC Berkeley Previously Published Works
Title Recovering stereo vision by squashing virtual bugs in a virtual reality environment.
Permalink https://escholarship.org/uc/item/6hb7321d
Journal Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 371(1697)
ISSN 0962-8436
Authors Vedamurthy, Indu Knill, David C Huang, Samuel J et al.
Publication Date 2016-06-01
DOI 10.1098/rstb.2015.0264
Peer reviewed
eScholarship.org Powered by the California Digital Library University of California Submitted to Phil. Trans. R. Soc. B - Issue
Recovering stereo vision by squashing virtual bugs in a virtual reality environment.
For ReviewJournal: Philosophical Only Transactions B Manuscript ID RSTB-2015-0264.R2
Article Type: Research
Date Submitted by the Author: n/a
Complete List of Authors: Vedamurthy, Indu; University of Rochester, Brain & Cognitive Sciences Knill, David; University of Rochester, Brain & Cognitive Sciences Huang, Sam; University of Rochester, Brain & Cognitive Sciences Yung, Amanda; University of Rochester, Brain & Cognitive Sciences Ding, Jian; University of California, Berkeley, Optometry Kwon, Oh-Sang; Ulsan National Institute of Science and Technology, School of Design & Human Engineering Bavelier, Daphne; University of Geneva, Faculty of Psychology and Education Sciences; University of Rochester, Brain & Cognitive Sciences Levi, Dennis; University of California, Berkeley, Optometry;
Issue Code: Click here to find the code for your issue.:
Subject: Neuroscience < BIOLOGY
Stereopsis, Strabismus, Amblyopia, Virtual Keywords: Reality, Perceptual learning, stereoblindness
http://mc.manuscriptcentral.com/issue-ptrsb Page 1 of 29 Submitted to Phil. Trans. R. Soc. B - Issue
Phil. Trans. R. Soc. B. article template 1 2 3 4 Phil. Trans. R. Soc. B. doi:10.1098/not yet assigned 5 6 7 8 Recovering stereo vision by squashing virtual bugs in 9 10 a virtual reality environment 11 12 13 14 Indu Vedamurthy 1†, David C. Knill 1, Samuel J. Huang 1, Amanda Yung 1, 15 4 1,3 1, 2 4 16 Jian Ding , Oh Sang Kwon , Daphne Bavelier & Dennis M. Levi * 17 18 1Department of Brain & Cognitive Sciences, University of Rochester, Rochester, NY 14627-0268, USA 2 For Review Only 19 Faculty of Psychology and Education Sciences, University of Geneva, Switzerland 3School of Design & Human Engineering, UNIST, South Korea 20 4 21 School of Optometry and Helen Wills Neuroscience Institute, University of California, Berkeley, CA, 94720, USA 22 †Work was performed while at the University of Rochester 23 *Corresponding author 24 25 26 Keywords: Stereopsis; Virtual Reality; Perceptual learning; Strabismus; Amblyopia 27 28 29 Summary 30
31 Stereopsis is the rich impression of three dimensionality, based on binocular disparity—the 32 33 differences between the two retinal images of the same world. However, a substantial proportion of 34 the population is stereo deficient, and relies mostly on monocular cues to judge the relative depth or 35 distance of objects in the environment. Here we trained adults who were stereo blind or stereo 36 deficient due to strabismus and/or amblyopia in a natural visuo motor task – a “bug squashing” game 37 in a virtual reality (VR) environment. The subjects’ task was to squash a virtual dichoptic bug on a 38 slanted surface, by hitting it with a physical cylinder they held in their hand. The perceived surface 39 slant was determined by monocular texture and stereoscopic cues, with these cues being either 40 consistent or in conflict, allowing us to track the relative weighting of monocular versus stereoscopic 41 cues as training in the task progressed. Following training most participants showed greater reliance 42 43 on stereoscopic cues, reduced suppression and improved stereo acuity. Importantly, the training 44 induced changes in relative stereo weights were significant predictors of the improvements in 45 stereoacuity. We conclude that some adults deprived of normal binocular vision and insensitive to the 46 disparity information can, with appropriate experience, recover access to more reliable stereoscopic 47 information. 48 49 50 51 1. Introduction 52 53 54 Stereopsis is the impression of three dimensionality—of objects “popping out in depth”—that 55 most humans get when they view real world objects with both eyes, based on binocular disparity, the 56 differences between the two retinal images of the same world. However, a substantial proportion of 57 the population is stereoblind or stereo deficient. The exact proportion depends on the specific test for 58 stereopsis and the age of the subjects, but estimates of impaired stereopsis range from ≈ 5 percent [1] 59
60 *Author for correspondence ([email protected]). 1 http://mc.manuscriptcentral.com/issue-ptrsb Submitted to Phil. Trans. R. Soc. B - Issue Page 2 of 29
1 2 3 4 to as high as 34% in older subjects [2]. This impairment may have a substantial impact on 5 visuomotor tasks, difficulties in playing sports in children and locomoting safely in older adults, and 6 may also limit career options (see ref. 3 for a recent review). 7 Over the past five years there has been a renewed interest in restoring stereopsis in adults with 8 strabismus, since the publication of “Fixing My Gaze”[4] in which Susan Barry, a neuroscientist, 9 recounts her recovery from strabismus (a turned eye) and her amazement as she regains stereo vision, 10 and the description by Bruce Bridgeman, a vision scientist who had been stereo deficient all his life, 11 of experiencing stereoscopic depth perception after viewing the 3D movie Hugo [5]. However, there 12 are a limited number of experimental studies documenting recovery of stereopsis in adults who have 13 14 long been deprived of normal binocular vision. Nakatsuka et al. [6] reported that adult monkeys 15 reared with prisms had mild stereo deficiencies that improved through perceptual learning (PL) after 16 10,000 20,000 trials. Astle, Webb & McGraw [7] reported on two cases of humans with 17 anisometropic amblyopia whose stereopsis improved following a learning based course of training, 18 which included refractiveFor adaptation Review followed by monocular Only PL as well as stereoscopic PL. Ding & 19 Levi [8] provided the first evidence for the recovery of stereopsis through PL in human adults long 20 deprived of normal binocular vision due to strabismus and/or amblyopia. They used a training 21 paradigm that combined monocular cues that were perfectly correlated with the stereoscopic cues. 22 23 Following PL (thousands of trials) with stereoscopic gratings, adults who were initially stereoblind or 24 stereo deficient showed substantial recovery of stereopsis. Importantly, these subjects reported that 25 depth “popped out” in real life, and they were able to enjoy 3 D movies for the first time, similar to 26 the experiences of Susan Barry and Bruce Bridgeman. Their recovered stereopsis is based on 27 perceiving depth by detecting binocular disparity, but has reduced resolution and precision. Similar 28 improvements were recently reported in a group of anisometropic and ametropic amblyopes who 29 were trained with anaglyphic textures with different disparities [9]. 30 How does training improve stereopsis? There are multiple cues to depth – both binocular (retinal 31 disparity; convergence) and monocular (motion parallax, relative size, familiar size, cast shadows, 32 33 occlusion, accommodation, texture gradient, linear perspective, aerial perspective, shading, lighting, 34 and defocus blur). Stereo blind or deficient observers rely mainly on monocular cues. Ding & Levi 35 [8] speculated that stereoblind or stereo deficient observers could learn to associate monocular and 36 binocular cues to depth if they were highly correlated through repeated practice. In the current study 37 we trained adult observers who were stereo deficient due to strabismus and/or amblyopia (lazy eye) 38 in a natural visuo motor task – a “bug squashing” game in a virtual reality (VR) environment. The 39 subjects’ task was to squash a virtual dichoptic bug on a slanted surface, by hitting it with a cylinder. 40 The slant of the surface was determined by (i) purely stereoscopic cues (pure stereo cue trials), or (ii) 41 consistent monocular texture and stereoscopic cues (cue consistent trials), or (iii) conflicting 42 43 monocular texture and stereoscopic cues (cue conflict trials). Importantly, our bug squashing training 44 involves integrating not just multiple visual cues, but also the rich information from tactile and 45 kinesthetic feedback. We hypothesized that training with multiple cues to depth with rich feedback 46 might enable stereoblind or stereo deficient observers to increase their reliance on stereoscopic cues. 47 Following training these observers showed increased reliance on stereoscopic cues (relative to 48 monocular cues), reduced interocular suppression and significantly improved stereoacuity. 49 Our results have important implications for the recovery of visual function late in life, well 50 outside the childhood period, until recently thought to offer the only real scope for plasticity. More 51 broadly, they demonstrate the use of virtual reality as a promising approach for perceptual training of 52 53 all kinds. 54 55 56 2. Methods 57 58 (a) Participants 59 60 http://mc.manuscriptcentral.com/issue-ptrsb 2 Page 3 of 29 Submitted to Phil. Trans. R. Soc. B - Issue
1 2 3 4 Eleven adults (mean age 34.7 years, range 19 56 years) with long standing abnormal 5 binocular vision completed the training study. Informed consent was obtained from all subjects 6 conforming to the guidelines of the Research Subjects Review Board at the University of Rochester. 7 Subjects were recruited mainly through referrals from local eye doctors and through print 8 advertisements, and were paid $10/hour for study participation. The inclusion criteria for the 9 experimental group were (a) impaired stereopsis associated with one or more of the following 10 conditions − anisometropic amblyopia, strabismic amblyopia, mixed (both anisometropic and 11 strabismic) or pure strabismus, i.e., without amblyopia (b) normal ocular and general health, and (c) 12 13 no history of eye surgeries except for those to correct strabismic deviation. Subjects with non 14 comitant and/or large angle strabismus (>30 prism diopters) were excluded. Anisometropia was 15 defined as ≥ 1D difference in spherical equivalent refraction between the two eyes. Those with 16 manifest ocular deviation (strabismus), as indicated by the cover test, were classified as pure 17 strabismics, and those having both anisometropia and strabismus were classified as mixed etiology. 18 The clinical details ofFor subjects areReview summarized in Table 1.Only Nine adults with normal acuity and 19 binocular vision were recruited to provide normal control stereo weight data at baseline before any 20 training. Three of these participants were then entered in the training phase with two completing 30 21 training sessions and one completing 20 training sessions. Mean results for these 3 normal control 22 23 observers are shown in figures 3, 5, 6, 7 and 8. 24 Subjects completed the training study in five phases screening, pretesting, training, post 25 testing, and follow up. In the screening phase, subjects received a complete eye exam to determine 26 whether or not they met the study inclusion criteria. Qualified subjects returned to the lab to complete 27 a pre training test battery (pretesting) that included baseline assessment of stereopsis, suppression, 28 vergence and visual acuity. They then underwent a virtual reality based stereo training program 29 (described below) for 35 sessions distributed over 8 11 weeks. The pre training test battery was re 30 administered a few days after training to assess any training related changes (post testing), and for 31 the third time after a 2 month period of no intervention to assess retention effects (follow up). 32 33 34 (b) Training 35 36 The visual stimuli consisted of textured, slanted virtual disks with a central fixation target 37 38 presented in a virtual reality display (described in detail in ref. 10). Subjects wore Crystal Eyes 39 shutter goggles during the training to view the stimuli (StereoGraphics Corporation, San Rafael, CA). 40 All stimuli were drawn in red to minimize inter ocular crosstalk by utilizing the relatively fast red 41 phosphor of the monitor. A small dichoptic bug was rendered in the plane of the disk and served as 42 the fixation target (Fig. 1 A & B). In the absence of suppression, and with accurate vergence, subjects 43 perceive a complete bug with six legs and a pair of antennae. If the non dominant eye is suppressed, 44 the observer sees only the bug’s thorax, abdomen and 4 diagonal legs. If the dominant eye is 45 suppressed, the observer sees only the head and remaining legs. We used two types of textures to 46 render the disks, so as to create stimuli with and without effective monocular cues to slant in depth 47 48 regular tiled textures, for which subjects with normal binocularity assign nearly equal weights to 49 monocular and stereo cues (Fig. 1C), and randomly shaped dot textures (Fig. 1D) for which the 50 texture cues are relatively uninformative [10, 11] 51 52 53 54 55 56 57 58 59 60 http://mc.manuscriptcentral.com/issue-ptrsb 3 Submitted to Phil. Trans. R. Soc. B - Issue Page 4 of 29
1 2 3 4 5 A. B. Figure 1 Examples of 6 fixation ‘bug’ stimulus and texture surfaces. A) 7 The fixation bug’s thorax, 8 abdomen and 4 diagonal 9 legs were presented to the 10 dominant eye. B) The 11 head and remaining bug 12 parts were presented to the 13 non dominant eye. When 14 fused, a complete bug 15 with six legs and a pair of antennae would be 16 17 perceived. Note that Fig. 18 For Review Only 1A and 1B included here 19 C. D. are for illustrative purposes only and were 20 not designed as a stereo 21 pair. 22 C) Tiled texture 23 background, (D) Random 24 dot background. 25 For tiled backgrounds 26 rendered stereoscopically, 27 the monocular cues to 28 surface slant stem from 29 distortions of the regular 30 grid pattern mapped onto the 31 surface, and the disparity signal to surface slant stems 32 from the differences 33 between right and left eyes’ 34 retinal images. Examples of 35 trial feedback are also 36 depicted here. The dichoptic 37 bug crawls away (C) for an 38 incorrect response, and (D) 39 it explodes in little bits for a 40 correct response. 41 42 The subjects’ task was to squash the virtual bug by hitting it with a plexiglass cylinder 43 measuring 6.4 cm in diameter and 12.7 cm in height and weighing 227 g. Subjects grabbed the 44 cylinder and moved it from a starting plate positioned to the right of the virtual image (see Fig. 2), 45 and were required to place it as flush as possible with the target surface over the bug. A three camera 46 47 Optotrak 3020 motion capture system (Northern Digital Inc., Waterloo, Canada) was used to track 48 the four infrared markers on the cylinder so as to estimate its 3D position and orientation in real time. 49 A virtual cylinder was rendered co aligned with the real cylinder as it moved within the workspace. 50 The cylinder was rendered with perspective and stereoscopic cues. As described below, since 51 subjects viewed the stimuli through circular apertures to eliminate contextual cues provided by the 52 monitor, the virtual cylinder was visible to the subject only when it approached the bug (typically for 53 about ~250 300 ms before impact). When the cylinder was not within the subject’s field of view, the 54 only information available about the orientation of the cylinder was proprioceptive information. 55 Every trial was programmed to end when the cylinder contacted the target plate, which provided 56 57 subjects with haptic feedback about the target slant. 58 59 60 http://mc.manuscriptcentral.com/issue-ptrsb 4 Page 5 of 29 Submitted to Phil. Trans. R. Soc. B - Issue
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 For Review Only 19 20 21 22 Figure 2: Stimuli were displayed on an inverted 22 inch Mitsubishi Diamond Pro 2070SB CRT monitor, with screen 23 resolution 1152 x 864 pixels and refresh rate 120 Hz. Observers viewed the reflections of the stereoscopically presented 24 stimuli through a mirror using Crystal Eyes shutter glasses (StereoGraphics Corporation, San Rafael, CA). An opaque 25 black plate was placed beneath the mirror so that subjects could see only the virtual image of the display formed below 26 the mirror. A PUMA 260 robot arm, invisible to the subject, coaligned a circular metal (target) surface with the virtual 27 image at a distance of 63.5cm from the viewer’s eyes. The slant of the target surface was defined as its orientation around 28 the x axis relative to subject’s line of sight (see dashed line in the figure inset), and a zero degree slant indicated 29 frontoparallel. Monocular and binocular slants differed on cue conflict trials, but subjects perceived the stimuli as a single 30 slanted disk. Head movements were restricted with head and chin rests. Subjects’ task was to move a cylinder from the starting platform and place it flush onto the real surface to squash the virtual image. The slant of the real surface was 31 determined by the mean of the monocular and binocular slants. The starting platform was 40 cm to the right of the target 32 surface, 20 cm closer to the subject than the target surface, and 16.5 cm above the target surface (all measured from the 33 subject's point of view). 34 35 At the beginning of each session, subjects performed a calibration procedure to estimate the 36 37 positions of their eyes relative to the monitor to ensure accurate rendering of the virtual 3D space (see 38 ref. 10 for details). The method of adjustment was then used to equalize the perceived contrast of the 39 bug’s half images presented to the dominant and non dominant eyes. Measurements were repeated 40 six times, and the mean contrast was used to display the dominant eye’s nonius bug during the 41 training task. Prisms were used to optically align the nonius bug parts for experimental subjects with 42 strabismus. No subject reported diplopic background textures post alignment. Contextual cues from 43 monitor edges and other surrounding objects were removed by placing an adjustable circular aperture 44 before each eye that restricted the viewing angle to 11.9°. 45 46 47 Participants completed 35 sessions over a period of 8 11 weeks with an average of 3 sessions 48 per week in the training phase. The first three usable sessions (pre) and last two sessions (post) were 49 used to estimate changes from pre to post training stereo weights. Two more sessions were run two 50 months after training to estimate changes at follow up. In each session, target stimuli were rendered 51 at five possible slants starting at 20 degrees and ending at 50 degrees away from fronto parallel in 52 7.5 degree steps, with one of the 5 slants chosen randomly between trials. Sessions were run in 53 blocks of 60 trials, with 6 blocks per session totaling 360 trials. 240 of these trials incorporated 54 stimulus disks with tiled texture. In half of those trials, the monocular (tiled texture) and stereoscopic 55 (disparity) cues to the surface slant were consistent (cue consistent trials). In the other half, the slant 56 57 specified by stereoscopic and texture cues differed by 7.5° (for example, monocular/stereoscopic 58 slants: 42.5 / 35 or 27.5 /35 – cue conflict trials). The remaining 120 trials contained random dot 59 textures that provided only stereoscopic cues (pure stereo cue trials). 60 http://mc.manuscriptcentral.com/issue-ptrsb 5 Submitted to Phil. Trans. R. Soc. B - Issue Page 6 of 29
1 2 3 4 Each virtual reality training session began only after subjects reported that the dichoptically 5 rendered bug halves were equally visible and aligned, ensuring fusion. Thereafter, each trial 6 commenced with participants placing the cylinder squarely on the starting plate. This signaled the 7 robot arm to orient a real surface, co aligned with the slant specified for that trial. For cue consistent 8 trials, the slant of the real surface matched the slant suggested by (consistent) monocular and 9 stereoscopic cues. For cue conflict trials, the robot arm oriented the real surface at the average of the 10 slants specified by the monocular and stereoscopic cues. A slanted virtual disk with the bug was then 11 displayed for 2 seconds. At the end of 2 seconds, the bug spun 360 degrees in 250 ms (one full turn) 12 13 to indicate the ‘go’ signal. Subjects were given 1.3 seconds after the go signal to move the cylinder 14 from the starting plate and place it flush on the target surface to squash the virtual bug. Subjects were 15 instructed to squash the bug only if both halves of the dichoptic bug were visible and aligned. The 16 feedback strategy was identical for both cue conflict and cue consistent trials. If the orientation of the 17 cylinder was within five degrees of the orientation of the target surface, the bug exploded. If not, the 18 bug quickly ran acrossFor the surface Review and disappeared. When Only subjects took longer than 1.3 seconds, the 19 trial was discarded and a message was displayed instructing the participants to respond faster. The 20 discarded trials were randomly administered later in the block. The same procedure was repeated 21 until the entire block was completed. Likewise, if subjects moved the cylinder prior to the go signal, 22 23 the computer aborted the trial, displayed an error message, and repeated the trial later in the same 24 block. For both cue consistent and cue conflict trials, in order to avoid a participant being able to tell 25 where the robot arm moved based on auditory cues, the robot arm was made to move to two different 26 slant angles before settling on the slant specified for that trial. The two slants were randomly chosen 27 from 10 60°. 28 29 After a few initial training sessions, subjects took roughly 7 8 minutes to complete a block. 30 Including calibration, each session lasted 60 75 minutes. Video game like scores were given for 31 correct and incorrect responses to motivate subjects. In order to make sure that subjects were not 32 33 suppressing during the task and were accurately withholding movements on trials in which they did 34 not perceive a bug, an additional 16 trials per block were included as ‘no go’ trials, that contained 35 only one half of the nonius bug (presented to the dominant eye only) for fixation. Performance on 36 these trials was, however, not recorded and will thus not be discussed further. 37 38 Nine subjects with normal vision provided baseline measurements for this task by 39 participating in the first 3 sessions of the VR task. Three of these subjects continued with the VR 40 task, one of them for 20 sessions and the other two for 30 sessions. Although these subjects were just 41 run to pilot the VR task, we report below their data for qualitative comparison with the stereo 42 43 deficient patients. 44 45 46 (c) Analysis of VR Training Data 47 48 49 In order to measure the influence of stereoscopic cues on the surface slant estimate used by 50 subjects to plan their hitting movements, we regressed the slant of the cylinder just prior to contact 51 with the surface (three optotrak frames 24 msec. prior to contact) against the slant depicted by the 52 texture on the surface and the slant depicted by stereoscopic disparities using the equation – 53 scyl = wmonosmono + wstereosstereo + k 54 , 55 where scyl is the slant of the cylinder just prior to making contact with the surface, smono is the slant 56 suggested by texture cues and sstereo is the slant suggested by stereo cues. We normalized the weights 57 to obtain a measure of the relative weight that subjects give to stereo to plan their movements 58 wstereo 59 Relative _ Stereo_Weight = . 60 wstereo + wmono http://mc.manuscriptcentral.com/issue-ptrsb 6 Page 7 of 29 Submitted to Phil. Trans. R. Soc. B - Issue
1 2 3 4 Only trials containing regular tiled texture, and therefore effective monocular cues, were used 5 for the regression. Trials were further limited to those containing slants between 27.5 and 42.5 6 degrees to minimize any effects on the regression that non linearities in the mapping between 7 stimulus and perceived slant might have. The cue consistent trials were pooled together with the cue 8 inconsistent trials to determine the regression coefficients. 9 10 11 (d) Behavioural assessments 12 13 Suppression: Subjects’ adjusted the relative contrast of the dominant eye’s stimulus (the half bug) to 14 match the perceived contrast of the non dominant eye’s half bug before each training session. We 15 use the interocular contrast ratio (ratio of non dominant to dominant eye contrast) that appears equal 16 as a measure of interocular suppression (a ratio of 1 = no suppression; high values = strong 17 suppression). 18 For Review Only 19 20 Stereoacuity: We used a standard clinical stereo test to evaluate changes in stereoacuity – the Randot 21 Circles Stereotest (Stereo Optical Co., Inc. – see ref 12 for details). Since this test contains monocular 22 cues, we also included the Pure Disparity Test (PDT using 1 cpd sine wave grating stimuli) described 23 by Ding and Levi [8], which contains no monocular cues. 24 25 Vergence Control Test: Vergence instability may negatively impact subjects’ stereo performance. 26 We developed a novel psychophysical test to track the effect of training on vergence accuracy. Test 27 28 stimuli consisted of two thin vertical lines presented either monocularly or dichoptically using 29 Crystal Eyes shutter goggles. These were surrounded by a fusion frame consisting of 4 small wedge 30 shaped markers with a central fixation dot shown binocularly at 0 disparity. Stimuli were viewed 31 from a test distance of 1.5 meters. Phase 1 of the test required subjects to adjust the contrast of the 32 dominant eye’s stimulus (method of adjustments) so that it perceptually matched that of the non 33 dominant eye. Phase 2 generated a baseline measure of monocular line alignment acuity of the 34 dominant eye by presenting the test lines monocularly (using the contrast estimated in phase 1). 35 Subjects were instructed to determine if the target (top) vertical line was displaced to the right or left 36 with respect to the bottom (reference) vertical line, and respond accordingly by right or left mouse 37 38 clicks. A random horizontal jitter was applied equally to both vertical lines in every trial, to prevent 39 the use of the central fixation point as a reference for target displacement. Four interleaved staircases 40 were constructed using 3 right /1 left, 1 right / 3 left, 5 right / 1 left and 1 right/5 left rules. Phase 3 41 of the test required subjects to perform the same task as described in Phase 2, but using dichoptically 42 presented lines and a binocularly presented fusion frame. This measure of dichoptic nonius alignment 43 is similar to that described by McKee & Levi [13]. The target line was presented to the dominant eye 44 and the reference line was presented to the non dominant eye. Additionally, as a suppression check, 45 subjects were instructed to click the mouse wheel if they only saw one line. Three consecutive 46 ‘suppression’ responses warranted readjusting the line contrasts to the dominant eye, and the 47 48 experiment was reset. A total of 600 trials were run. 49 Cumulative Gaussian psychometric functions were fit to the monocular and dichoptic test data. The 50 variance of vergence noise was estimated from the standard deviation parameters of the cumulative 51 Gaussian psychometric function fits for the monocularly and dichoptically presented line stimuli, 52 σ = σ 2 −σ 2 53 vergence dichoptic monocular . The 50% point on the psychometric function reflects bias or the 54 constant shifts in eye alignment. 55 56 57 58 (e) Statistical Analyses 59 60 Data analyses were performed using the SPSS package. To test the effect of time, we used one way http://mc.manuscriptcentral.com/issue-ptrsb 7 Submitted to Phil. Trans. R. Soc. B - Issue Page 8 of 29
1 2 3 4 repeated measures ANOVA including the three time points, pre test, post test and follow up. These 5 were followed by Bonferroni adjusted pairwise comparisons to better identify where the specific 6 differences lie. 7 Some of the measurements violated the normality assumption however; this was the case for 8 suppression, randot, and vergence measurements. For those, the Friedman statistical test was applied 9 This test is the non parametric alternative to the one way repeated measures ANOVA. Wilcoxon 10 Signed ranks tests were conducted to identify where the specific differences lie, with Bonferroni 11 correction to control for inflation of type I error. All p values reported are two tailed, except in pair 12 13 wise tests as mentioned. 14 15 3. Results 16 17 18 (a) Training increasesFor theReview accuracy of performance. Only 19 20 Training resulted in improved accuracy of slant judgments for both dot stimuli (slant specified by 21 stereoscopic cues only) and for textured stimuli containing either consistent monocular and 22 23 stereoscopic cues, or conflicting monocular and stereoscopic cues, for both normal and stereo 24 deficient observers. Fig. 3 shows the mean ‘hit rate’ over sessions (left column) for the 11 stereo 25 deficient (top) and for the 3 stereo normal observers (bottom). A trial was considered a hit trial, if 26 the orientation of the cylinder was within plus or minus five degrees of the orientation of the target 27 plate. The mean ‘hit rate’ was the average hit rate across all 5 slants. 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 http://mc.manuscriptcentral.com/issue-ptrsb 8 Page 9 of 29 Submitted to Phil. Trans. R. Soc. B - Issue
1 2 3
4 0.9 Stereo-deficient Mean (N = 11) Data Model 5 Cue consistent 0.3 Cue conflict 6 Dots: Pure stereo 7 0.8 8 9 0.7 10 0.2 e
11 t a r
12 t 0.6 i H
13 14 0.5 15 0.1 16 weight stereo Relative 17 0.4 18 For Review Only 19 Relative stereo weight 20 0.3 0.0 21 0 10 20 30 0 5 10 15 20 25 30 22 Session Session 23 0.9 Normal Mean (N = 3)
24 0.6 25 26 0.8 27 0.5 28 0.7 29 0.4
30 e t a
31 r
t 0.6 i 0.3
32 H 33 0.5 34 0.2 35 weight stereo Relative 36 37 0.4 0.1 38 39 0.3 0.0 40 0 10 20 30 41 0 5 10 15 20 25 30 Session Session 42 43 Fig. 3. Accuracy (left) and relative stereoweights (right) as training progresses. Left column : Mean ‘hit rate’ for the 11 44 stereo deficient (top) and for 3 stereo normal observers (bottom). A trial was considered a hit trial, if the orientation of 45 the cylinder was within plus or minus five degrees of the orientation of the target surface (the plate), which was set to be 46 at the mean orientation of the stereoscopic and monocular cues. The lines are the model output with the parameters shown 47 in Fig. 4 (see the text). Right column : The mean of the relative stereoscopic weights is plotted for the 11 stereo deficient 48 (top) and for 3 stereo normal observers (bottom; see section C of the data analysis for how these weights were derived). 49 50 In order to understand the improvement in performance, we fitted a simple Bayesian model 51 to the mean accuracy data. The model has four parameters: stereoscopic cue noise, monocular cue 52 noise, motor noise and a constant bias (k ). Sensory and motor noise are assumed to follow a zero 53 mean Gaussian distribution with variances σ 2 , σ 2 , and σ 2 . We assume that subjects have a 54 stereo mono motor 55 Gaussian prior distribution of surface slants whose variance reflects the actual variance of
56 stereoscopic and monocular cues presented in the task (σ prior = 9.2deg) [14], and integrate this prior 57 knowledge with the stereoscopic and the monocular sensory signals in a statistically optimal fashion. 58 Motor noise and bias are then added to the integrated sensory estimation. The resulting slant of the 59 2 60 cylinder predicted by the model follows a Gaussian distribution whose mean ( θ ) and variance (σ θ ) http://mc.manuscriptcentral.com/issue-ptrsb 9 Submitted to Phil. Trans. R. Soc. B - Issue Page 10 of 29
1 2 3 4 can be computed as follows: 5 θ = wsense (wstereoθstereo +(1− wstereo )θmono )+ k, where 6 w = (σ 2 σ 2 +σ 2 σ 2 )/(σ 2 σ 2 +σ 2 σ 2 +σ 2 σ 2 ), 7 sense prior stereo prior mono mono stereo prior stereo prior mono 2 2 2 8 wstereo = σ mono /(σ mono +σ stereo ), 9 θ is the slant of the surface in stereoscopic cue, and 10 stereo θ is the slant of the surface in monocular cue. 11 mono 12 σ 2 = w2 (σ 2 σ 2 )/(σ 2 +σ 2 )+σ 2 13 θ sense mono stereo mono stereo motor 14 15 As can be seen in Figure 4, the stereo deficient observers show a substantial reduction in 16 stereoscopic cue noise over the course of training, a much smaller reduction in monocular cues noise, 17 and essentially no change in the motor noise or bias parameters. The blue and red lines in Fig. 3 (top 18 left column) show theFor simulated Reviewchange in hit rate based onOnly these model parameters. 19 20 21 35 22 23 Stereo-deficient Mean (N = 11) Stereoscopic cue noise 24 Monocular cue noise 25 30 Motor noise 26 Bias 27 28 ) 29 s 25 30 e e 31 r g
32 e
33 d (
20
34 .
35 D .
36 S
37 e
s 15
38 i
39 o n 40 y
41 r 42 o 10 s
43 n 44 e S 45 46 5 47 48 49 50 0 51 52 53 54 0 10 20 30 55 56 Session 57 58 Fig. 4 Output of a simple Bayesian model fit to the mean accuracy data in the 11 stereo deficient subjects. As participants 59 trained in the task a significant reduction in stereoscopic and monocular cue noise is observed with motor noise and bias 60 remaining stable throughout training. Thus, participants’ estimate of depth cues became more reliable with training. http://mc.manuscriptcentral.com/issue-ptrsb 10 Page 11 of 29 Submitted to Phil. Trans. R. Soc. B - Issue
1 2 3 4 (b) Training increases the weighting of stereo cues. 5 The principal outcome measure, derived from the bug squashing data, is the relative stereo 6 weight used by subjects to plan their hitting movements for stimuli containing regular textures. For 7 these stimuli, the nine normally sighted control observers had strong relative stereo weights (0.39 ± 8 0.05) when tested under binocular conditions. Four of these subjects were retested with the 9 dominant eye patched. Under these monocular conditions, their stereo weights were negligible (0.03 10 ± 0.02). 11 12 When first put on the task, our stereo deficient subjects were quite poor at the task as 13 measured by the error in their hitting movements and a strong regression to the mean slant. One to six 14 practice sessions were therefore used, depending on when accuracy and bias reached an asymptote. 15 The number of practice sessions varied per subject, but data from the first three sessions with 16 relatively stable performance were used to compute the pre training weights. Owing to reduced 17 variability in weight estimates with training, post training (and follow up) weights were based on 18 data derived from twoFor sessions afterReview training. Only 19 The relative stereo weights of our 11 stereo deficient subjects increased by, on average, a 20 factor of 3.07 ± 0.85 (Fig. 3 – top right panel shows the mean data). In contrast, the three stereo 21 22 normal control observers showed only a factor of ≈ 1.3 change in relative stereo weights as training 23 proceeded (see Figure 3, bottom right panel) – from 0.35 ± 0.05 (average of the first 3 sessions) to 24 0.46 ± 0.01 (average of the last 2 sessions). It is interesting to note that the feedback given in the 25 experiment would have provided a signal not to weight the stereo cue any higher than 0.5. 26 A one way repeated measures omnibus ANOVA on log transformed relative stereo weights 27 (to offset violations of normality) was conducted to compare group performance at pre, post, and 28 follow up. The results showed that subjects assigned significantly more weight to stereoscopic cues 29 after training [F (2, 20) = 7.77, p = 0.003]. Bonferroni adjusted pairwise comparisons showed 30 increased stereo weights (decreased monocular cue weights) after training (p = 0.004 one tailed, 31 32 mean change = 0.15 ± 0.04), with the difference being still seen at follow up (p = 0.028 one tailed, 33 mean change = 0.13 ± 0.04), indicating that the improvements were largely retained after two months 34 of no intervention. 35 Fig. 5 shows the post vs. pre training (solid symbols) relative stereo weights for each of the 36 11 stereo deficient observers. Points above the unity line indicate an increase in relative stereo 37 weights following training. Eight of the eleven subjects showed a numerical increase in stereo 38 weights following training, with these improvements being largely maintained at follow up, 2 months 39 later (except for two subjects who showed regression in gains). The inset in Fig. 5 shows the 40 41 evolution of this increase in relative stereo weights for a strabismic subject (S1) who initially showed 42 very low weighting of the stereo cues. A bootstrap analysis (10,000 iterations), performed to assess 43 the intra individual changes in relative stereo weights after training compared with their respective 44 baselines indicates that the upweighting of the stereoscopic cues following training was statistically 45 significant for six of the eleven stereo deficient subjects at post training (all ps < .001). In addition, 46 two of them (M2 and S4) showed the right trend from pre to post training to follow up, but failed to 47 reach significance in part due to the high variance of the pre test estimates. 48 49 50 51 52 53 54 55 56 57 58 59 60 http://mc.manuscriptcentral.com/issue-ptrsb 11 Submitted to Phil. Trans. R. Soc. B - Issue Page 12 of 29
1 2 3 4 0.8 0.5 5
6 0.4 7 0.7 8 0.3 9 10 0.2 0.6 11 0.1 12 Relativeweight stereo 13 0.0 14 0 10 20 30 40 15 0.5 Session 16 17 18 0.4 For Review Only 19 20 Anisometropic 21 Post FU 22 0.3 A1 A2 23 24 Strabismic 25 S1 0.2 S2 26 S3 27 S4
28 Mixed 29 0.1 RelativePost-training & FU Stereo Weight M1 30 M2 M3 31 M4 32 M5 33 0.0 Controls (N = 3) 34 35 36 37 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 38 Pre-training Relative Stereo Weight
39 40 Figure 5. Post vs. Pre training (solid symbols) and Follow Up (open symbols) relative stereo weights. Each colored 41 symbol shows the data of a single stereo deficient observer. The gray diamond near the abscissa shows the pre and 42 post training mean relative stereo weights of the 3 normal control observers who underwent training. Data above the 43 diagonal unity line indicate increases in relative stereo weights. The inset shows the evolution of the increased relative 44 stereo weights for strabismic subject S1 (indicated in the main figure by an arrow). Solid symbols in the inset represent 45 the pre to post training weights and the open symbols the pre to follow up training weights. 46 47 48 49 (b) Behaviourial measures of suppression, stereoacuity and visual acuity following 50 training. 51 52 Suppression: 53 Suppression was substantially reduced following training. The Friedman statistical test was 54 applied since the data were non normally distributed. We found that the bug squashing training 55 2 56 resulted in a significant change to levels of suppression (Fig. 6, Friedman test, χ (2, N = 11) = 13.35, 57 p = 0.001). Specifically, reduced suppression was found in all but one subject after training 58 (Wilcoxon signed ranks test, Z = 2.67, p = 0.0025 one tailed, r = 0.57; Median change in 59 interocular contrast ratios = 0.44). These effects were retained at follow up compared to baseline (Z = 60 2.67, p = 0.0025 one tailed, r = 0.57; Mdn change = 0.35), with no significant difference between http://mc.manuscriptcentral.com/issue-ptrsb 12 Page 13 of 29 Submitted to Phil. Trans. R. Soc. B - Issue
1 2 3 4 the magnitudes of suppression measured at post training vs. at follow up (Z = 1.46, p = 0.12 one 5 tailed, r = 0.31; Median change = 0.0). 6 7 3.0 Figure 6. The effect of training on 8 Anisometropic suppression. 9 Post FU Pre vs. Post training (solid symbols) and
) A1 Follow Up (open symbols) interocular 10 g A2 n contrast ratio (weak/strong). Each 11 o r t colored symbol shows the data of a single
s Strabismic
12 /
k 2.5 S1 stereo deficient observer. The gray 13 a S2 e diamond shows the pre and post training
14 w S3 ( mean data of the 3 normal control S4
15 o i observers who underwent training. Data t
16 a
r below the diagonal unity line indicate
Mixed 17 t s M1 reduced suppression. A horizontal offset a
18 r 2.0 M2 t For Review Onlyhas been applied to avoid symbols (post 19 n M3 o and FU) from overlapping.
c M4 20 M5 U
21 F
& Controls (N=3) 22
g
23 n i 1.5 n 24 i a r t
25 - t
26 s o
27 P 28 29 1.0 30 31 1.0 1.5 2.0 2.5 Pre-training contrast ratio (weak/strong) 32 33 34 Stereoacuity: 35 Importantly, bug squashing led to improvements not only on the trained task, but also in the 36 untrained stereo acuity tasks. All six subjects who showed significant increases in relative stereo 37 weights at post test, also demonstrated an improvement in stereoacuity as measured by the Randot 38 39 Circles Test, (Fig. 7a solid symbols), and five of these six subjects also showed significant 40 improvements in stereoacuity (all p values < 0.01; 1000 bootstrap iterations were performed to assess 41 intra individual differences in pre and post stereothresholds) on the Pure Disparity Test (Fig. 7b). 42 These improvements were largely maintained at follow up two months after the cessation of training 43 (Fig. 7 b open symbols), with the exception of two anisometropic amblyopes who either 44 discontinued use of contact lenses post training (A2), or used them only sparingly (A1). These two 45 subjects demonstrated some regression of stereoscopic gains at follow up. Finally, the two subjects 46 who showed the right numerical trend in relative stereo weights as the experiment proceeded (M2 47 and S4) did not show stereoacuity transfer as measured either with the Randot or with the Pure 48 49 Disparity test. 50 51 52 Randot Circles Test: Statistical tests were run on log stereo thresholds. A value of 2.78 log arc 53 seconds (or 600 arc seconds) was arbitrarily assigned if subjects failed the Randot Circles test. 54 Subjects unable to perform at that largest disparity were labeled as ‘stereoblind’. There was a 55 2 56 significant change in stereoacuity over time, Friedman test, χχχ (2, N = 11) = 6.64, p = 0.031. Post hoc 57 comparisons showed that subjects exhibited a trend for improved stereoacuity after training 58 (Wilcoxon, Z = 1.866, p = 0.039 one tailed, r = 0.4; Median change = 0.46 log arc sec, equivalent 59 to 65% median improvement), and at follow up compared to pre training baselines (Z = 2.106, p = 60 .02 one tailed, r = 0.45; Median change = 0.15 log arc sec, equivalent to 30% improvement). http://mc.manuscriptcentral.com/issue-ptrsb 13 Submitted to Phil. Trans. R. Soc. B - Issue Page 14 of 29
1 2 3 4 Pure Disparity Test: A one way repeated measures ANOVA showed that stereoacuity as measured 5 by the Pure Disparity Test improved significantly after training (main effect of time: F (2, 20) = 4.86, 6 p =0.019). Post hoc comparisons using Bonferroni correction showed that the stereoacuity improved 7 post training compared to pre training baselines (p = 0.02 one tailed; mean change = 0.28 ± 0.09 log 8 arc sec, equivalent to 35.7 ± 10.9% improvement), and this improvement was largely retained at 9 follow up (p = 0.046 one tailed; mean change at follow up from pre training = 0.41 ± 0.16 log arc 10 sec). 11 12 a. 1000 Anisometropic Randot circles b. 13 Post FU Pure Disparity Test A1 14 A2 1000 15 Strabismic
Stereoblind S1 16 S2 S3 17 S4 18 Mixed For Review Only M1 19 M2 M3 20 M4 100 M5 21 100 Controls (N = 3) 22 23 24 25 26 Post-training stereoacuity (arcsec)Post-training & FU
27 Post-training&FU stereoacuity (arcsec) 10 28 10 29 10 100 Stereoblind1000 10 100 1000 30 Pre-training stereoacuity (arcsec) Pre-training stereoacuity (arcsec) 31 c. d. 32 10 10
33 9 9 34 35 8 8 36 37 7 7
38 6 6 39 40 5 5 41 4 42 4
43 3 3 44 45 2 2 Changestereoin Ratio) acuity (Pre:Post 46 acuityChangestereo(Pre:Post in Ratio) 1 Randot circles 1 47 Pure Disparity Test
48 0.0 0.1 0.2 0.3 0.4 0.5 0.0 0.1 0.2 0.3 0.4 0.5 49 Change in Relative Stereo Weight (Post -Pre) Change in Relative Stereo Weight (Post -Pre)
50 Figure 7. Panel A Randot Circles test and Panel B Pure Disparity Test . Pre vs. Post training (solid symbols) 51 and Follow Up (open symbols) stereo thresholds (Randot Circles test – Panel A; Pure Disparity Test – Panel B). 52 Each colored symbol shows the data of a single stereo deficient observer. The gray diamond is the mean pre/post data 53 of the 3 control subjects who underwent training. Data below the diagonal unity line indicate decreases in 54 stereothresholds (i.e., improved stereoacuity). The red arrow in A indicates the subject (S1), with the largest change in 55 stereo weights. Note that small horizontal shifts have been applied to some of the data points to avoid symbols 56 overlapping. Panel C Randot Circles test and Panel D Pure Disparity Test. The change in stereo thresholds 57 (Pre:Post ratio) vs the change in relative stereo weights (Post Pre) for the Randot circles test (Panel C), and for the 58 Pure Disparity Test (PDT) (Panel D), which has no monocular cues. The lines show the best fitting linear regressions 59 for the Randot circles (solid line) and PDT (dashed line) respectively. 60 http://mc.manuscriptcentral.com/issue-ptrsb 14 Page 15 of 29 Submitted to Phil. Trans. R. Soc. B - Issue
1 2 3 4 While these results show that learning a visuomotor task that focused on surface slant 5 estimation transfers to stereo depth judgments, a stronger test for generalization would be the ability 6 to predict improvements on the transfer tests from the post training changes in stereo weights. Fig. 7 7 C and D plot changes in stereo threshold (Pre:Post ratio) as measured by the Randot Circles and Pure 8 Disparity Test, respectively, against changes in stereo weights (post – pre relative stereo weights). 9 The regression lines fit the data well, indicating that training induced changes in stereo weights 10 significantly predicted improvements on the Randot Circles (β = 0.75 , t = 3.44, p = 0.007; R2 = 0.57), 11 2 12 and Pure Disparity Test (β = 0.9, t = 6.0, p < 0.0001; R = 0.8). 13 14 Visual acuity: 15 We examined whether direct stereo training might also offer training induced benefits to 16 visual acuity (Fig. 8). Six of the eleven subjects showed an improvement in visual acuity 17 immediately post training (> 0.04 logMAR), with two of these subjects regressing and one lost at 18 follow up. Of the threeFor subjects thatReview showed substantial (≥Only 0.10 logMAR) improvements, two showed 19 no post training improvements in stereopsis, suggesting changes in the non dominant eye acuity are 20 not tightly coupled to the training related increases in relative stereo weights, in agreement with the 21 22 results of Ding & Levi (2011). For the group as a whole, there was a trend for improved visual 23 acuity in the non dominant eye after training compared to the pre training baseline; however, this 24 effect was only marginally significant (repeated measures ANOVA, F (2, 20) = 3.23, p = 0.061). 25 Acuity of the dominant eye was unaltered post training (F (2, 20) = 0.72, p = 0.5). 26 27 0.3 Anisometropic Figure 8. 28 Post FU Pre vs. Post training (solid symbols) and A1 29 A2 Follow Up (open symbols) visual acuity 30 (LogMAR). Each colored symbol Strabismic shows the data of a single stereo deficient 31 S1 observer. The gray diamond shows the 32 0.2 S2 33 S3 mean pre/post data of the 3 normal S4 control observers who underwent 34 Mixed training. Data below the diagonal unity 35 M1 line indicate increases in visual acuity. 36 M2 37 0.1 M3 M4 38 M5 39 Controls (N=3) 40 41 42 0.0 43 44
45 Post-trainingAcuity&FU (LogMAR) Visual 46 47 -0.1 48 -0.1 0.0 0.1 0.2 0.3 49 Pre-training Visual Acuity (LogMAR) 50 51 52 Vergence: 53 54 We measured vergence noise in eight of the participants (Supplemental Figure S.1). Prior to 55 bug squashing training, the vergence noise in the experimental group was 4.9 times higher (9.72 ± 56 3.5 arc minutes, n = 8) compared to the normally sighted subjects (1.97 ± 0.2 arc minutes, n = 4). For 57 the stereo deficient group as a whole, there were no significant differences in vergence noise 58 measured pre training, post training and at follow up (Friedman test, χ2 (2, N = 8) = 2.25, p = 0.36). 59 Likewise, there were no changes in vergence bias with training (Friedman test, χ2 (2, N = 8) = 3.25, p 60 http://mc.manuscriptcentral.com/issue-ptrsb 15 Submitted to Phil. Trans. R. Soc. B - Issue Page 16 of 29
1 2 3 4 = 0.24). Four of these eight stereo deficient subjects showed improvements in relative stereo weights 5 following training, two of whom showed a post training reduction of vergence noise, one showed an 6 increase, and one showed no change. There was a lack of consistency in the direction of change post 7 training. Thus there is insufficient evidence to indicate that changes in the vergence noise can explain 8 the training related improvements in stereo weights in these subjects. We acknowledge that our 9 vergence measure is a subjective measure, and thus may not be sufficiently sensitive to detect very 10 small changes in oculomotor control (≈ 1.5 – 2 arc minutes in our normal control subjects – 11 Supplemental Figure S1). 12 13 14 Determinants of stereo weights improvements: 15 16 We also examined other potential contributing factors to post training changes in subjects’ 17 relative stereo weights. First, changes in interocular suppression failed to predict changes in stereo 18 weights ( β = 0.2, t =For 0.65 p = 0.5;Review R2= 0.04). Second, weOnly investigated whether any of the pre 19 training measures – logMAR acuity (non dominant eye), interocular difference in acuity, stereo 20 weights, age, and Randot and PDT stereosensitivities – could predict training related changes in 21 stereo weights. Multiple regression analysis revealed that only the initial Pure Disparity Test 22 2 23 significantly predicted training induced changes in stereo weights ( β = 0.8, t = 3.9, p = 0.003 R = 24 0.63 ). None of the other measures did (p > 0.05). Subjects with the best stereo acuity on the pure 25 disparity tests at the outset, showed the greatest increases in relative stereo weights, while those with 26 no measurable stereopsis showed no change. 27 28 29 30 4. Discussion 31 32 While there have been a large number of studies of the effects of perceptual learning and 33 videogame play in adults with amblyopia [for recent reviews see refs 15 18], only a few studies have 34 trained stereopsis directly (for a review ref 3). Our results show that training in a natural visuo motor 35 task in which some stimuli contained monocular texture as well as stereoscopic cues (cue consistent 36 37 trials), some contained only stereoscopic cues and some contained conflicting monocular and stereo 38 cues, enabled adults with long standing deficits in stereo vision to upweigh their reliance on 39 stereoscopic cues (relative to monocular cues). Following training participants showed improved 40 accuracy for detecting slants in depth. Our simple Bayesian modeling indicates a very substantial 41 reduction in stereoscopic cue noise over the course of training in stereo deficient observers, along a 42 more modest reduction in monocular cues noise. The dramatic increase in stereoscopic cue reliability 43 as training processed is consistent with the increased reliance of the observers on stereoscopic cues 44 (i.e., increased relative stereo weights). In addition, stereo deficient observers also showed reduced 45 suppression, significant improvement in stereoacuity and a weak trend for improved visual acuity. 46 47 48 Out of the 11 participants trained, 6 showed significantly greater reliance on stereoscopic cues 49 as their training on our virtual reality bug squashing game progressed, with an additional two 50 showing the right numerical trend. Of the six who showed a significant effect, all showed improved 51 stereo acuity as measured by the randot at pre and post training. Although only marginally 52 significant, we note, as others have done before, that the Randot may lack the needed sensitivity to 53 assess pre to post test improvements. For example, strabismic subject S1 (shown by the arrow in 54 Figs 5 and 7), achieved a stereoacuity of 20 arc sec., the lower limit of the Randot Circles test – this 55 is the subject shown in the inset of Figure 5 whose relative stereo weights improved substantially 56 57 over the course of training. We suspect that this floor (lowest tested value on the test) underestimates 58 S1’s stereoacuity, since he demonstrated a stereoacuity of ≈ 9 arc sec on the Pure Disparity Test. 59 Five participants showed highly significant post training improvements on the Pure Disparity Test 60 http://mc.manuscriptcentral.com/issue-ptrsb 16 Page 17 of 29 Submitted to Phil. Trans. R. Soc. B - Issue
1 2 3 4 (Table S1), and crucially, the best determinant of whether stereo weights improved was performance 5 on the Pure Disparity test at pre test. 6 7 Use of different sensory cues: 8 9 Adults with normal visual experience reduce sensory uncertainty by integrating information 10 from different modalities (e.g. touch and vision [19, 20]) and within the same modality (e.g. 11 binocular disparity and texture information) for judging slant in depth [10, 21, 22]. However, 12 13 observers deprived of normal binocular visual experience early in life have poor or absent stereopsis, 14 and therefore rely on texture information for making judgments of surface slant. Thus, in our sample 15 of observers with abnormal binocular vision, the pre training relative stereo weight was, on average ≈ 16 0.11 ± .03. In contrast, normal control subjects had a relative stereo weight of 0.39 ± 0.05. Over the 17 course of training, the relative stereo weights of the stereo deficient subjects increased substantially. 18 It is interesting to noteFor that while Review the adult visual system isOnly optimized for reducing sensory 19 uncertainty by integrating texture and stereoscopic cues to slant, mature sensory integration is not 20 evident until age 12 [23]. For infants and young children the limitation is not due to insensitivity to 21 one cue but to an immaturity in sensory integration. In contrast, adults deprived of normal binocular 22 23 vision are initially insensitive to the disparity information. With experience in bug squashing, the 24 disparity information becomes more reliable. Importantly, our bug squashing training involves 25 integrating not just multiple visual cues, but also the rich information from tactile and kinesthetic 26 feedback [22]. 27 28 Real life impact: 29 30 Following training two participants reported better distance judgement during driving, and 31 one was able to appreciate depth from autostereograms for the first time. Finally, one important 32 33 practical outcome of our study is the finding that strabismic subjects who demonstrated post training 34 stereo improvements were able to perform the clinical Randot test without the use of prisms, despite 35 the presence of uncorrected ocular deviation. Compensating the deviation with prisms did not further 36 improve stereo acuity in any of our strabismic subjects. We hypothesized that if this finding were 37 true, then the stereo weights should similarly be unaffected by lack of compensation for deviation. To 38 assess this, we measured stereo weights without prisms for six subjects who were initially trained on 39 the bug squashing task with prisms. Our results showed that the difference between relative stereo 40 weights assessed with and without prisms was negligible (mean = 0.01 ± 0.02). 41 42 43 Our strabismic subjects who recovered stereopsis had stereoacuities after training, as low as 9 44 arc sec (S1 on the PDT). How do we reconcile the improvement in stereopsis in the presence of an 45 uncorrected strabismus? It has long been known that fusion is not a requirement for stereopsis. For 46 example, observers with normal vision can correctly localize objects in depth even when they appear 47 double [24]. Westheimer & Tanzman [25] demonstrated that observers with normal binocular vision 48 can correctly identify the depth of diplopic stimuli with disparities up to about 7 degrees. Blakemore 49 [26] reported that the largest disparity that supported signed depth judgments was ~12 50 degrees. More recently, Dengler & Kommerell [27] showed that normal subjects could distinguish 51 binocularly disparate images from monocular double images with the same angular separation over 52 53 large distances (up to 21 degrees), where one eye’s target was presented in the fovea, and the other 54 eye’s target was presented in the periphery. This is similar to the situation in strabismus, and, as 55 Dengler & Kommerell speculate, these long range connections between the fovea of one eye and the 56 periphery of the other may be the basis for anomalous correspondence in patients with strabismus. 57 One potential issue with these studies is that subjects can assign a 'correct' disparity to an isolated 58 monocular target in one eye, as though it were matched to an invisible target in the fovea [28, 59 29]. Dengler & Kommerell, like the Foley et al. [30] study before it, did employ monocular targets 60 as controls, where the subjects had to identify the monocular targets with a label "on". What is http://mc.manuscriptcentral.com/issue-ptrsb 17 Submitted to Phil. Trans. R. Soc. B - Issue Page 18 of 29
1 2 3 4 interesting is that many subjects did respond based on 'eye of origin' information, and so their data 5 were discarded. Perhaps there were subtle differences that allowed some subjects to discriminate the 6 targets that were supposed to be labeled 'crossed' from the targets that were supposed to be labeled 7 'on'. However, the main point is that there is evidence for at least coarse stereopsis for unfused 8 stimuli in normal vision. Coarse stereopsis is considered to be important to extend the range of 9 disparity sensitivity, as a guide to vergence eye movements, and as a “back up” system for 10 individuals with strabismus [31]. Indeed, there is recent evidence that coarse stereopsis develops 11 much earlier than fine stereopsis [32], and that it may be relatively “spared” in individuals with a 12 13 history of amblyopia [33]. Whether the same mechanisms are capable of supporting stereoacuity 14 better than one arc minute in strabismic patients, or whether there are specialized mechanisms 15 supporting anomalous correspondence is a matter for further research. For now, the present study 16 documents that it is possible for strabismic patients to recover stereoacuity as low as 9 arc sec. 17 18 Why does recovery ofFor stereopsis Review require heroic methods? Only 19 20 Isn’t the rich real world stimulation enough? We believe that there are several reasons that 21 the natural environment is not sufficient. First, under normal binocular viewing, stimuli in the two 22 23 eyes do not have equal perceived contrast, leading to suppression of the weak eye by the strong eye 24 [18, 34, 35]. Moreover, for strabismic subjects, the two eyes’ images fall on non corresponding 25 retinal areas, precluding normal fusion and triggering suppression. Thus under normal viewing higher 26 brain areas involved in sensory integration [36 39] receive weak and unreliable information from 27 the weak eye, resulting in a down weighting of disparity information. In contrast, in our bug 28 squashing game, the stimuli to the two eyes were perceptually matched, by reducing the contrast 29 presented to the strong eye and were carefully aligned, so that the images can be fused. By 30 combining this new aligned and balanced visual input with a visuomotor task (bug squashing), 31 providing monocular depth cues as a scaffold, and giving trial by trial force feedback, subjects can 32 33 learn to attend to input from the amblyopic eye [40] and can learn the correlations between the 34 “corrected” visual input and the depth of objects in the world. Finally, we should emphasize that our 35 bug squashing training was conducted in a virtual reality environment enabling us to recreate “the 36 natural way in which perception and action are intimately entwined with the environment” [41]. 37 38 Our current results, taken together with previous studies (reviewed in ref 3), suggest that to 39 optimize stereo recovery it is critical to initially provide aligned and balanced input to the two eyes. 40 This may require substantial fusion training [8] prior to direct stereo training in a rich environment 41 which combines a natural visuomotor task. 42 43 44 5. Summary and Conclusions 45 46 47 We trained adults who were stereoblind or stereo deficient on a natural visuo motor “bug 48 squashing” task in which some stimuli contained monocular texture cues as well as stereoscopic cues 49 that were consistent with each other, some contained only stereoscopic cues and some contained 50 conflicting monocular and stereoscopic cues. Following training, 8 out of 11 observers upweighted 51 their reliance on stereoscopic cues (relative to monocular cues), while 6 of them showed reduced 52 suppression and significant improvements in stereoacuity. 53 Our results have important implications for the recovery of visual function late in life, well 54 outside the “critical period”, which until recently was thought to offer the only real scope for 55 plasticity. More broadly, our approach demonstrates the potential power offered by virtual reality for 56 57 perceptual training of all kinds. 58 59 60 http://mc.manuscriptcentral.com/issue-ptrsb 18 Page 19 of 29 Submitted to Phil. Trans. R. Soc. B - Issue
1 2 3 4 Additional Information 5 6 7 Acknowledgments 8 Our dear friend and colleague David Knill died tragically last year. We, and indeed the entire vision community, will miss 9 him sorely, and we dedicate this paper to his memory. We thank our research assistants and study coordinators Olga 10 Pikul, Leslie Chylinsky, Aleksandra Fazlipour and Gary Volkell. We also thank Dr. Hu for assistance with programming. 11 We thank Drs. Gearinger and DePaolis for referring patients to our study. Finally, this work could not have happened without the cooperation of our subjects. 12 13 14 Ethics 15 The study was approved by the Institutional Review Board at the University of Rochester, and informed consent was 16 obtained from all subjects. 17 18 Data Accessibility All manuscripts which reportFor primary dataReview (usually research articles) Only should include a Data Accessibility section which 19 states where the article's supporting data can be accessed. This section should also include details, where possible, of 20 where to access other relevant research materials such as statistical tools, protocols, software etc. If the data has been 21 deposited in an external repository this section should list the database, accession number and link to the DOI for all data 22 from the article that has been made publicly available, for instance: 23 DNA sequences: Genbank accessions F234391-F234402 (http://dx.doi.org/xxxxx ) 24 Phylogenetic data, including alignments: TreeBASE accession number S9123 ( http://dx.doi.org/xxxxx ) 25 Climate data and MaxEnt input files: Dryad doi:10.5521/dryad.12311 ( http://dx.doi.org/xxxxx )
26 If the data is included in the article’s Supplementary Material this should be stated here, for instance: 27 The datasets supporting this article have been uploaded as part of the Supplementary Material. 28 29 Authors' Contributions 30 The initial conceptual framework for the VR training was developed by DK. DK, DL and DB designed the study and 31 obtained a research grant from the National Eye Institute. Adapting the VR training paradigm for clinical application: IV 32 for design aspects and feedback, AY and DK for the programming. Piloting and fine-tuning of the training and vision 33 assessments, primarily IV with contributions from AY, SH and DK; Running the Study: Primarily IV with contributions from SH and the supervision of DB; Data analysis: primarily IV with the help of DK and DB; modeling of setero-weight 34 data: O-SK; JD developed the PDT and assisted with its data analysis. Writing: primarily IV and DL with the feedback of 35 DK and DB, and other authors' contributing as needed. Final figures: DL. 36 37 DB and DL are co-senior authors. 38 39 Competing Interests We have no competing interests. 40
41 Funding 42 This research was supported by grants RO1EY020976 from the National Eye Institute to D.L., D.K. and D.B. 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 http://mc.manuscriptcentral.com/issue-ptrsb 19 Submitted to Phil. Trans. R. Soc. B - Issue Page 20 of 29
1 2 3 4 References 5 1. Tam, W. J., and L. B. Stelmach. (1998) 15. Levi, D. M., and R. W. Li. (2009) 29. Wilcox, L. M., J. M. Harris, and S. P. 6 "Display Duration and Stereoscopic Depth "Perceptual Learning as a Potential McKee. (2007) "The Role of Binocular Discrimination," Can J Exp Psychol , vol. 52 , Treatment for Amblyopia: A Mini-Review," Stereopsis in Monoptic Depth Perception," 7 no. 1 56-61. Vision Res , vol. 49 , no. 21 2535-49. Vision Res , vol. 47 , no. 18 2367-77. 8 9 2. Zaroff, C. M., M. Knutelska, and T. E. 16. Levi, D. M. (2012) "Prentice Award 30. Foley, J. M., T. H. Applebaum, and W. A. Frumkes. (2003) "Variation in Stereoacuity: 10 Lecture 2011: Removing the Brakes on Richards. (1975) "Stereopsis with Large Normative Description, Fixation Disparity, Plasticity in the Amblyopic Brain," Optom Vis Disparities: Discrimination and Depth 11 and the Roles of Aging and Gender," Invest Sci , vol. 89 , no. 6 827-38. Magnitude," Vision Res , vol. 15 , no. 3 417-21. 12 Ophthalmol Vis Sci , vol. 44 , no. 2 891-900.
13 3. Levi, D. M., D. C. Knill, and D. Bavelier. 17. Hess, R. F., and B. Thompson. (2015) 31. Wilcox, L. M., and R. S. Allison. (2009) 14 (2015) "Stereopsis and Amblyopia: A Mini- "Amblyopia and the Binocular Approach to "Coarse-Fine Dichotomies in Human Its Therapy," Vision Res , vol. 114 4-16. Stereopsis," Vision Res , vol. 49 , no. 22 2653- 15 Review," Vision Res , vol. 114 17-30. 65. 16 4. Barry, S.O. (2009). Fixing My Gaze: A 18. Tsirlin, I., L. Colpa, H. C. Goltz, and A. M. 17 Scientist's Journey Into Seeing in Three Wong. (2015) "Behavioral Training as New 32. Giaschi, D., S. Narasimhan, A. Solski, E. 18 Dimensions. New York, NY: Basic Books. Treatment for Adult Amblyopia: A Meta- Harrison, and L. M. Wilcox. (2013a) "On the For ReviewAnalysis and Systematic Review," InvestOnly Typical Development of Stereopsis: Fine and 19 5. Bridgeman, B. (2014) "Restoring Adult Ophthalmol Vis Sci , vol. 56 , no. 6 4061-75. Coarse Processing," Vision Res , vol. 89 65-71. 20 Stereopsis: A Vision Researcher's Personal 21 Experience," Optom Vis Sci , vol. 91 , no. 6 19. Ernst, M. O., and M. S. Banks. (2002) 33. Giaschi, D., R. Lo, S. Narasimhan, C. e135-9. "Humans Integrate Visual and Haptic Lyons, and L. M. Wilcox. (2013b) "Sparing of 22 Information in a Statistically Optimal Coarse Stereopsis in Stereo-deficient 23 6. Nakatsuka, C., B. Zhang, I. Watanabe, J. Fashion," Nature , vol. 415 , no. 6870 429-33. Children with a History of Amblyopia," J Vis , Zheng, H. Bi, L. Ganz, E. L. Smith, R. S. vol. 13 , no. 10 . 24 Harwerth, and Y. M. Chino. (2007) "Effects of 20. Ernst, M. O., M. S. Banks, and H. H. 25 Perceptual Learning on Local Stereopsis and Bulthoff. (2000) "Touch Can Change Visual 34. Ding, J., S.A. Klein, and D.M. Levi. (2013) Neuronal Responses of V1 and V2 in Prism- 26 Slant Perception," Nat Neurosci , vol. 3 , no. 1 "Binocular Combination in Abnormal Reared Monkeys," J Neurophysiol , vol. 97 , 69-73. Binocular Vision," J. Vis. 27 no. 4 2612-26. 28 29 7. Astle, A. T., P. V. McGraw, and B. S. Webb. 21. Knill, D. C., and J. A. Saunders. (2003) "Do 35. Vedamurthy, I., M. Nahum, D. Bavelier, (2011) "Recovery of Stereo Acuity in Adults Humans Optimally Integrate Stereo and and D. M. Levi. (2015) "Mechanisms of 30 with Amblyopia," BMJ Case Rep , vol. 2011 . Texture Information for Judgments of Recovery of Visual Function in Adult 31 Surface Slant?," Vision Res , vol. 43 , no. 24 Amblyopia through a Tailored Action Video 8. Ding, J., and D. M. Levi. (2011) "Recovery 2539-58. Game," Sci Rep , vol. 5 8482. 32 of Stereopsis through Perceptual Learning in 33 Human Adults with Abnormal Binocular 22. Knill, D. C., and D. Kersten. (2004) 36. Welchman, A. E., A. Deubelius, V. 34 Vision," Proc Natl Acad Sci U S A , vol. 108 , "Visuomotor Sensitivity to Visual Conrad, H. H. Bulthoff, and Z. Kourtzi. (2005) 35 no. 37 E733-41. Information About Surface Orientation," J "3d Shape Perception from Combined Depth Neurophysiol , vol. 91 , no. 3 1350-66. Cues in Human Visual Cortex," Nat Neurosci , 36 9. Xi, J., W. L. Jia, L. X. Feng, Z. L. Lu, and C. vol. 8 , no. 6 820-7. B. Huang. (2014) "Perceptual Learning 37 23. Nardini, M., R. Bedford, and D. Improves Stereoacuity in Amblyopia," Invest Mareschal. (2010) "Fusion of visual cues is 37. Morgan, M. L., G. C. Deangelis, and D. E. 38 Ophthalmol Vis Sci , vol. 55 , no. 4 2384-91. not mandatory in children," Proc Natl Acad Angelaki. (2008) "Multisensory Integration in
39 Sci U S A , vol. 107 , no. 39 17041-46. Macaque Visual Cortex Depends on Cue 10. Knill, D. C. (2005) "Reaching for Visual 40 Reliability," Neuron , vol. 59 , no. 4 662-73. Cues to Depth: 41 The Brain Combines Depth Cues Differently 24. Ogle, K. N. (1952) "Disparity Limits of 42 for Motor Control Stereopsis," AMA Arch Ophthalmol , vol. 48 , 38. Ban, H., T. J. Preston, A. Meeson, and A. 43 and Perception," J Vis , vol. 5 , no. 2 103-15. no. 1 50-60. E. Welchman. (2012) "The Integration of Motion and Disparity Cues to Depth in 44 11. Knill, D. C. , J. A. Saunders (2003) "Do 25. Westheimer, G. And I. J. Tanzman. (1956) Dorsal Visual Cortex," Nat Neurosci , vol. 15 , 45 humans optimally integrate stereo and “Qualitative depth localization with diplopic no. 4 636-43. 46 texture information for judgments of slant " images,” ," J Opt Soc Am , vol. 46 , no. 2 116- Vis Res , vol. 43 , 2539-58. 17. 39. Murphy, A. P., H. Ban, and A. E. 47 Welchman. (2013) "Integration of Texture 48 12. Simons, K. (1981) "A Comparison of the 26. Blakemore, C. (1970) "The Range and and Disparity Cues to Surface Slant in Dorsal 49 Frisby, Random-Dot E, Tno, and Randot Scope of Binocular Depth Discrimination in Visual Cortex," J Neurophysiol , vol. 110 , no. 1 Circles Stereotests in Screening and Office Man," J Physiol , vol. 211 599-622. 190-203. 50 Use," Arch Ophthalmol , vol. 99 , no. 3 446-52. 51 27. Dengler, B., and G. Kommerell. (1993) 40. Li, R. W., C. V. Ngo, and D. M. Levi. 52 13. McKee, S. P., and D. M. Levi. (1987) "Stereoscopic Cooperation between the (2015) "Relieving the Attentional Blink in the 53 "Dichoptic Hyperacuity: The Precision of Fovea of One Eye and the Periphery of the Amblyopic Brain with Video Games," Sci Rep , Nonius Alignment," J Opt Soc Am A , vol. 4 , Other Eye at Large Disparities. Implications vol. 5 8483. 54 no. 6 1104-8. for Anomalous Retinal Correspondence in 55 Strabismus," Graefes Arch Clin Exp 41. Scarfe, P., and A. Glennerster. (2015) 56 14. Acerbi, L, D.M. Wolpert, and S. Ophthalmol , vol. 231 , no. 4 199-206. "Using High-Fidelity Virtual Reality to Study Perception in Freely Moving Observers," J 57 Vijayakumar. (2012) Internal Representations of Temporal Statistics and 28. Kaye, M. (1978) "Stereopsis without Vis , vol. 15 , no. 9 3. 58 Feedback Calibrate Motor-Sensory Interval Binocular Correlation," Vision Res , vol. 18 , 59 Timing. PLoS Comput Biol 8(11): e1002771. no. 8 1013-22. 60 mdoi:10.1371/journal.pcbi.1002771 http://mc.manuscriptcentral.com/issue-ptrsb 20 Page 21 of 29 Submitted to Phil. Trans. R. Soc. B - Issue
1 2 3 4 5 6 7 8
9 10 Table 1 11 12 Stereo Age ETDRS Ocular Alignment acuity 13 Type (yrs)/ NDE Refractive Correction VA Treatment history (prims diopters) (arc sec) 14 Gender (logMAR) 15 Distance Near Pre/Post R: -12.00 DS/ -2.00 DCx10 R: 0.2 Ortho Ortho Detected at age 4, A1 19/F R 200/20 16 L: +0.50 DS/-2.75 DC x 5 L: 0.0 no treatment 17 Detected at age 11, R: plano R: 0.1 18 A2 19/M L Ortho 3 XP 200/70 patched at age 11 for ForL: +4.50 DS/ Review-1.00DC x 180 L: 0.16 Only 19 one year R: -5.00DS/ -2.25DCx20 R: 0.02 RXT RXT Detected at age 2, no 20 M1 23/F R 70/20 21 L: -6.50 DS/-2.25 DCx170 L: 0.0 AXT, 12 AXT, 10 treatment Detected in infancy. 22 R: plano R: 0.1 M2 38/F L 12 LXT 12 LXT >400/>400 Two surgeries to L: +1.75 DS/-0.50 DC x 166 L: 0.06 23 correct strabismus 24 R: plano R: 0.1 10 LXT, 2 10 LXT, 2 Detected at age 2, M3 20/F L 200/>400 25 L: +2.25 DS/ -0.75 DC x 165 L: 0.08 L HypoT L HypoT patched for 2 yrs. 26 R: -5.25 DS/ -2.00 DC x 2 R: 0.18 Detected at age 19, M4 30/F R 4 RET 4RET 200/70 27 L: -3.25 DS/ -0.50 DS x 15 L: 0.14 no treatment R: +3.50 DS/ 0.5 DCx10 R: 0.16 28 M5 23/F R 5 RET 5 RET >400/70 Detected at age 13 29 L: +0.75 DS L: 0.1 R: -1.00 DS/-1.00 DCx90 R: 0.04 RXT RXT Detected at age < 10 30 S1 56/F R 200/20 L: -1.00 DS/ -1.25 DC x 85 L: 0.02 AXT, 22 AXT, 24 yrs, no treatment 31 R: +3.00 DS R: 0.12 Detected at age 3, S2 55/F R 3RET 3 RET >400/>400 32 L: +3.75 DS/ +0.25 DC x 26 L: 0.04 patched few weeks 33 R: -2.25 DS/ -0.75 DC x 30 R: 0.24 6 RET, 2 6 RET, 2 Detected at age 2, S3 52/F R 400/400 34 L: -2.50 DS/ -0.75 DC x 170 L: 0.06 LHyper T LHyper T patched for 1 yr R: 4.25 DS R: 0.20 Detected at age 31, 35 S4 47/F R 6 RET 5 RET 200/200 36 L: 4.25 DS L: 0.0 no treatment 37 38 Abbreviations: (1) Type; A, anisometropia; M, both strabismus and anisometropia; S, pure strabismus (2) NDE, non 39 dominant eye (R – right; L – left); (3) Ocular Alignment; Ortho, Orthophoria; XP, exophoria; XT, exotropia; ET, 40 esotropia; AXT, Alternating exotropia; HyperT, hypertropia; HypoT, hypotropia; (4) Stereoacuity; Randot stereoacuity. F, failed (>400 arcsec); Note that treatment history includes any treatment beyond refractive correction with glasses or 41 contact lenses. Age appropriate near correction was used for the various test distances. Units: visual acuity is given in 42 logMAR units. Subjects A1, A2, M1, M4 and M5 were contact lens wearers. 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 http://mc.manuscriptcentral.com/issue-ptrsb 21 Submitted to Phil. Trans. R. Soc. B - Issue Page 22 of 29
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 For Review Only 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 Figure 1. Examples of fixation ‘bug’ stimulus and texture surfaces. A) The fixation bug’s thorax, abdomen 35 and 4 diagonal legs were presented to the dominant eye. B) The head and remaining bug parts were 36 presented to the non-dominant eye. When fused, a complete bug with six legs and a pair of antennae would 37 be perceived. Note that Fig. 1A and 1B included here are for illustrative purposes only and were not 38 designed as a stereo pair. C) Tiled texture background, (D) Random dot background. 39 For tiled backgrounds rendered stereoscopically, the monocular cues to surface slant stem from distortions 40 of the regular grid pattern mapped onto the surface, and the disparity signal to surface slant stems from the 41 differences between right and left eyes’ retinal images. Examples of trial feedback are also depicted here. 42 The dichoptic bug crawls away (C) for an incorrect response, and it explodes in little bits (D) for a correct 43 response. 44 45 107x86mm (300 x 300 DPI) 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 http://mc.manuscriptcentral.com/issue-ptrsb Page 23 of 29 Submitted to Phil. Trans. R. Soc. B - Issue
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 For Review Only 19 20 21 22 23 24 25 26 27 28 29 30 31 32 Figure 2. Stimuli were displayed on an inverted 22-inch Mitsubishi Diamond Pro 2070SB CRT monitor, with 33 screen resolution 1152 x 864 pixels and refresh rate 120 Hz. Observers viewed the reflections of the 34 stereoscopically presented stimuli through a mirror using Crystal Eyes shutter glasses (StereoGraphics 35 Corporation, San Rafael, CA). An opaque black plate was placed beneath the mirror so that subjects could 36 see only the virtual image of the display formed below the mirror. A PUMA 260 robot arm, invisible to the subject, coaligned a circular metal (target) surface with the virtual image at a distance of 63.5cm from the 37 viewer’s eyes. The slant of the target surface was defined as its orientation around the x-axis relative to 38 subject’s line of sight (see dashed line in the figure inset), and a zero degree slant indicated frontoparallel. 39 Monocular and binocular slants differed on cue-conflict trials, but subjects perceived the stimuli as a single 40 slanted disk. Head movements were restricted with head and chin rests. Subjects’ task was to move a 41 cylinder from the starting platform and place it flush onto the real surface to squash the virtual image. The 42 slant of the real surface was determined by the mean of the monocular and binocular slants. The starting 43 platform was 40 cm to the right of the target surface, 20 cm closer to the subject than the target surface, 44 and 16.5 cm above the target surface (all measured from the subject's point of view). 223x167mm (72 x 72 DPI) 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 http://mc.manuscriptcentral.com/issue-ptrsb Submitted to Phil. Trans. R. Soc. B - Issue Page 24 of 29
1 2 3 0.9 4 Stereo-deficient Mean (N = 11) 5 6 7 Data Model 8 9 10 Cue consistent 0.3 11 12 Cue conflict 13 14 15 Dots: Pure stereo 16 17 18 0.8 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 0.7 34 35 36 37 0.2 38 39 40 41 42 43 44 45 46 47 48 0.6 49 50 51 52 53 Hit rate 54 55 56 57 58 59 60 0.5 0.1 Relative stereo weight
0.4
Relative stereo weight 0.3 0.0 For Review Only 0 10 20 30 0 5 10 15 20 25 30 Session Session
0.9 Normal Mean (N = 3)
0.6
0.8 0.5
0.7 0.4
0.6 0.3 Hit rate
0.5 0.2 Relative stereo weight
0.4 0.1
0.3 0.0 0 10 20 30 0 5 10 15 20 25 30 Session Session http://mc.manuscriptcentral.com/issue-ptrsb Page 25 of 29 Submitted to Phil. Trans. R. Soc. B - Issue
1 2 35 3 4 5 6 7 Stereo-deficient Mean (N = 11) 8 9 10 Stereoscopic cue noise 11 12 Monocular cue noise 13 14 15 30 Motor noise 16 17 18 Bias 19 20 21 22 23 24 25 26 27 28 25 29 30 31 32 33 34 35 36 37 38 39 40 41 42 20 43 44 45 For Review Only 46 47 48 49 50 51 52 53 54 55 15 56 57 58 59 60
10 Sensory noise S.D. (degrees) 5
0
0 10 20 30 Session http://mc.manuscriptcentral.com/issue-ptrsb Submitted to Phil. Trans. R. Soc. B - Issue Page 26 of 29
0.8 1 2 0.5 3 4 5 6 7 0.4 8 9 10 11 0.7 12 13 0.3 14 15 16 17 18 0.2 19 20 21 22 23 0.6 24 0.1 25 Relative stereo weight 26 27 28 29 0.0 30 31 32 0 10 20 30 40 33 34 35 0.5 Session 36 37 38 39 40 41 42 43 44 For Review Only 45 46 47 0.4 48 49 50 51 52 53 54 55 Anisometropic 56 57 Post FU 58 0.3 59 A1 60 A2
Strabismic S1 0.2 S2 S3 S4
Mixed 0.1
Post-training & FU Relative Stereo Weight M1 M2 M3 M4 M5 0.0 Controls (N = 3)
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
Pre-traininghttp://mc.manuscriptcentral.com/issue-ptrsb Relative Stereo Weight Page 27 of 29 Submitted to Phil. Trans. R. Soc. B - Issue
3.0 1 2 3 Anisometropic 4 5 6 Post FU 7 8 9 A1 10 11 12 A2 13 14 15 16 17 18 Strabismic 19 20 21 2.5 S1 22 23 24 25 S2 26 27 28 S3 29 30 31 S4 32 33 34 35 36 37 Mixed For Review Only 38 39 40 M1 41 42 43 M2 44 2.0 45 46 M3 47 48 49 M4 50 51 52 M5 53 54 55 56 57 58 59 Controls (N=3) 60 1.5 Post-training & FU contrast ratio (weak/strong)
1.0
1.0 1.5 2.0 2.5
Pre-training http://mc.manuscriptcentral.com/issue-ptrsbcontrast ratio (weak/strong) Submitted to Phil. Trans. R. Soc. B - Issue Page 28 of 29
a. 1000 b. 1 Randot circles 2 Anisometropic 3 Pure Disparity Test 4 Post FU 5 6 A1 7 8 A2 9 10 1000 11 12 Strabismic 13
14 Stereoblind 15 S1 16 17 S2 18 19 S3 20 21 22 S4 23 24 Mixed 25 26 27 28 M1 29 30 M2 31 32 M3 33 34 35 M4 36 100 37 M5 38 39 100 40 41 Controls (N = 3) 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 Post-training & FU stereoacuity (arcsec) Post-training & FU stereoacuity (arcsec) 10 10 For Review Only
10 100 Stereoblind1000 10 100 1000 Pre-training stereoacuity (arcsec) Pre-training stereoacuity (arcsec) c. 10 d. 10
9 9
8 8
7 7
6 6
5 5
4 4
3 3
2 2 Change in stereo acuity (Pre:Post Ratio) Change in stereo acuity (Pre:Post Ratio) 1 Randot circles 1 Pure Disparity Test
0.0 0.1 0.2 0.3 0.4 0.5 0.0 0.1 0.2 0.3 0.4 0.5
Change in Relative Stereo Weight (Post -Pre)http://mc.manuscriptcentral.com/issue-ptrsb Change in Relative Stereo Weight (Post -Pre) Page 29 of 29 Submitted to Phil. Trans. R. Soc. B - Issue
0.3 Anisometropic 1 2 Post FU 3 4 5 A1 6 7 8 A2 9 10 11 12 13 Strabismic 14 15 16 S1 17 18 19 0.2 S2 20 21 S3 22 23 24 S4 25 26 27 28 29 30 Mixed 31 32 M1 For Review Only 33 34 35 M2 36 37 38 M3 39 0.1 40 41 M4 42 43 M5 44 45 46 47 48 49 Controls (N=3) 50 51 52 53 54 55 56 57 58 0.0 59 60 Post-training & FU Visual Acuity (LogMAR)
-0.1
-0.1 0.0 0.1 0.2 0.3
Pre-traininghttp://mc.manuscriptcentral.com/issue-ptrsb Visual Acuity (LogMAR)