Downloaded by guest on September 26, 2021 a amblyopia in Eckmann failure Samuel its and of vision development binocular the explains coding efficient Active www.pnas.org/cgi/doi/10.1073/pnas.1908100117 for themselves signals efficiency. move- sensory coding of eye maximal of statistics via representation the simultaneously, neural optimizing while This ments, the statistics 12). signal optimizing (11, sensory by movements the the achieved eye self-calibration explain vergence been the and to has and pursuit able fields smooth been receptive of lines, visual have these of models Along development (13). coding perception efficient cod- active efficient active (11, to the of coding (2) generalization efficient hypothesis a as ing active understood as be can advanced It 12). been This recently behavior. by and has efficiency representations idea coding neural them optimize their gives adapting to This jointly freedom 1). of (Fig. degrees behavior additional their through sensory inputs organism’s sensory the cortex of visual product in a shapes (7–10). and experience field malleable receptive highly efficiently been that clas- are to extended showing has learn have path- results findings that it These visual sic 6). models early (5, instance, through images the natural explained For encode in be 4). properties can field (3, way receptive explained that (2), been shown hypothesis have of coding modal- sensory sensory statistics ities efficient different in the in this observations to experimental on efficiency numerous representations Based energy neural signals. attune increasing sensory to of advan- is evolutionary means systems substantial One a provide tage. may efficient energy O accommodation coding efficient of development vision. unify- for the binocular a understanding period offers for critical framework model a conceptual our on ing Overall, findings development. with line vision receptive in if binocular plastic, levels still performance are healthy reach fields errors, only refractive for can due correcting model suppressed After the is eye. eye other the one from of state inputs region to amblyopia-like foveal an the the differs, which where in eyes develops two case, the anisometropic of accommo- simulated power and refraction a vergence In coordinated responses. to direction dation leads the deduce which to defocus, cues of disparity model exploits accommodation It the and movements. the vergence eye conditions, maximize accurate healthy perform to to adapted Under self-calibrates efficiency. jointly as are coding movements active encoding overall the eye stimulus of that formulation as proposes a well on which based theory, vision is coding binocular model efficient The active gap. of this development fill to com- the frame- a of theoretical propose we model accepted Here, putational widely development. joint no this processes still describing learning work is two there the that coupled, suspected movements. are neu- been eye long appropriate an accurate has is of of it While learning life learning the of the and months representations 2019) comprises 13, ral first May that review the for process during (received 2020 active vision 29, January of approved development and CA, The Berkeley, California, of University Banks, Kong S. Hong Bay, Martin Water by Clear Edited Technology, and Science of University Kong Hong Engineering, Computer Germany; Main, am Frankfurt 60629 rnfr nttt o dacdSuis 03 rnfr mMi,Germany; Main, am Frankfurt 60438 Studies, Advanced for Institute Frankfurt motnl,hwvr nml a hp h ttsiso their of statistics the shape can animals however, Importantly, rban r epnil o 0 foreeg con- energy be our to of circuits 20% neural organizing for Therefore, responsible (1). are sumption brains ur | cieperception active a,b,c,1 ua Klimmasch Lukas , | c amblyopia a lnkIsiuefrBanRsac,648Fakuta an emn;and Germany; Main, am Frankfurt 60438 Research, Brain for Institute Planck Max | vergence a,b eta .Shi E. Bertram , | d n ohnTriesch Jochen and , b doi:10.1073/pnas.1908100117/-/DCSupplemental at online information supporting contains article This under 1 ModelDB in available is model no. the accession of code MATLAB Documented deposition: Data oeiaie ies . C BY-NC-ND) (CC 4.0 NoDerivatives License distributed under is article access open This Submission.y Direct PNAS a is article This paper.y performed the interest.y wrote L.K. competing J.T. no and and declare B.E.S., authors S.E. S.E., The and research; data; designed analyzed J.T. L.K. and and S.E. research; B.E.S., S.E., contributions: Author including vision, con- binocular a of through development approach the this of model demonstrate crete We problem. In 15). this (14, problem” room H “dark the maximizing lit- formulation, as processing our known predictive is and this energy free erature, the In altogether. eyes minimize can quantity since this response, that reducing is simply by formulation minimized prior be encoding). this cortical of the in limitation represented The not (i.e., lost is that mation entropy representation. cortical the given entropy conditional its is where iie eorecntan.Temta nomto a be can information mutual The as constraint. decomposed resource limited a activity informa- cell mutual ganglion Shannon the by tion measured as efficiency coding eateto optrSineadMteais oteUniversity, Goethe Mathematics, and Science Computer of Department owo orsodnemyb drse.Eal csmotokcmo triesch@ or [email protected] Email: addressed. be fias.uni-frankfurt.de. may correspondence whom To mloi-iesaeadsget odtosfrsuccessful an for develops conditions model suggests the the treatment. and In however, state of errors, conditions. amblyopia-like development by refractive healthy of the under theory of case explains vision development this binocular model the of accurate The efficient of plausibility vision. model active the binocular computational called test a theory we proposing repre- it a Here, neural how behavior, coding. optimizing and and jointly behavior sentations requires opti- organism’s Therefore, performance environment. the mal statis- the from by these information of shaped however, samples statistics Importantly, are the signals. tics to representations input achieve neural sensory systems adapting sensory effi- by that The this states manner. hypothesis energy-efficient coding an cient in operate must Brains Significance (R ro omltosfcsdo iiiigteconditional the minimizing on focused formulations Prior norfruaino cieefiin oig emaximize we coding, efficient active of formulation our In I ) (R H n minimizing and H , (R C 261483.y H (R ) ) (R steetoyo h eia epneand response retinal the of entropy the is ewe h eia tmlsrpeetdb retinal by represented stimulus retinal the between | C | y H C ) I (R ) ny(6). only (R a,b,1 y o xml,dfcsn rcoigthe closing or defocusing example, for by, R , | C C H n t otclrepresentation cortical its and = ) ) (R H ≤ H . H y | I C (R (R (R (R raieCmosAttribution-NonCommercial- Commons Creative ) H ) hs natv gn could agent active an Thus, ). | , d iutnosy hsavoiding thus simultaneously, C H − C eateto lcrncand Electronic of Department (R . y ) ) h nrp fteretinal the of entropy the ), samaueo h infor- the of measure a is https://www.pnas.org/lookup/suppl/ sahee ymaximizing by achieved is (R NSLts Articles Latest PNAS | C ), H C (R | under f7 of 1 | [1] C )

BIOPHYSICS AND NEUROSCIENCE COMPUTATIONAL BIOLOGY the algorithms is not important for the model to function. In fact, different cortical coding and reinforcement learning models have been successfully applied in previous active efficient coding models (24, 25). Our model is presented with a textured planar object. The object is sampled by the two eyes for 10 iterations, which con- stitute one fixation. After each fixation, a new object is presented at a new random distance. The retinal images are rendered based on the positions of the accommodation, vergence, and object planes (Fig. 3). The inputs are whitened, contrast adjusted by an interocular suppression mechanism, and then, binocularly Fig. 1. The action–perception loop in active efficient coding. The sen- encoded by a population of cortical neurons. The reinforcement sory input is obtained by sampling input signals from the environment learning modules control the retinal input of the next itera- (e.g., via eye movements). A percept is formed by neural encoding, which tion by shifting accommodation and vergence planes along the drives the selection of actions and thereby, shapes the sampling pro- egocentric axis (Fig. 3). cess. Therefore, perception depends on both neural encoding and active input sampling. Classic efficient coding theories do not consider the active sampling component (orange). Cortical Encoding. In our model, the cortical population activity represents the binocular “percept” based on which behavioral commands are generated (compare Fig. 1, Upper). The corti- the simultaneous calibration of vergence and accommodation cal encoding comprises two efficient coders: one for fine details control. in the foveal region and one for the periphery that receives a Indeed, newborns have difficulties bringing objects into focus low pass-filtered input. Both are implemented using the standard and cannot yet verge their eyes properly (16). How infants man- matching pursuit algorithm (26) (Materials and Methods). age to self-calibrate their control mechanisms while interacting To find a set of neurons that best encodes the input, instead with their visual environment is currently unknown. Additionally, of minimizing the conditional entropy directly, we minimize an in certain medical conditions, the calibration of vergence and 2 upper bound (i.e., the average of the encoding error kSk ) (27): accommodation control is impaired. For example, anisometropia describes a difference in the refractive error between the eyes. If h ˆ 2i  2 not corrected early during development, this can evoke ambly- H(R | C ) ≤ E kR − R (C )k ≡ E kSk , [2] opia: a disorder of the developing visual system that is charac- terized by an interocular difference in visual acuity that is not where Rˆ is an estimate of the input R based on the activities cj of immediately resolved by refractive correction. Amblyopia can be cortical neurons with receptive fields bj (Materials and Methods associated with a loss of stereopsis and in severe cases, leads to has details): X monocular blindness (17). Furthermore, vergence and accom- Rˆ = c b . [3] modation eye movements are either less accurate or completely j j j absent (18, 19). Although there have been recent advances in the treatment of In every iteration, both activities and receptive fields adjust 2 amblyopia (20, 21), existing treatment methods do not lead to online to minimize the encoding error kSk (Materials and Meth- satisfactory outcomes in all patients. This is aggravated by the ods). Thus, the receptive fields reflect the stimulus statistics (28) fact that treatment success strongly depends on the stage of neu- and resemble those of simple cells in the visual cortex (6) (SI ral circuit maturation (20). When young patients are still in a Appendix, Fig. S1). critical period of visual cortex plasticity (10), they often recover after refractive errors are corrected, while adults mostly remain Vergence Learning. The vergence reinforcement learner also aims impeded (22, 23). to minimize the conditional entropy H(R | C ) (i.e., the encod- The above findings are all readily explained by our model. ing error kSk2). Therefore, vergence movements are favored Under healthy conditions, our model develops accurate ver- gence and accommodation eye movements. When the model is impaired due to strong monocular hyperopia, we observe that an amblyopia-like state develops. We show that this is due to the abnormal development of binocular receptive fields in the model and demonstrate that healthy binocular vision is regained as the receptive fields readapt after refraction correction. How- ever, if the sensory encoding is no longer plastic and does not adapt to the changes in the visual input statistics, suppression prevails. Overall, our model suggests that coding efficiency may provide a unifying explanation for the development of binocular vision. Model Formulation The active efficient coding model that we propose has a mod- ular structure (Fig. 2). A cortical coding module models the Fig. 2. Model architecture, with solid arrows representing the flow of learning of an efficient representation C of the binocular reti- sensory information and dashed arrows representing the flow of control nal representation R by minimizing the conditional entropy commands. Sampled input images with given defocus blur and disparity are H(R | C ) whitened at the retinal stage R and contrast adjusted through an interocular . At the same time, an accommodation reinforcement suppression mechanism based on the recent history of cortical activity (Left). learning module maximizes H(R), and a vergence reinforce- Thereafter, they are encoded by a set of binocular neurons that represents ment learning module minimizes H(R | C ) (Eq. 1). All three the cortical encoding C (Center). The cortical population activity serves as modules are plastic and adjust simultaneously in response to input to two reinforcement learning modules (Right) that control vergence changes in the sensory input statistics. The exact choice of and accommodation commands. Details are in Materials and Methods.

2 of 7 | www.pnas.org/cgi/doi/10.1073/pnas.1908100117 Eckmann et al. Downloaded by guest on September 26, 2021 Downloaded by guest on September 26, 2021 ihsaldsaiisadms ern r ue osmall H to tuned Learning. Accommodation are input neurons favors most always and model dis- disparities. disparities the small until say, small even for, magnified with disparity bias be initial this an can of dis- Thus, selective parities encoding 4B). this become the (Fig. by to make efficient dominated neurons more and more is disparity even this learner that cause for input vergence self- will visual the a This dis- produce well, parity. certain to to particularly a leads try of encoded This will inputs be If fields. can 4A). receptive parity (Fig. encoded of cycle accurately feedback set most reinforcing be current can the that with input visual produce that actual to compared increased are better blur For values. defocus configurations. and position shifts in plane disparity different visibility, as for scheme images Same input retinal conditions. different the vergence and under and planes (C planes respective a.u. Accommodation the be in example. between cannot position distance this it object the Note 0, in as eyes. position measured eye (green) example, are right errors for right the at, and placed by blue) is focused (light stimulus left the the when possible that, for accommoda- (i.e., positions reachable range plane indicate presented fixation axes tion are the Horizontal objects indicates positions). where plane also, vergence range and the simulation of the indicates Abstraction (acc.) during bar (B) accommodation horizontal positions. (r.) gray plane right different The and as (l.) represented are left distance and distance, (verg.) vergence 3. Fig. cmn tal. et a Eckmann to leads This Methods). high and at (Materials experimentally (33–35) observed levels contrast as and is monocular experience energy for input visual levels in total activity binocular neurons similar the ensure time, tuned to same balanced the similarly kept At 32). agree- of (31, input in excitation cortex eye is visual reciprocal (right) This with left iterations. the subsequent ment If of in contrast encoding. suppressed dur- the becomes input recruited are encoding, the fields cortical of receptive ing balance monocular (left) ocular interocular right mod- the basic mostly contrast on dynamic a based describe use to ulation 5A) We (Fig. amblyopia. model suppression in mechanism central Model. Suppression input focused favor to model the 4C). for (Fig. matter not distri- receptive by does exact shape suggested the the field as However, of 30). input 29, retinal independent (9, the experiments thus, deprivation in and frequencies spatial static of be bution retinal of to tuning cells frequency spatial representation ganglion the assume retinal We Methods). the and of activity this, squared For module. learning ment B A (R ) (Eq. bet(b. position, (obj.) Object (A) environment. the from sampling Input smxmzdvateacmoainreinforce- accommodation the via maximized is 1) neoua upeso stogtt ea be to thought is suppression Interocular oiinrneo comdto n vergence and accommodation of range Position ) h nrp ftertnlresponse retinal the of entropy The H D C sapoiae ythe by approximated is (R) kR xmlsof Examples (D) B. k 2 (Materials A. , Appendix (SI cues disparity learns from model defocus the of Instead, sign 37). the (36, infer direction steer higher-order right to to help other the could nor in which model, focus chromatic our in neither front provided in are Also, or aberrations behind object. focuses the eye an defocus of whether environment, of visual independent S5 simplified is and our blur Figs. in population cues: Appendix , sign obvious the (SI across increase poorly entropy responses 4 S6 respond response (Fig. and of and presented the range is thus, contrasts input the sharp focus shape, input, of for center-surround gan- out their retinal selective when of With are nature fields. edge-enhancing they receptive the cell to due glion is This well. 4A (Fig. bootstraps S4). coding Fig. This efficient accurately Appendix, active SI information. of more unbi- loop less feedback input equally positive contains an zero-disparity the disparities input even all encode such of to Thus, because inputs tend overlap. encode will can maximally well that eyes encoder two ased response eye the right retinal and the onto left in the contained of information H overlap the the dispar- fields, most to still input visual Due is uniform 4B). input model. (Fig. zero-disparity a of the efficient to encoding of introduc- the adapted initialization distribution, artificially are during ity to fields disparity due receptive zero not for When is Appendix, bias This fields a (SI receptive S1). ing eyes binocular Fig. develop the Appendix, neurons by (SI most tracked and S3A), continuously Fig. is object The aeil n Methods and fil- (Materials smallest simulation The the input. for whitening focused used smaller was for details). However, blue) preference filter. inde- (dark stronger whitening reward ter a the accommodation of induce highest size filters the the of yields pendent input smaller Zero-blur for (i.e., (abs.) filters. when frequently the absolute pronounced more (i.e., more encountered zero σ reward been even have is at vergence disparities advantage (avg.) seen small This average encoding). stimuli efficient under highest case, most trained the model each produce indicates indi- In disparity blue purple conditions. dark and and healthy pink distributed, with unbiased/uniform, disparities Laplacian input indicates cate of Gray fields distributions receptive SDs. different The to color-coded distributions textures). adapted disparity neurons 300 300 different over Normal- of (averaged for neu- (B) populations reward and neural coding). learning), (verg.) and (neural stimulus, (statistical vergence efficiency overrepresentation (norm.) the encoding this ized to increases reflect exposed further the to which frequently Therefore, adapt (acting). more circuits stimuli ral is other system over efficiently An preferred sensory coding. is efficient active stimulus of encoded loop feedback Positive (A) dencies. 4. Fig. accommo- 6 and vergence (Fig. precise movements perform eye Binoc- to dation Active learns of model Self-Calibration the to Vision. Leads ular Coding Efficient Active Results inputs eye 5B). right (Fig. and dissimilar left are when cycle suppression self-reinforcing ABC .(C ). (R utemr,acrt comdto sahee without achieved is accommodation accurate Furthermore, as accurate highly becomes performance Accommodation ) omlzdacmoain(c. eadfrdfeetwhitening different for reward (acc.) accommodation Normalized ) ssals o eodsaiywe h mgsprojected images the when disparity zero for smallest is ). h edaklo fatv fcetcdn n eaddepen- reward and coding efficient active of loop feedback The ntehatycniinwtotrfatv errors, refractive without condition healthy the In A and i.S2A). Fig. Appendix, SI NSLts Articles Latest PNAS .Frsharper For C). | f7 of 3 and has

BIOPHYSICS AND NEUROSCIENCE COMPUTATIONAL BIOLOGY AB Early but Not Late Refractive Correction Rescues Binocular Vision. To test if the anisometropic model can recover from amblyopia upon correction of the refractive error, we first trained a fully plastic model under anisometropic conditions until it had con- verged to the amblyopic state. Then, all refractive errors were corrected. When the receptive fields were fixed after the refrac- tive error was corrected, receptive fields remained monocular, and the model did not recover from the amblyopic state. Instead, it maintained a high level of vergence error (Fig. 6B). In con- trast, when receptive fields remained plastic and could adapt Fig. 5. Interocular suppression model. (A) When mostly right (left) monoc- to the changed input statistics, the vergence error decreased ular neurons cj are activated to encode an input image patch, the right (left) (Fig. 6B), and the strong suppression of the formerly impaired contrast unit yr (yl) is excited, and the left (right) retinal image is suppressed eye was restored to lower values (SI Appendix, Fig. S9B). This in subsequent iterations. Color hue indicates response selectivity for left eye was due to a shift from monocular to binocular receptive fields (blue) or right eye (green). Dashed (solid) lines indicate inhibitory (excit- C atory) interactions. Connection strength is represented by line thickness. We as a result of the changed input statistics (Fig. 6 ). This is in model interocular suppression as being scale specific (i.e., when the high- line with a large body of evidence suggesting that limited cor- resolution foveal region of the left eye is suppressed, the low-resolution tical plasticity in adults prevents recovery from amblyopia after periphery of the left eye may still provide unattenuated input) (Materials the correction of refractive errors (10, 20, 21). Furthermore, it and Methods). (B) Feedback cycle of the suppression model. Disparate inputs predicts that therapies reinstating visual cortex plasticity should to both eyes lead to preferential recruitment of monocular neurons, which be effective. results in interocular suppression inducing competition between the eyes. This impedes precise vergence eye movements and exacerbates disparate Discussion input (purple; left cycle). On a slower timescale, receptive fields (RFs) adapt We have shown how simultaneously optimizing both behavior to suppression by becoming more monocular, which makes future suppres- sion more likely (red; right cycle). Dashed lines indicate feedback that affects and encoding for efficiency leads to the self-calibration of active future input processing. binocular vision. Specifically, our model, which is based on the active efficient coding theory, accounts for the simultaneous development of vergence and accommodation. Fig. S7). We further examined this entanglement under abnor- Previous computational models have focused on either the mal input conditions (e.g., when simulated lenses were placed in development of disparity tuning or the development of vergence front of the eyes of an agent trained under healthy conditions). and accommodation control but have failed to capture their rich We find the responses of the model to qualitatively agree with interdependence (28, 44–46). For example, a model by Hunt experimental results (38, 39) (SI Appendix, Fig. S8). et al. (28) explained how disparity tuning may emerge through sparse coding and how alternate rearing conditions could give Anisometropia Drives Model into Amblyopic State. To test how the rise to systematic differences in receptive field properties, but model evolves under abnormal rearing conditions, we simulated their model completely neglected vergence and accommodation an anisometropic case by adding a simulated lens in front of the behavior. Conversely, others have presupposed populations of right eye such that it became hyperopic and was unable to focus cells readily providing error signals for vergence and accommo- objects at close distances (Fig. 3C, Middle). Therefore, unlike dation control without explaining their developmental origin (44, the healthy case, where neither eye is favored over the other, 46). Therefore, previous models have failed to explain how the in the anisometropic case, the impaired eye receives system- visual system solves the fundamental “chicken and egg” of dis- atically more defocused input. Cortical receptive fields reflect parity tuning and eye movement control: the development of fine this imbalance and become more monocular, favoring the unim- disparity detectors requires the ability to accurately focus and paired eye (compare Fig. 6C, Lower and SI Appendix, Fig. S1, align the eyes, which in turn, relies on the ability to detect fine Upper Center). The combined effect of imbalanced input and disparities. Our active efficient coding model solves this prob- adapting receptive fields results in a vicious cycle that drives the lem through the positive feedback loop between disparity tuning, model into an amblyopia-like state (Fig. 5B). Foveal input from which facilitates the control of eye movements, and improved the hyperopic eye becomes actively suppressed (SI Appendix, accommodation and vergence behavior, which enhances the Fig. S9A), while the low-resolution peripheral input is unaf- fected and still provides binocular information such that a coarse control of vergence is maintained (Fig. 6A and SI Appendix, Figs. S2B and S3B). This results in stable binocular recep- ABC tive fields in the periphery (SI Appendix, Fig. S1, Lower Cen- ter), which provide enough information for coarse stereopsis as observed in experiments (40–42). Accommodation adapts such that the stimulus is continuously tracked with the unimpaired eye (SI Appendix, Fig. S3). When both eyes were similarly impaired but with opposite sign of the refractive error (Fig. 3C, Bottom), receptive fields still Fig. 6. Model performance. (A) Average (avg.) absolute (abs.) vergence become more monocular, but no eye is preferred (SI Appendix, (verg.) and accommodation (acc.) errors of the left (l.) and right (r.) eye after Fig. S1, Upper Right). As a result, the relatively more myopic eye training under healthy and anisometropic conditions. The dashed line indi- is used for near vision, the relatively less myopic eye is used for cates the expected average vergence error when accommodation planes are distant vision (SI Appendix, Fig. S10A), and the respective other moved randomly under healthy conditions. (B) Vergence performance of the defocused eye is suppressed. At intermediate ranges, the stim- formerly anisometropic model after correction of all refractive errors at iter- ation 5 × 106 (vertical gray line). The (dotted) solid line indicates the model ulus history determines which eye gets recruited (SI Appendix, with (non-)plastic receptive fields (RFs). The initial increase in the vergence Fig. S10 B and C). This configuration is similar to monovi- error is due to the recalibration of the reinforcement learning module. (C) sion, which results from a treatment method for presbyopia, Histogram of foveal RFs binocularity as measured by the right monocu-

where the ametropic condition is achieved via optical lenses or lar (monoc.) dominance d(·,r) before and after refractive error correction surgery (43). (Materials and Methods has details).

4 of 7 | www.pnas.org/cgi/doi/10.1073/pnas.1908100117 Eckmann et al. Downloaded by guest on September 26, 2021 Downloaded by guest on September 26, 2021 h mnae aeoyfo h cilDtbs 5) n mg was image One 3 (54). (Fig. position Database random McGill a the at presented from category “manmade” the Rendering. Image Input to Methods easy and input Materials sensory the make to predict. or and/or altogether eyes encode the eyes defocus the to accommodation closing using prob- deliberately the of avoids sensory and lem cortical environment of the the amount its from the sampled maximize and information maximizing input to implies sensory This learned effi- the representation. are active between of commands information formulation generated mutual motor our are a coding, In commands in predictions. cient behavior motor sensory of fulfill There, to generation fashion. the infer- general consider active very does Friston’s not, approach behavior. does ence optimizing coding efficient consider Classic theories classic however, coding (51–53). in predictive examples which rooted special of are is (2–6), ideas coding coding and efficient efficient species Active could across systems and modalities. perception general sensory active very many is to our applied coding be control, efficient vergence active of and formulation accommodation including vision, lar in therapies of improved role to the lead of future. could the understanding amblyopia standard better in the A suppression with (50). (e.g., method) treatment patching and suppression disease how the is efficient during interest from current active develops of conclusions the topic A within the the framework. suppression coding understanding change but on interocular focus 49), of to should principles (48, work them future Future expect model. the suppres- present not in sophisticated incorporated do More be an we could fovea. of models in the implementa- development sion fields of the This receptive representation monocular for 35). the mostly account (34, with to state, field amblyopia-like suppression sufficient receptive incon- proved is own model’s that tion its the input suppresses to behind neuron gruent every logic that is simple mechanism The pression. for the period in critical fields a of receptive (10). development findings this if binocular matching from possible plastic, Recovery remain only stereopsis. model is fine unaf- state of monocular is with loss amblyopia-like eye state and other amblyopia-like fields the an receptive while into error develops refractive it fected, a from one suffers where conditions, eye however, eye anisometropia, accommodation healthy simulated and a Under For vergence movements. (47). accurate produced develops behavior statistics model own image the the system’s reflect the neurons by sensory of erties representation cmn tal. et Eckmann reti- below). (see whitened mechanism The suppression S13). interocular Fig. the Appendix, via adjusted (SI contrast four of size patches factor of nal patches a Appendix binocular by 81 (SI pyramid into merged (6) and Field cut 2 were and images Olshausen scale, each by described as Processing. ( Input image original the of center the S13). 72 Fig. from size For cropped of plane. images object were retinal the pixels two to scale, distances (peripheral) as dis- foveal (a.u.) the on units 1-a.u. arbitrary linearly The in Appendix, depended (SI measured planes. SDs applied SD were accommodation of was the px and filter where 0.8 equals blur image, object tance resulted Gaussian eye between This A right (right). distance unit. the left the distance and the per left to px the 2 (pixel) to of px disparity 1 a shifted in was image eye (right) randomly were fixation. position next and the image for next selected the before iterations) subsequent ×8 hl ehv oue ntedvlpeto ciebinocu- active of development the on focused have we While sup- interocular is amblyopia in mechanism important An o vr itneui ewe egneadojc ln,teleft the plane, object and vergence between unit distance every For x hr h eihrlsaewsdw-ape ihaGaussian a with down-sampled was scale peripheral the where px, 8 eenraie ozr enitniyadsubsequently and intensity mean zero to normalized were R* h etadrgtrtnliptiae eewhitened were images input retinal right and left The ffiedsaiis nteed h uigprop- tuning the end, the In disparities. fine of eue 0 rycl-ovre aua mgsof images natural grayscale-converted 300 used We A IText SI and uigoefiain(.. 10 (i.e., fixation one during B) .Errors S12). and S11 Figs. and a eal) For details). has × 2(160 72 IAppendix, SI × 160) ecn na prxmto ftetmoa ifrneerror 56): difference ref. temporal in the 3 of approximation an on descent The are vectors nor- properties vector state were the as state scale estimated they to foveal combined After action similar and 58). the next is peripheral (57, polar- the into This cortex the norm, squaring. concatenated visual on unit the primary depend to to in not malized cells due does complex Therefore, input and of scale. patches the per over of averaging patches ity to of due number ant the is 81 responses squared neurons: patch-averaged cortical the the of comprises vector information state positions: plane respective accommo- the actions the Possible in control plane. vergence to the architectures and identical planes dation with 56) (55, learners ment Learning. Reinforcement and norm. distribution unit (SI Gaussian field to encoding zero-mean receptive normalized a for all subsequently the from simulation, used randomly For each drawn was of fields. were neurons beginning receptive weights the 300 neural At of S13). 300 Fig. set same Appendix, separate the a of scale, subset peripheral a by encoded was where product field scalar receptive the with a measured between similarity the where response cortical The image. the encode efficiently (26). algorithm where S1): Fig. Appendix, (SI statistics input the on descent maximized). is residual efficiency the coding with fields receptive c their of Eq. similarity to according the on based selected the after residual generalized where subtracted is neuron first the input by original encoded the already from was that information all step, encoding selected: first was that the input neurons In retinal zero. of remained activities encoding the n for where selected process, not iterative were an via determined patches contrast-adjusted The r ae ntesurdrsos fe htnn n otclencoding. cortical and whitening after response squared the on based are Information. Mutual Approximating network all simulation, each randomly. of initialized beginning were the weights At factor. discounting poral where i k = fe noig l eetv ed eeudtdtruhgradient through updated were fields receptive all encoding, After o ahpth erecruited we patch, each For nec noigse,tercntuto error reconstruction the step, encoding each in ,tenuo ihtercpiefield receptive the with neuron the 1, weights r S c 2η t 1 (i steacmoaino egnerwr and reward vergence or accommodation the is ,p) stersda mg fe h rtecdn tpand step encoding first the after image residual the is = kS is 5 b V a π N u j × (f h ciiyo neuron of activity the a k ← aj scoe ihprobability with chosen is 2 = ygedl eetn h ernwt aiu response maximum with neuron the selecting greedily By 4. ): 10 n omlzdt ntlnt.Tu,tertnn reflects tuning their Thus, length. unit to normalized and and kb i −5 P n b exp(z j j = j + ∆b + S exp(z salann ae ahptho h oelscale foveal the of patch Each rate. learning a is v argmax 1 δ R: j ∆b eue w eaaentrlatrcii reinforce- critic actor natural separate two used We t = = r pae i prxmt aua gradient natural approximate via updated are a j ) R−c j j j k r S ) t n f , , + i = hecdn tp usqetnuosare neurons Subsequent step. encoding nth z R i = 1 a |hb γ ∆b b R eeecddwt h acigpursuit matching the with encoded were a = V n N i hb 1 1 − {−2, ∈ j p (f ead o h enocmn learners reinforcement the for Rewards , , X = j j S X = X k (t j , n−1 =1 0o 0 otclnuost most to neurons cortical 300 of 10 n hnslcigtenx neuron, next the selecting When Ri. ) p i c ) −η u i − 1 fe noigpatch encoding after aj c c i| = i (i 2 k f ,0 ,2} 1, 0, −1, V ,p) ∂
j b b , ∂ hb (f , π b j i , k j a (t htwsms iia othe to similar most was that i , kS (f 1 V −1) NSLts Articles Latest PNAS , b ,adtesaevleis value state the and ), (f S N j S 0 ), k ) f 0 n eia input retinal and kS i, = 2 ≡ = fsz 300 size of N C X a cmaeFg ) The 3). Fig. (compare f R, k i 2η j = orsodt shifts to correspond 2 γ ssailyinvari- spatially is smnmzd(i.e., minimized is v c (c = j j S f 1 j N , stetem- the is 0.6 . . . . , δ p (algorithm , and × c S 300 | n .The 2. R sthe is was ) 5 n was [10] f7 of p [4] [9] [7] [5] [8] [6] =

BIOPHYSICS AND NEUROSCIENCE COMPUTATIONAL BIOLOGY Together, this can be understood as an empirical estimate of the mutual ˆr(t + 1) = (1 − α)ˆr(t) + αr(t), [16] information between the whitened response R and cortical response C: σˆ(t + 1)2 = (1 − α)σˆ(t)2 + α [ˆr(t) − r(t)]2, [17] I(R, C) = H(R) − H(R | C), [11] where α = 0.001 is an update rate that sets the decay of the exponential where the conditional entropy H(R | C) is upper bounded by the reconstruc- weighting (60). tion error kSk2 (Eq. 2 and SI Appendix). Due to the “energy conservation” property of the matching pursuit algorithm (26), the energy of the residual Suppression Mechanism. There are two separate suppression modules, one image is equal to the energy of the retinal representation minus the energy per scale, that adjust the contrast of left and right input images (Fig. 5). of the cortical representation (SI Appendix): that is, We introduce a contrast measure xk, k ∈ {l, r}that gives an estimate of the amount of left (right) monocular input over the previous iterations: 2  2 2 2 2 − kSk = − kRk − kCk = kCk − kRk . [12] * ! + X fi xk(t) = P d(i,k) . [18] j fj Therefore, we take the difference between cortical and retinal response i t,τ energy as the reward for the vergence learner.

For the accommodation learner, we maximize the entropy of the The monocular dominance of each neuron d(i,k) is weighted with its P whitened retinal response H(R). We take each entry of R as an indepen- relative patch-averaged squared activation fi/ j fj. Here, h·it,τ is the expo- dent sample of the same underlying random variable and estimate the nential moving average over time with decay constant τ = 10 (SI Appendix). entropy of its probability distribution. The distribution is well approximated The monocular dominance is defined as by a Laplace distribution, independent of the level of blur in the input (SI Appendix, Fig. S5). Therefore, we approximate H(R) with the entropy of a kb(i,k)k d(i,k) = , k ∈ {l, r}, [19] Laplace distribution with the same SD σR: kb(i,k)k + kb(i,¯k)k  q  2 where b(i,k) is the left/right monocular subfield of neuron i. The contrast H(R) ≈ ln e 2σR . [13] estimate xk of the left and right subfields of the input image is separately processed by two contrast units: Since the expected squared activity of the retinal representation E(kRk2) is equal to the variance σ2, it is also a monotonic function of the entropy m R y(xk) = [xk − θ]+. [20] H(R). More generally, since the retinal response has bounded support and its 1 − θ

probability distribution is unimodal and Lipschitz continuous, the variance As the contrast estimate xk crosses the threshold θ, the output y(xk) increases 2 σR is a monotonic function of a lower bound of the entropy (59). There- from zero until it saturates at m. We chose the threshold θ = 0.6 just above 2 2 fore, we use kRk as an empirical estimate of E(kRk ) for the reward of the perfect binocular input at xk = 0.5 to provide some margin before the self- accommodation reinforcement learning module. reinforcing feedback loop becomes active (Fig. 5B). Furthermore, we set As one would expect for the entropy H(R), kRk2 also decreases for the saturation m = 0.8 to prevent total suppression of one eye. Finally, the increasing input blur. Under the assumption of a flat frequency spectrum subsequent input subpatches for the cortical coder are adjusted to after whitening, one finds (SI Appendix) ∗ Rk(t) = Rk (t) (1 + y(xk(t − 1)) − y(x¯k(t − 1))). [21] kRk2 ∝ 1/σ, [14] Note that I(R, C) = I(R*, C) since R is homeomorphic to R* (61). Therefore, in where σ is the SD of the Gaussian blur filter that is applied before whitening our theoretical framework, we do not distinguish between the contrast- to simulate defocus blur. adjusted and raw retinal response. For the model implementation, the contrast-adjusted retinal response R is used. SI Appendix has additional Reward Normalization. Before being passed to the reinforcement learning detail. agents, the accommodation and the vergence rewards were normalized online to zero mean and unit variance: that is, Software and Documentation. Documented MATLAB code of the model is available in ModelDB under accession no. 261483 (62). r(t) −ˆr(t) r(t) ← , [15] σˆ(t) ACKNOWLEDGMENTS. This work was supported by German Federal Min- istry of Education and Research Grants 01GQ1414 and 01EW1603A, Hong where ˆr is the exponentially weighted running average of the reward r and Kong Research Grants Council Grant 618713, and the Quandt Foundation. σˆ2 is an online estimate of its variance We thank Maria Fronius and Rowan Candy for numerous discussions.

1. D. D. Clark, L. Sokoloff, “Circulation and energy metabolism of the brain” in Basic 12. C. Teuliere` et al., Self-calibrating smooth pursuit through active efficient coding. Neurochemistry: Molecular, Cellular and Medical Aspects, G. J. Siegel, B. W. Agranoff, Robot. Autonom. Syst. 71, 3–12 (2015). R. W. Albers, S. K. Fisher, M. D. Uhler, Eds. (Lippincott-Raven, Philadelphia, PA, 1999), 13. J. J. Gibson, The Senses Considered as Perceptual Systems (Houghton Mifflin, Boston, pp. 637–670. MA, 1966). 2. H. B. Barlow et al., Possible principles underlying the transformation of sensory 14. K. Friston, C. Thornton, A. Clark, Free-energy minimization and the dark-room messages. Sens. Commun. 1, 217–234 (1961). problem. Front. Psychol. 3, 130 (2012). 3. E. P. Simoncelli, B. A. Olshausen, Natural image statistics and neural representation. 15. A. Sims, “The problems with prediction” in Philosophy and Predictive Processing, Annu. Rev. Neurosci. 24, 1193–1216 (2001). T. K. Metzinger, W. Wiese, Eds. (MIND Group, Frankfurt am Main, Germany, 2017), 4. M. S. Lewicki, Efficient coding of natural sounds. Nat. Neurosci. 5, 356–363 (2002). chap. 23. 5. J. J. Atick, A. N. Redlich, What does the retina know about natural scenes? Neural 16. G. M. Tondel, T. R. Candy, Accommodation and vergence latencies in human infants. Comput. 4, 196–210 (1992). Vis. Res. 48, 564–576 (2008). 6. B. A. Olshausen, D. J. Field, Sparse coding with an overcomplete basis set: A strategy 17. C. Blakemore, R. C. Van Sluyters, Experimental analysis of amblyopia and strabismus. employed by v1? Vis. Res. 37, 3311–3325 (1997). Br. J. Ophthalmol. 58, 176–182 (1974). 7. T. N. Wiesel, D. H. Hubel, Single-cell responses in striate cortex of kittens deprived of 18. R. V. Kenyon, K. J. Ciuffreda, L. Stark, Dynamic vergence eye movements in strabis- vision in one eye. J. Neurophysiol. 26, 1003–1017 (1963). mus and amblyopia: Symmetric vergence. Invest. Ophthalmol. Visual Sci. 19, 60–74 8. H. V. B. Hirsch, D. N. Spinelli, Visual experience modifies distribution of hori- (1980). zontally and vertically oriented receptive fields in cats. Science 168, 869–871 19. V. Manh, A. M. Chen, K. Tarczy-Hornoch, S. A. Cotter, T. R. Candy, Accommodative (1970). performance of children with unilateral amblyopia. Invest. Ophthalmol. Visual Sci. 9. J. A. Movshon, R. C. Van Sluyters, Visual neural development. Annu. Rev. Psychol. 32, 56, 1193–1207 (2015). 477–522 (1981). 20. D. M. Levi, D. C. Knill, D. Bavelier, Stereopsis and amblyopia: A mini-review. Vis. Res. 10. J. Sebastian Espinosa, M. P. Stryker, Development and plasticity of the primary visual 114, 17–30 (2015). cortex. Neuron 75, 230–249 (2012). 21. M. P. Stryker, S. Loewel, Amblyopia: New molecular/pharmacological and environ- 11. Y. Zhao, C. A. Rothkopf, J. Triesch, B. E. Shi, “A unified model of the joint devel- mental approaches. Vis. Neurosci. 35, E018 (2018). opment of disparity selectivity and vergence control” in 2012 IEEE International 22. S. A. Cotter et al.; Pediatric Eye Disease Investigator Group, Treatment of an- Conference on Development and Learning and Epigenetic Robotics (ICDL) (IEEE, isometropic amblyopia in children with refractive correction. Ophthalmology 113, Piscataway, NJ, 2012), pp. 1–6. 895–903 (2006).

6 of 7 | www.pnas.org/cgi/doi/10.1073/pnas.1908100117 Eckmann et al. Downloaded by guest on September 26, 2021 Downloaded by guest on September 26, 2021 23. 4 .Zu .Tish .E h,Jitlann fbnclrydie acdsadvergence and saccades driven binocularly of learning Joint Shi, E. B. Triesch, J. Zhu, Q. 24. 6 .G alt .Zag acigprut ihtm-rqec dictionaries. time-frequency with pursuits active Matching Zhang, of Z. Mallat, G. “Learning S. Triesch, 26. J. Shi, E. B. Lichtenstein, A. Lelais, A. Klimmasch, L. 25. 0 .A Movshon cats. A. deprived J. visually in retina 30. the of normality cortex Physiological Stone, visual J. Sherman, primary M. predict S. can 29. coding Sparse Goodhill, J. G. Dayan, P. Hunt, J. J. Thomas, A. 28. J. Cover, M. T. 27. 1 .Ysik,G .Badl .B eit .S ud eainbtenpten fintrinsic of patterns between Relation Lund, S. J. Levitt, B. J. Blasdel, G. G. Yoshioka, T. 31. 2 .FoecaIcrs,I .Gse,S .Hfr yatcognzto fvsa pc in space visual of organization Synaptic Hofer, B. S. Gasler, T. I. Iacaruso, Florencia M. 32. 8 .R hrda,T .Cny comdtv n egnersosst conflicting to at responses depth vergence of and perception Accommodative the Candy, and R. T. Blur Bharadwaj, Banks, R. S. S. M. Narain, 38. R. Love, D. G. Zannoli, M. higher-37. with Accommodation Williams, R. D. Singer, B. Hofer, computation. H. Kruger, neural B. P. canonical Chen, a L. as 36. Normalization Heeger, J. D. Carandini, suppression. M. cross-orientation and rivalry 35. binocular of model A Heeger, J. D. Said, cortex. P. visual C. human in 34. normalization contrast Inter-ocular Heeger, J. D. Moradi, F. 33. 0 .Hlpga,R lk,M .Genad eetv ossi ioua iinin vision binocular in losses Selective Greenwald, J. M. Blake, R. Holopigian, K. accommo- 40. on anisometropia lens-induced of effect The Candy, R. T. Bharadwaj, R. S. 39. 2 .Baly .D rea,Cnrs estvt naioerpcamblyopia. anisometropic in sensitivity Contrast Freeman, of D. extent R. Regional Bradley, Hess, F. A. R. 42. Thompson, B. Bobier, R. W. Clavagnier, S. Babu, J. R. 41. cmn tal. et Eckmann .L Steele L. A. alone. yatv fcetcoding. efficient active by inlProcess. Signal Robotics Epigenetic and Learning in system” and oculomotor Development (ICDL-EpiRob) the on of model Conference biomechanical International a in vision binocular i.Pyilgclobservations. Physiological iii. Res. Brain input. visual abnormal by (2013). induced e1003005 changes field receptive aea onciiy clrdmnne n yohoeoiaeratv ein in regions oxidase-reactive cortex. cytochrome striate monkey and macaque dominance, ocular connectivity, lateral rmr iulcortex. visual primary lraddsaiysiuidrn development. during stimuli disparity and blur occlusions. (2006). optics. 1–8 adaptive 23, with corrected aberrations monochromatic order Neurosci. Rev. Biol. Comput. PLoS Vis. nsmtoi amblyopes. anisometropic (2011). development. 3595–3603 visual 52, human during vergence and dation ptaml i.Sci. Vis. Ophthalmol. amblyopia. (2017). in suppression peripheral 3(2009). 13 9, .A.Asc eit.Ohhlo.Strabismus Ophthalmol. Pediatr. Assoc. Am. J. 60 .Vis. J. ucsflteteto nsmtoi mloi ihspectacles with amblyopia anisometropic of treatment Successful al., et 2–3 (1973). 224–230 IE,Psaaa,N,21) p 21–26. pp. 2017), NJ, Piscataway, (IEEE, 16 (2012). 51–62 13, fet feryuiaea lro h aau’ iulsystem. visual macaque’s the on blur unilateral early of Effects al., et 3731 (1993). 3397–3415 41, 7(2016). 17 16, 1091(2013). e1002991 9, 6–7 (1981). 467–476 21, Nature lmnso nomto Theory Information of Elements rn.Neurorob. Front. i.Res. Vis. 4–5 (2017). 449–452 547, .Neurosci. J. eer Cortex Cerebr. 2–3 (1986). 621–630 26, net ptaml i.Sci. Vis. Ophthalmol. Invest. 3015 (1987). 1340–1351 7, 8(2017). 58 11, .Vis. J. 9–1 (1996). 297–310 6, 74 (2006). 37–43 10, –8(2009). 1–18 9, net ptaml i.Sci. Vis. Ophthalmol. Invest. Jh ie os 2006). Sons, & Wiley (John LSCmu.Biol. Comput. PLoS .Ot o.A.A Am. Soc. Opt. J. 2329–2340 58, EETrans. IEEE on IEEE Joint Invest. Nat. 9, J. 3 .J .Eas ooiinAreview. A Monovision Evans, W. J. B. 43. 8 .F es .Topo,Abypaadtebnclrapoc oistherapy. its to approach binocular the and strategy Amblyopia Fixation Thompson, stereopsis: B. of Hess, side F. active R. The 48. Sabatini, P. S. Canessa, A. Gibaldi, A. 47. binocular for model cortical A Solari, F. Sabatini, P. S. Canessa, A. Chessa, M. Gibaldi, A. 46. Hyv A. Hoyer, O. P. 45. model control feedback Negative Stevenson, S. Cormack, L. Alexander, J. Schor, M. C. 44. 2 .Ekan .Kimsh cieefiin oigepan h eeomn fbinoc- of development the explains coding efficient Active Klimmasch, L. Eckmann, S. 62. St H. Kraskov, A. 61. distributions. unimodal Finch, for variance T. on Bounds 60. Hero, O. A. Sadler, M. B. functional Chung, W. H. and 59. interaction Bergen, binocular R. J. fields, Adelson, H. Receptive E. Wiesel, 58. N. algorithms. T. actor-critic Hubel, Natural H. Lee, D. M. Ghavamzadeh, 57. M. Sutton, R. Bhatnagar, S. Barto, 56. G. A. Sutton, sensory S. a shading of R. in recovery the 55. information for algorithm Predictive inspired biologically Bialek, A Kingdom, W. A. F. Olmos, Berry, A. J. M. 54. Marre, O. Palmer, E. S. theory? brain 53. unified A principle: inter- free-energy functional The A Friston, K. cortex: visual 52. the in coding Predictive Ballard, H. D. Rao, N. P. R. 51. amblyopic in suppression interocular of Dynamics Fronius, M. Kohnen, T. Kehrein, S. 50. Hallum E. L. 49. n dpaint aua environments. natural to adaptation and (2010). disparity. of 1073, calculation explicit without control vergence images. stereo (2000). and colour from extraction (1992). accommodation. and convergence proximal of lrvso n t alr nabypa oeD.http://modeldb.yale.edu/261483. ModelDB. amblyopia. 2020. in February failure 15 Deposited its and vision ular (2004). 066138 69, 2009). Service, Computing Cambridge Theor. Inf. Trans. IEEE 3–20. pp. 1991), Press, (MIT cortex. visual cat’s the in architecture Automatica 2018). images. reflectance and population. (2010). 127–138 effects. receptive-field extra-classical (1999). some of pretation insight. First (2016). therapy: 51–62 occlusion 24, monitored electronically during children monkeys. macaque amblyopic of cortex (2017). visual in sion Res. –6(2015). 4–16 114, nrmna aclto fWihe enadVariance and Mean Weighted of Calculation Incremental 4128 (2009). 2471–2482 45, U.S.A. Sci. Acad. Natl. Proc. lee aac frcpiefil xiainadsuppres- and excitation field receptive of balance Altered al., et gae,P rsbre,Etmtn uulinformation. mutual Estimating Grassberger, P. ogbauer, ¨ rnn needn opnn nlssapidt feature to applied analysis component Independent arinen, ¨ 9664 (2017). 6936–6949 63, Perception h lnpi ucinadteEeet fEryVision Early of Elements the and Function Plenoptic The enocmn erig nIntroduction An Learning: Reinforcement 4317 (2004). 1463–1473 33, ptamcPyil Optic. Physiol. Ophthalmic .Physiol. J. 9861 (2015). 6908–6913 112, c.Rep. Sci. ew opt erlSyst. Neural Comput. Netw. ptamcPyil Optic. Physiol. Ophthalmic 0–5 (1962). 106–154 160, 40 (2017). 44800 7, NSLts Articles Latest PNAS .Neurosci. J. Neurocomputing a.Neurosci. Nat. a.Rv Neurosci. Rev. Nat. 1–3 (2007). 417–439 27, 8216–8226 37, Uiest of (University 191–210 11, 307–318 12, hs e.E Rev. Phys. MTPress, (MIT Strabismus 1065– 73, | 79–87 2, 7 f7 of Vis. 11,

BIOPHYSICS AND NEUROSCIENCE COMPUTATIONAL BIOLOGY