Optimum Spatiotemporal Receptive Fields for Vision in Dim Light

Journal of Vision (2009) 9(4):18, 1–16 http://journalofvision.org/9/4/18/ 1

Optimum spatiotemporal receptive ﬁelds for vision in dim light

Department of Neuroscience, Karolinska Institute, Stockholm, Sweden,& Department of Cell and Organism Biology, Lund University, Andreas Klaus Lund, Sweden

Department of Cell and Organism Biology, Lund University, Eric J. Warrant Lund, Sweden

Many nocturnal insects depend on vision for daily life and have evolved different strategies to improve their visual capabilities in dim light. Neural summation of visual signals is one strategy to improve visual performance, and this is likely to be especially important for insects with apposition compound eyes. Here we develop a model to determine the optimum spatiotemporal sampling of natural scenes at gradually decreasing light levels. Image anisotropy has a strong influence on the receptive field properties predicted to be optimal at low light intensities. Spatial summation between visual channels is predicted to extend more strongly in the direction with higher correlations between the input signals. Increased spatiotemporal summation increases signal-to-noise ratio at low frequencies but sacrifices signal-to-noise ratio at higher frequencies. These results, while obtained from a model of the insect visual system, are likely to apply to visual systems in general. Keywords: neural summation, apposition compound eye, receptive field, anisotropy, signal-to-noise ratio, optimization Citation: Klaus, A., & Warrant, E. J. (2009). Optimum spatiotemporal receptive fields for vision in dim light. Journal of Vision, 9(4):18, 1–16, http://journalofvision.org/9/4/18/, doi:10.1167/9.4.18.

Nilsson, 1989; Warrant, 2004). A general strategy to Introduction increase light capture is to simply increase eye size (Jander & Jander, 2002; Land & Nilsson, 2002). This, Vision is an important sense for most species within the however, is constrained by the metabolic costs of neural animal kingdom and even in the dimmest habitats animals information processing (Laughlin, de Ruyter van Steveninck, have functional eyes (Warrant, 2004). A nocturnal life- & Anderson, 1998; Niven, Andersson, & Laughlin, 2007), style has several advantages, such as reduced predation and by locomotory costs (especially for flying insects), pressure and access to underexploited sources of food introduced by the additional weight of the eyes. (Roubik, 1989; Warrant, Porombka, & Kirchner, 1996; In addition to optical improvements, visual sensitivity Wcislo et al., 2004). However, nocturnal life also has a can also be increased by neural adaptations. Adjacent crucial disadvantage: the scarcity of light. At night, light points in natural images are correlated, and this correla- intensities can be up to 9 orders of magnitude lower than tion falls exponentially as the distance between the points during the day at the same location and this scarcity of increases (Srinivasan, Laughlin, & Dubs, 1982), a fact that light can make vision unreliable: visual signals in dim has been further studied by analyzing the power spectra of light are contaminated by visual “noise”. Part of this noise natural scenes (van der Schaaf & van Hateren, 1996). The arises from the stochastic nature of photon arrival and spectral power in terrestrial images falls with increasing 2 absorption: each sample of absorbed photons (or signal) spatial frequency fs as 1/fs (Dong & Atick, 1995; Field, has a certain degree of uncertainty (or noise) associated 1987; Simoncelli & Olshausen, 2001; van der Schaaf & with it. The relative magnitude of this uncertainty is van Hateren, 1996). Due to correlation within natural greater at lower rates of photon absorption, and these images, neighboring visual channels share some informa- quantum fluctuations set an upper limit to the visual tion, and thus a proportion of the signal generated in one signal-to-noise ratio (de Vries, 1943; Rose, 1942). The channel can be predicted from the signals generated in dimmer the light level, the lower the signal-to-noise ratio neighboring channels (Srinivasan et al., 1982; Tsukamoto, (Land, 1981; Snyder, 1979), and a visual system straining Smith, & Sterling, 1990; van Hateren, 1992c). In bright to see at night should therefore capture as much light as light this “predictive coding” leads to visual sampling possible. This can be achieved optically by enlarging the involving lateral inhibition between adjacent channels receptive fields of the photoreceptors and/or by widening (band-pass filtering), whereas in dim light sampling the pupil of the eye (Kirschfeld, 1974; Land, 1981; involves spatial summation of signals from groups of

doi: 10.1167/9.4.18 Received November 3, 2008; published April 22, 2009 ISSN 1534-7362 * ARVO

Downloaded from jov.arvojournals.org on 04/06/2019 Journal of Vision (2009) 9(4):18, 1–16 Klaus & Warrant 2

neighboring channels (low-pass filtering). As shown the apposition compound eye (Figure 1B), a design in theoretically, a strategy of neural summation of light in which single photoreceptors receive light only from the space and time can enable an eye to improve visual single corneal facet lens residing in the same ommati- reliability in dim light (Tsukamoto et al., 1990; van dium. Not surprisingly, apposition eyes are typical of Hateren, 1992c; Warrant, 1999). Neural summation will, diurnal insects active in bright sunlight, such as bees, flies, however, cause a decrease in spatial and temporal butterflies, and dragonflies. Interestingly, despite their resolution (Warrant, 1999). disadvantages for vision in dim light, some nocturnal Despite its difficulties, many nocturnal animalsV insects have apposition compound eyes. In tropical rain- including insectsVrely on vision for the tasks of daily forests, for instance, the competition for resources and the life (Warrant, 2007, 2008a). Insects have compound eyes, threat of predation are fierce, and many species of bees an eye type constructed from an assembly of repeated and waspsVall with apposition eyesVhave evolved a optical units, the “ommatidia” (Figure 1A). Nocturnal nocturnal lifestyle, an evolutionary transition that is insects, including most moths and many beetles, typically probably relatively recent (Wcislo et al., 2004). Animals have refracting superposition compound eyes, a design with apposition eyes have evolved a number of adapta- that allows single photoreceptors in the retina to receive tions that improve sensitivity in dim light (Barlow, focussed light from hundreds (and in some extreme cases, Kaplan, Renninger, & Saito, 1987;Warrant,2007, thousands) of corneal facet lenses (Figure 1C). This 2008b) and that allow visually guided behavior at lowest design represents a vast improvement in sensitivity over light intensities (Barlow & Powers, 2003; Warrant et al., 2004). For example, the apposition eyes of the nocturnal sweat bee, Megalopta genalis, possess larger corneal facet lenses and wider rhabdoms than those of closely related diurnal bees, and thus enjoy an increased optical sensitivity (Greiner, Ribi, & Warrant, 2004). Further improvements in visual sensitivity are likely to come from enhanced visual gain in the photoreceptors (Barlow, Bolanowski, & Brachman, 1977; Barlow et al., 1987; Frederiksen, Wcislo, & Warrant, 2008) and from a strategy of visual summation in space and time (Theobald, Greiner, Wcislo, & Warrant, 2006; Warrant et al., 2004). Morphological, electrophysiological, and theoretical evi- dences (Warrant, 2008b) suggest that widely arborizing second-order cells in the lamina (the LMC cells) might mediate a spatial summation of visual signals generated in the retina (Greiner, Ribi, & Warrant, 2005; Greiner, Ribi, Wcislo, & Warrant, 2004). Figure 1. Compound eyes. (A) A schematic longitudinal section Thus, when a diurnal insect with apposition eyesVlike (and an inset of a transverse section) through a generalized the ancestor of the nocturnal bee MegaloptaVbegan to Hymenopteran ommatidium (from an apposition eye, see B), evolve a nocturnal lifestyle, the optical and neural showing the corneal lens (c), the crystalline cone (cc), the primary structures of the eye altered according to natural selection pigment cells (pc), the secondary pigment cells (sc), the rhabdom (Frederiksen & Warrant, 2008). As the diurnal ancestor (rh), the retinula cells (rc), the basal pigment cells (bp), and the conquered dimmer and dimmer niches, its eyes evolved basement membrane (bm). The upper half of the ommatidium greater sensitivity: widening rhabdoms increased the shows screening pigment granules in the dark-adapted state, while visual fields of the photoreceptors, and the dendritic trees the lower half shows them in the light-adapted state. Redrawn from of the LMC cells became more extensive, possibly in Stavenga and Kuiper (1977), with permission. (B) A focal order to mediate spatial summation. How should these apposition compound eye. Light reaches the photoreceptors optical and neural changes evolve in order to continuously exclusively from the small corneal lens (co) located directly provide optimal visual performance during the evolution above, within the same ommatidium. This eye design is typical of a nocturnal lifestyle? of day-active insects. cc = crystalline cone. (C) A refracting The aim of this investigation is to model how an array superposition compound eye. A large number of corneal facets, of visual channels samples natural grayscale scenes at and bullet-shaped crystalline cones of circular cross-section different light intensities and image velocities. How, for (inset), collect and focus light across the clear zone of the eye instance, do the spatiotemporal properties of this sampling (c.z.) toward single photoreceptors in the retina. Several hun- “evolve” in response to an increasingly nocturnal life- dreds, or even thousands, of facets service a single photo- style? In addition, do the spatiotemporal properties of receptor. Not surprisingly, many nocturnal and deep-sea animals visual sampling depend on the image statistics of the have refracting superposition eyes and benefit from the significant natural scene being viewed? The average power spectrum improvement in sensitivity. (B) and (C) Courtesy of Nilsson (1989). derived from large numbers of individual natural scenes is

Downloaded from jov.arvojournals.org on 04/06/2019 Journal of Vision (2009) 9(4):18, 1–16 Klaus & Warrant 3

largely isotropic (Balboa & Grzywacz, 2003; van der Schaaf & van Hateren, 1996), and most visual modeling studies thus consider the scene to be likewise isotropic (Srinivasan et al., 1982; van Hateren, 1992c). However, the power spectra of individual natural scenes can show signiﬁcant anisotropy (Balboa & Grzywacz, 2003; see also Coppola, Purves, McCoy, & Purves, 1998; Hansen & Essock, 2004), such as the rainforest habitat of the nocturnal bee Megalopta, which is dominated by vertical tree trunks. How might such natural scenes affect the sampling strategy that “evolves”?

Theory and methods

Rationale

Natural grayscale images were used to stimulate an array of visual channels whose spatial and temporal properties could be varied. Natural grayscale images were simulated at different light levels I by superimposing photon noise (Figure 2C). Different image velocities were simulated by shifting static grayscale images by known numbers of pixels per unit time (Figure 2B). Here we consider only image motion in a horizontal direction and denote the horizontal image velocity by ! (measured in units of visual channel widths per 10 ms, see below). We then generate an array of visual channels with a spatial receptive field of vertical and horizontal half- $> $> widths v and h, respectively, and with a temporal . $ Figure 2 Modeling of image motion and light intensity. (A) Original integration time of t ms. The array of visual channels image data of size 1536 1024 pixels (width height) were corresponds to the array of image pixels, and thus, $>v $> taken from the natural image collection of van Hateren and van and h are measured in units of pixel widths (or units of der Schaaf (1998). For stimulus creation and data analysis, visual channel widths). square subimages of size M M are considered. (B) A moving For each light intensity I (at any given image velocity !) $> $> $ stimulus is created by horizontally shifting the array of visual the parameters v, h, and t are allowed to “evolve” in channels (white square in A) over the image. The result is a an iterative fashion to provide an optimal sampling of the sequence of image frames (t =0,1,2,I). The time between two input image. By “optimal” we mean the sampling that subsequent image frames was chosen to be 10 ms. (C) Super- results in a filtered output image that is most like the imposing Poisson noise results in a moving stimulus that unfiltered initial (noiseless) image. Thus, using this resembles the scene gathered by the visual channels at a given rationale we will determine how the spatiotemporal light intensity. properties of visual sampling “evolve” to optimize vision at dimmer and dimmer light levels. correspond to the vertical and horizontal directions in the image, respectively. In Figure 3 two images with different Image data and data generation isotropy, and their normalized power spectra are shown (for the calculation of the one-dimensional power spectrum, see Natural image data are taken from the database of van Appendix C). The power for the isotropic scene is equally Hateren and van der Schaaf (1998). The images are distributed in all directions (Figure 3A). In the case of the provided in binary format and can be easily converted into anisotropic scene there is more power in the horizontal than the FITS format or imported into Matlab (MathWorks, in the vertical direction for most spatial frequencies fs MA, USA). For all simulations, square images of size M, (Figure 3B). usually set to 128 or 256, are used. An image can be Horizontal image motion is simulated by shifting an considered as a two-dimensional array of pixel intensities array of visual channels over an image gk,l, resulting in a (!) I gk,l with spatial coordinates k and l (k, l =0,T1, I), which sequence of images gk,l,t, t =0,1,2, (Figure 2B; see

Downloaded from jov.arvojournals.org on 04/06/2019 Journal of Vision (2009) 9(4):18, 1–16 Klaus & Warrant 4 Modeling the spatiotemporal receptive ﬁeld

A receptive field can be described by its spatial and temporal response functions. For high signal-to-noise ratios (bright light condition) receptive fields of lamina neurons are predicted to show center–surround antagonism (Srinivasan et al., 1982; van Hateren, 1992a, 1992c). This means that it is primarily changes in the incoming signal that are encoded and transferred to later stages of processing in the visual system. Visual channels in the surround predict the output of visual channels in the center, and only the difference in signals generated by the center and surround is sent to later stages of processing. The mechanism of lateral inhibition was first described between neighboring ommatidia in the horseshoe crab Limulus polyphemus (Hartline, Wagner, & Ratliff, 1956). Center–surround antagonism (or lateral inhibition) is generally associated with redundancy reduction in the early visual system (van Hateren, 1992a; but see also Figure 3. Image data of size 128 128 pixels. (A) An isotropic Barlow, 2001). Natural images contain “redundant” image and its one-dimensional and normalized power spectra. information due to their intrinsic autocorrelation, and the (For the calculation of the one-dimensional power spectrum, see limited dynamic ranges of photoreceptors and neurons Appendix C.) Note that there is no distinct difference between the strongly imply the importance of eliminating predictable power in horizontal and vertical directions. (B) An anisotropic signal components. Lateral inhibition leads to band-pass image and its one-dimensional and normalized power spectra filtering that is characterized by the enhancement of with a considerable difference between horizontal and vertical spatial edges and temporal transients (van Hateren, power over a wide range of spatial frequencies. 1992b). However, for low signal-to-noise ratios (dim light conditions) the band-pass characteristic of receptive fields is predicted to change to a low-pass characteristic. With Appendix A). The region of interest is given by k, l =0, decreasing signal-to-noise ratio the size of the excitatory I, M j 1. The time step between two subsequent image center widens and the contribution of the inhibitory (!) (!) frames gk,l,t and gk,l,t+1 was chosen to be 10 ms. The surround diminishes, and disappears altogether (Barlow, parameter ! describes the velocity of horizontal motion Fitzhugh, & Kuffler, 1957; Batra & Barlow, 1982; and is measured in units of receptor widths per 10 ms. For Renninger & Barlow, 1979; van Hateren, 1992c). Since (!) ! 9 0 the images gk,l,t contain motion blur. These images we are interested in the transition from bright to dim light represent the noiseless images at the level of the photo- we decided to account only for the more dominant receptors in the retina at time steps t =0,1,2,I. changes in the excitatory center and model the spatial A We can now superimpose photon noise onto the image receptive field by a Gaussian function with parameters v (!) A sequence gk,l,t (Figure 2B) and the result is a sequence of and h (see also Hemila¨, Lerber, & Donner, 1998): noisy images g(!,I), t =0,1,2,I (Figure 2C) with a mean "# ! k,l,t 1 u 2 v 2 intensity of I photons per visual channel per second (cf. GuðÞ¼; v exp j þ ; ð2Þ Supplementary Figure 4). Photon arrival is a random 2 Av Ah event, which follows a Poisson distribution. Assuming a mean intensity of I photons per visual channel per second which has its maximum value at u = v = 0. The spatial (!,I) $> $> we can generate the noisy image sequence g by scaling half-widths v and h can be calculated by simply k,l,t A A the noiseless image with its mean value to unity, multi- multiplying v and h in Equation 2 by a factor of 2.35. plying it by I/100 (one image frame corresponds to 10 ms), The temporal response of photoreceptors can be modeled $ C and drawing Poisson distributed random numbers at each by a log-normal function with parameters t and p pixel location according to its intensity value: (Howard, 1981; Howard, Dubs, & Payne, 1984; Payne & Howard, 1981). $t and Cp are measured in milliseconds ð!;IÞ ð!Þ g ; ; ¼ Poiss ðI=100 g^ ; ; Þ; ð1Þ and denote the half-width and the time-to-peak value of k l t k l t the impulse response function, respectively: (!) (!) where g^k,l,t represents the normalized image sequence gk,l,t with mean intensity equal to 1 (see Appendix A). Poiss(1) ln2ðt=C Þ returns a Poisson distributed random number with mean VtðÞ¼exp j p ; t 9 0: ð3Þ value 1. 2A2

Downloaded from jov.arvojournals.org on 04/06/2019 Journal of Vision (2009) 9(4):18, 1–16 Klaus & Warrant 5

The integration time $t can be calculated by padding, m has to be chosen according to the size of the receptive field (1% threshold for summation coefficients). ! t ¼ 20C sinhð1:177AÞ: ð4Þ For a given light intensity I and image velocity p smaller values of the MSE indicate a better “quality” of the filtered image, that is, a better match to the original $ C noiseless image. The goal is to find the “optimal” In our model the values t and p are not modeled $> $> $ independently but the parameter A is set to a constant spatiotemporal parameters v, h, and t that minimize C the difference between the filtered and noiseless image value. This means an increase of p comes with an ! increase in $t, and vice versa (cf. Equation 4). Howard sequences for a given combination of I and : A (1981) reported a of 0.31 in the dark- and light-adapted photoreceptors of the locust. The dark-adapted photo- MSE Y min: ð7Þ receptor in the worker honeybee has a A of 0.28, whereas A in M. genalis = 0.32 (Warrant et al., 2004). In all simulations we use a value of A = 0.32. MSE is the average over the MSE values of 5 noisy Filtering an image sequence with the spatiotemporal instances of the same image sequence filtered with the same receptive field yields a number of N filtered images receptive field. To find a local minimum of Equation 7 we apply the Broyden–Fletcher–Goldfarb–Shanno (BFGS) XTj1 method, which is widely used for nonlinear optimization ¼ ð ^ð!;IÞÞ ; ¼ ; I; j ; ð5Þ problems and belongs to the class of quasi-Newton hk;l;n Vt G*g k;l;tþn n 0 N 1 t¼0 methods (see, e.g., Press, Teukolsky, Vetterling, & Flannery, 2007). The optimal parameter values as where Vt and G = Gk,l are discretized and normalized obtained by the BFGS method depend, in general, on the versions of Equations 2 and 3, respectively (see initial parameter values. ! Appendix B). Vt is shifted such that its peak value is at For a given image velocity all simulations begin at t0 = )T/32 ()I2 denotes the floor function). This means that bright light intensities with a sampling that includes only a the images hk,l,n, n =0,I, N j 1inEquation 5 represent sole visual channel integrating signals over a single image the spatiotemporally filtered versions of the frames frame (i.e., $>v = $>h = 0.75 receptor widths, $t = 4.6 ms). (!,I) (!,I) ^ I ^ j Dimmer and dimmer light levels are subsequently simu- gk,l,t0, , gk,l,t0 + N 1 .(G * g) denotes convolution of an image g = gk,l,t with a Gaussian kernel G = Gk,l and yields lated and the spatiotemporal properties found optimal at a a spatially filtered image sequence (see Appendix B). For certain light intensity are then used as initial values for the all calculations the size of the spatial receptive field Gk,l is next dimmer light intensity. Thus, the spatiotemporal determined by the weighting coefficients where summa- receptive fields “evolve” from brighter to lower light levels. tion falls below 1% of the on-axis amplitude. The word “evolution” is used only for convenience and is not intended to suggest “biological evolution” since the model cannot account for all biological constraints. Modeling the “evolution” of visual sampling at decreasing light levels Results The benefits of seeing well in dim light are doubtless manifold for nocturnal animals (Roubik, 1989; Warrant, 2008a; Warrant et al., 1996). For this reason we do not Our model shows that optimal spatial and temporal model feature-based filtering but instead we aim to receptive field properties depend on light intensity, image determine the sampling strategy that gives the “best velocity, and on the anisotropy of the visual scene. An reconstruction” of a noisy stimulus. The difference analysis of the optimal spatiotemporal receptive fields also between the original, noiseless image and a filtered reveals that summation improves the signal-to-noise ratio version of the noisy image can be formulated by a mean at lower spatiotemporal frequencies but sacrifices it at square error criterion: higher frequencies. In Figures 4A and 4D a single frame (10 ms) from a moving image sequence at various light intensities is MXjmj1 XNj1 1 ð!Þ 2 shown for an isotropic and an anisotropic scene, respec- MSE ¼ hk;l;n j g^ ; ; þ ; ð6Þ ! N k l t0 n tively ( = 1.5 visual channel widths per 10 ms). The p k;l¼m n¼0 spatial “evolution” of the optimal receptive field is shown in Figures 4B and 4E for gradually decreasing light 2 with t0 = )T/32 (cf. Equation 5), and Np =(M j 2m) N.For intensities (from left to right). In bright light conditions the calculation of the MSE value (Equation 6)weuseN =10 spatial sampling only includes a single visual channel, and subsequent image frames. To exclude artifacts due to for the isotropic scene spatial summation involves more

Downloaded from jov.arvojournals.org on 04/06/2019 Journal of Vision (2009) 9(4):18, 1–16 Klaus & Warrant 6

Figure 4. Optimal spatial receptive fields at various light intensities (A–C) for an isotropic scene and (D–F) for an anisotropic scene (! = 1.5 receptor widths per 10 ms). (A) Single image frames at various light intensities for the isotropic case (each image frame corresponds to a 10-ms sample, size 128 128 pixels). (B) An array of 51 51 visual channels and their spatial contribution during summation. The sum of all weighting coefficients of the receptive field is normalized to unity. (C) Optimally spatiotemporally filtered image (!,I) frames from (A). The images are derived by filtering the whole noisy image sequence gk,l,t with the optimal spatial receptive field Gk,l, and

the optimal temporal response function Vt (cf. Equation 5). (D–F) The same as (A)–(C) but for an anisotropic scene. The resulting spatial receptive ﬁelds show a distinct anisotropy at low light intensities.

and more neighboring visual channels as light intensity spatiotemporally filtered image sequences are shown in falls. In the case of an anisotropic scene the spatial Figures 4C and 4F, respectively. For any given image receptive field “evolves” an anisotropy with summation velocity a decrease in light intensity gradually increases more pronounced in the vertical direction. This anisotropy spatial summation and temporal integration (Figure 5, ! =1.5; can only be observed at lower light intensities when the see also Supplementary Figures 1A and 2A). sampling begins to sum over multiple visual channels. At For the isotropic scene our model predicts an almost logI = 1.5 the isotropic receptive field includes 27 isotropic spatial receptive field with similar amounts of channels with at least 1% contribution (compared to the summation in the vertical and horizontal directions visual channel at the center of the receptive field). At a (Figure 5A). However, for the image sequence with ! =1.5 comparable light level (logI = 0.8) the anisotropic there is slightly more summation in the horizontal receptive field includes only 23 channels. Interestingly, direction (i.e., in the direction of image motion). At lower decreasing the light level by one further log unit inverts light levels the ratio between vertical and horizontal the situation: 61 channels contribute to the spatial pooling is less than 1 ($>v/$>h = 0.8, logI = j0.5). This summation in the isotropic case (logI = 0.5; at least 1% is due to image blur, which is introduced at the level of contribution), while 87 visual channels contribute in the the visual channels, and horizontal summation increases anisotropic case (logI = j0.2; see also Supplementary monotonically as image velocity ! increases. For ! = 1.5 Figure 3). Single image frames of the optimally temporal integration increases with decreasing light

Downloaded from jov.arvojournals.org on 04/06/2019 Journal of Vision (2009) 9(4):18, 1–16 Klaus & Warrant 7

velocities the receptive ﬁeld almost exclusively sums temporally (Figure 6, anisotropic image). Although this behavior can be observed for both the isotropic and anisotropic scenes, temporal integration is less extensive for the latter (cf. Supplementary Figures 1 and 2). For the anisotropic scene an increase in image velocity ! leads to an asymptotic increase of summation in the vertical direction (Figure 6A), whereas summation in the horizontal direction continuously increases (Figure 6B). This is due to motion blur, which increases as ! increases. Furthermore, increasing the image velocity at low light

Figure 5. Optimum spatiotemporal receptive ﬁeld parameters for ! = 1.5. (A) Isotropic image sequence (as used in Figure 4A). (B) Anisotropic image sequence (as used in Figure 4D). The dashed lines correspond to no spatial/temporal pooling, which includes only a single visual channel and/or a single image frame (10 ms).

intensity in a similar fashion to spatial pooling. In this particular case this indicates a balanced summation in both space and time. For the anisotropic scene, which is characterized by an increased power in the horizontal direction (Figure 3B), our model predicts significantly increased vertical summation (Figure 5B). The scene contains trunks of trees in a forest and these vertical structures cause a high degree of correlation between the intensities of vertically aligned image pixels. And despite motion blur (in the horizontal direction), the ratio between the optimal vertical and horizontal receptive field half-widths at lower light levels is greater than 1 ($>v/$>h = 7.8, logI = j1.2). The onset of temporal integration occurs for ! Q 1.5 only at light Figure 6. Optimum spatiotemporal receptive field parameters for intensities lower than logI = 0.8 (Figure 5B; Supplemen- the anisotropic scene at various light intensities I and image tary Figure 2A). velocities ! (! = 0.1, 0.2, I, 4.0 visual channel widths per 10 ms).

The inﬂuence of the light intensity I, however, strongly (A) Vertical half-width $>v. (B) Horizontal half-width $>h. (C) depends on the image velocity !. At very low image Temporal integration time $t.

Downloaded from jov.arvojournals.org on 04/06/2019 Journal of Vision (2009) 9(4):18, 1–16 Klaus & Warrant 8

intensities leads to a strong reduction of temporal strategy as we define it here. So what is the advantage of integration (Figure 6C). spatial and temporal summation? To answer this question The criterion for the “optimal” receptive field was given we compared the optimally filtered scenes with the as the mean square error (Equations 6 and 7), whose corresponding unfiltered scene for each light level. As minimum value gives the “best” prediction of the original can be seen in Figure 7B, the ratio between SNRopt and scene. However, what does this criterion mean for the SNRunfilt increases at lower spatial frequencies but drops performance of the visual system? To address this at higher frequencies. Spatiotemporal summation question, spatiotemporally filtered images were analyzed increases signal power at lower frequencies and blurs finer in the frequency domain. Not surprisingly, the signal- image details (at higher frequencies), thus reducing high to-noise ratio SNRopt =SNRopt( fs) for the optimally frequency signal power. The influence of spatiotemporal filtered images drops over the entire range of spatial summation on noise power, on the other hand, is to frequencies fs as light intensity falls (Figure 7A; see decrease it over the entire frequency range (not shown). Appendix D for the calculation of signal and noise). The At higher frequencies, however, signal power is suppressed model thus indicates that vision inevitably deteriorates at more than noise power. As a result, signal-to-noise ratio is lower and lower light levels even with the optimal pooling improved at low frequencies but reduced at higher frequencies. Thus, the effect of spatiotemporal summation at dimmer light levels is to increase the reliability of coarser spatial details, while sacrificing finer spatial details (Figure 7B). The same behavior can be observed in the temporal frequency domain (not shown): at dimmer light levels the reliability of slower details is increased, while faster details are sacrificed. The cut-off frequency at which SNRopt/SNRunfilt drops below 1 indicates the frequency at which signal- to-noise ratio is sacrificed at higher frequencies in order to improve the signal-to-noise ratio at lower frequencies. At logI = 3.5, the cut-off frequency is fs = 56 cycles/image but decreases to 18 cycles/image as light intensity falls to logI = j0.5.

Discussion

Nocturnally active animals possessing apposition compound eyes show several optical adaptations to improve their visual sensitivity in dim light (Barlow et al., 1987; Barlow & Powers, 2003; Greiner, 2006; Land, 1981; Nilsson, 1989). In addition to optical adaptations, monopolar cells in the first optic ganglion (the lamina) are thought to mediate spatial summation of visual signals from several neighboring ommatidia to further improve visual reliability (Greiner, 2006; Warrant, 2008b). A strategy of neural summation can increase visual reliability because adjacent points in natural imagesVand thus the input signals of adjacent visual channelsVare Figure 7. Optimal spatiotemporal filtering and visual performance correlated and share some amount of information (Smith, for an isotropic scene (! = 1.5). (A) Signal-to-noise ratio for the 1995; Srinivasan et al., 1982; Tsukamoto et al., 1990; van isotropic scene (Figure 3A) filtered by optimal spatiotemporal der Schaaf & van Hateren, 1996; van Hateren, 1992c). receptive fields. (B) Ratio between the signal-to-noise ratios of the Here, we modeled an array of visual channels to derive optimally filtered and the unfiltered image sequences. A decreas- the optimum spatiotemporal sampling of natural images ing cut-off frequency at lower light intensities indicates that for gradually decreasing levels of light intensity. Consis-

SNRopt(fs) is improved at lower spatial frequencies fs at the cost tent with previous theoretical studies (Theobald et al.,

of SNRopt(fs) at higher frequencies fs. (Note that at logI = j0.5 the 2006; van Hateren, 1992a, 1992c; Warrant, 1999), our

increase in SNRopt(fs) and SNRopt(fs)/SNRunﬁlt(fs) at high frequen- model consistently predicts an increase in spatial and cies is an artifact that originates from the Hann window.) temporal summation as light intensity falls (Figure 5,

Downloaded from jov.arvojournals.org on 04/06/2019 Journal of Vision (2009) 9(4):18, 1–16 Klaus & Warrant 9

Supplementary Figures 1A and 2A). The optimum LMC cells in MegaloptaVmight represent the neural combination of spatial and temporal summation also substrate mediating orientation selectivity or anisotropic depends on the amount of motion that is present in the spatial summation, respectively. image sequences (Figure 6, Supplementary Figures 1B The advantage of independent summation in different and 2B). Most importantly, the optimum spatial and orientations becomes evident when we consider the temporal properties account for differences in the image optimum values of a receptive field, which is constrained anisotropy in order to give the best reconstruction of the by the additional condition $>v = $>h. For example for the initial, noiseless image sequences (Figure 4). anisotropic image sequence with ! = 1.5 (at logI = j1.2), the optimization scheme in Equations 6 and 7, with the constraint $>v = $>h, yields a spatial half-width $> of 6.2 channel widths (which includes 193 visual channels); Image anisotropy and spatial summation this value is between the independently optimized half- widths $>v = 22.5 and $>h = 2.9 channel widths (with Studies of the power spectral density of natural images 337 visual channels in total). The isotropically constrained $> $> show that correlation in natural scenes is, on average, receptive field model ( v = h) results in too much almost equal in the vertical and horizontal directions, summation in the horizontal direction and too little respectively (Balboa & Grzywacz, 2003; van der Schaaf summation in the vertical direction, which together &vanHateren,1996). Individual scenes, or scenes degrades the summed output signal. Furthermore, in the derived from specific habitats, can exhibit, however, a anisotropic case the integration time $t increases from distinct anisotropy (Balboa & Grzywacz, 2003; van der 19 to 25 ms in order to counterbalance the reduced spatial Schaaf & van Hateren, 1996). In that case the input summation (193 instead of 337 channels). The longer $t, correlation between a visual channel and its neighboring however, cannot fully compensate for the predictive loss channels would vary with orientation and an optimal indicated by an increase of the MSE value for the sampling of anisotropic images should take that into constrained model (data not shown). This together shows account (Hosoya, Baccus, & Meister, 2005). Indeed, at that the shape of the receptive field should match the low light levels the spatial component of our receptive field anisotropy of the moving stimulus. Within the framework model “evolves” a pronounced anisotropy with up to 8-fold of the model presented here, and the optimization scheme greater summation in the vertical than in the horizontal we used in this study (see Theory and methods section), direction (Figure 4E, Supplementary Figure 2A): at low this is indeed possibleVthe receptive field model allows light intensities the ratio $>v/$>h is always greater than 1. the optimum combination of spatial and temporal param- At moderate light intensities and for high image veloc- eters to “evolve” for each combination of I and !. ities, however, horizontal summation dominates vertical summation due to motion blur. An increased summation in the horizontal direction, that is, in the direction of image motion, is also present for the isotropic scene. Not Spatial and temporal summation and the surprisingly, for the isotropic image sequence, the ratio influence of image motion $>v/$>h for optimum spatiotemporal receptive fields is less than or equal to 1 for all light levels I and image The optimum spatiotemporal parameters are strongly velocities ! (cf. Supplementary Figure 1A). influenced by the light intensity I and the image velocity In the retina of some amphibians and mammals it has !. A decrease in light intensity increases spatial and recently been reported that some ganglion cells adapt to temporal summation (Figure 5, Supplementary Figures 1A oriented stimuli over a time scale of several seconds and 2A). Image motion, however, has a different effect (Hosoya et al., 2005). This mechanism might also be on the spatial and temporal parameters, respectively found in insects since adaptation in retinal processing on a (Figure 6). Increasing image motion decreases $t, and to short time scale has not only been reported for mamma- counteract the loss in temporal summation the eye has to lian and amphibian species (Smirnakis, Berry, Warland, sum more extensively in space. Therefore, a shortening of Bialek, & Meister, 1997) but also for insects (de Ruyter the temporal integration time comes with an increase in van Steveninck, Bialek, Potters, & Carlson, 1994). summation in space; and vice versa: lengthening $t Adaptation to oriented stimuli on a short time scale might decreases $>v,h (Supplementary Figures 1B and 2B). be mediated by synaptic changes. Since such a mechanism Thus, the relative balance between spatial and temporal would exploit the correlation between neighboring input summation is likely to be strongly influenced by ecolog- signals (Hosoya et al., 2005), photon noise constitutes an ical and behavioral constraints: slow animals are likely to inherent limitation for such a strategy. Another possible invest more heavily in temporal summation than faster explanation for the adaptation to oriented stimuli is ones (Warrant, 1999). pattern-selective retinal interneurons (Bloomfield, 1994; An excellent case example is the fast flying nocturnal Hosoya et al., 2005). Such neuronsVand possibly also the bee Megalopta genalis, for which visual navigation plays

Downloaded from jov.arvojournals.org on 04/06/2019 Journal of Vision (2009) 9(4):18, 1–16 Klaus & Warrant 10 an important role in homing (Warrant et al., 2004). The Extent of spatiotemporal summation and integration time and acceptance angle of dark-adapted visual performance photoreceptors are both large in Megalopta; 32 ms and 5.6-, respectively (compared to 18 ms and 2.6-, respec- The optimization scheme that we follow in Equations 6 tively, in the diurnal honeybee Apis mellifera; Warrant and 7, that is, predictions based on comparisons of the et al., 2004). This coarser spatial and temporal resolution, original and noiseless image sequences, favors spatiotem- however, cannot on its own explain Megalopta’s behavior poral summation up to a point where the MSE value has at night. Since the dark-adapted integration time is not its minimum. Compared to a receptive field that does not exceptionally slow, spatial summation in the lamina is sum in space and/or time, the optimum receptive field likely to further improve Megalopta’s visual sensitivity. improves the signal-to-noise ratio of its output signals at This hypothesis is supported by morphological evidence low frequencies. This improvement, however, comes only in the lamina of M. genalis (Greiner et al., 2005; Greiner, by sacrificing the signal-to-noise ratio at higher frequen- Ribi, Wcislo et al. 2004; Warrant et al., 2004) and by cies (Figure 7B). This indicates a trade-off between visual theoretical predictions (Theobald et al., 2006; Warrant, reliability and spatiotemporal resolution (Batra & Barlow, 1999). Our model confirms that spatial summation should 1990; Land, 1997). The trade-off can be influenced by be more extensive for greater image motion, as would be shifting the cut-off frequency (see Results section) toward experienced by Megalopta during rapid forward flight (cf. higher or lower frequencies, that is, by reducing or Figure 6). To be “optimal” such an extension of spatial increasing the amount of spatiotemporal summation, summation should account for differences in the image respectively (Figure 8). At this point it is interesting to anisotropy (see above). Vertical structures such as tree consider the influence of the anisotropic receptive field trunks can change the image statistics sufficiently to model on the vertical and the horizontal signal-to-noise depart markedly from isotropy (cf. Figure 3; but see also ratio, respectively. For the optimum spatiotemporal van der Schaaf & van Hateren, 1996). Although the visual receptive field, the cut-off frequency in the vertical environment of Megalopta’s habitat has not been inves- direction is lower than in the horizontal direction. tigated systematically, the average image statistics of Inspection of the power spectral density of the anisotropic rainforest scenes is not likely to be isotropic. A strategy of image (Figure 3B) shows that there is more horizontal anisotropic neural summation might therefore benefit than vertical power in the image. The optimum receptive Megalopta during its nightly navigation flights in the field tries to preserve as much power as possible in both rainforests of Central and South America. Indeed, the directions. Since image details are richer in the horizontal dendritic fields of its L4 lamina monopolar cells are direction, a higher cut-off frequency results for the anisotropic in shape, being strongly elongated in the horizontal compared to the vertical direction. For exam- vertical direction (Greiner et al., 2005). However, whether ple, at logI = j0.2 and for ! = 1.5 channel widths per the L4 cells are involved in anisotropic spatial summation 10 ms the vertical cut-off is at 13 cycles/image in contrast is yet to be determined. to the horizontal cut-off at 34 cycles/image (Figure 8B). Measurements of the ambient light intensity have This clearly shows the advantage of an optimally adapted shown that Megalopta can find its nest entrance when as summation in the respective directions. few as 4.7 photons per second (logI = 0.67) are absorbed on average by a single green photoreceptor (Warrant et al., 2004). At this and somewhat higher light levels, Arrangement of visual channels and receptive Megalopta also flies through the rainforest, navigating field centers between the nest and foraging sites. The dark-adapted integration time of Megalopta is 32 ms (see above). For In this study we investigated the optimum spatiotem- the isotropic image (logI = 0.5) our simulations predict poral parameters of identical receptive fields. In our this integration time to be optimal for an image velocity model, the spacing between the centers of the receptive ! È 0.7 receptor widths per 10 ms (cf. Supplementary fields was equal to the spacing of the visual channels and Figure 1B). In the anisotropic case (logI = 0.8) we found this was not subject to variation (see Theory and methods this integration time to be optimal for ! È 0.2–0.3 section). However, the actual arrangement of the photoreceptor widths per 10 ms, that is, for much slower image receptors and/or of the receptive field centers in the retina velocities compared to the isotropic image (cf. Supple- are important determinants of the information that is sent mentary Figure 2B). In the latter case, an increase of ! to later stages of the visual system (Borghuis, Ratliff, above 0.3 receptor widths per 10 ms would result in Smith, Sterling, & Balasubramanian, 2008; Tsukamoto additional motion blur for the given integration time of et al., 1990). Changes in the densities and optical proper- 32 ms. In other words, the dynamics of the temporal ties of photoreceptors influence visual acuity and infor- receptive field would be too slow for ! 9 0.3, resulting in mation capacity both in compound eyes (Land, 1997; an impairment of spatial and temporal resolution (Batra & Land & Eckert, 1985) and camera-type eyes (Packer, Barlow, 1990; Juusola & French, 1997; Srinivasan & Hendrickson, & Curcio, 1989; Sterling & Demb, 2004). Bernard, 1975). For instance, a higher cone density improves the signal-

Downloaded from jov.arvojournals.org on 04/06/2019 Journal of Vision (2009) 9(4):18, 1–16 Klaus & Warrant 11

Figure 8. Different degrees of filtering at logI = j0.2, ! = 1.5. (A) Image frames of 10 ms, from left to right: unfiltered, too little filtering 1$> 1$ fi $> $ fi $> $ $> $ (2 v,h, 2 t), optimal ltering ( v,h, t), and too much ltering (2 v,h,2 t). Here, v,h and t denote the optimum spatiotemporal parameters at logI = j0.2 and for ! = 1.5 channel widths per 10 ms (cf. Equations 6 and 7). (B) The corresponding signal-to-noise ratio

curves in the vertical and horizontal directions (SNRv and SNRh, respectively). The optimum spatiotemporal receptive ﬁeld preserves ﬁner image details in the horizontal direction as indicated by a higher cut-off frequency. The cut-off frequencies are 19 (too little), 13 (optimal),

and 9 cycles/image (too much ﬁltering) for SNRv, and 44 (too little), 34 (optimal), and 22 cycles/image (too much ﬁltering) for SNRh.

to-noise ratio of retinal ganglion cells (Tsukamoto et al., in order to reliably distinguish coarse and/or slow-moving 1990), and the mosaic arrangement of overlapping structures. With a strategy of neural summation this ganglion cell receptive fields in mammals probably comes, however, only at the cost of a deterioration in prevents aliasing artifacts and helps to provide a uniform signal-to-noise ratio at higher frequencies. This means that spatial sensitivity (de Vries & Baylor, 1997). Moreover, fine and/or fast-moving image structures disappear either an optimal overlap between ganglion cell receptive fields due to noise, orVin the case of spatiotemporal reduces the amount of redundancy in the visual signal, summationVby the summation of visual signals. Our thereby maximizing the amount of information per results indicate that the optimum trade-off strongly receptive field (Borghuis et al., 2008; Laughlin, 1994). depends on the image statistics. In terms of optimum Taken together, the results of Borghuis et al. (2008), summation it is therefore predicted that an anisotropic Tsukamoto et al. (1990), and those of our study indicate power spectral density requires anisotropic spatial sum- that the spatial arrangement of the visual channels (and of mation. Thus, animals that live in environments that are their receptive fields), in addition to their spatiotemporal highly anisotropic are predicted to spatially sum visual properties, can be optimized in order to maximize the signals in dim light in an anisotropic fashion. Finally, amount of (relevant) information that is sent to later stages even though these conclusions have been derived from a of visual processing. For the visual system as a whole, model of an insect visual system, the results obtained in however, all its components must be correspondingly this study should apply to any visual system that operates tuned in order to optimally support visually guided under similar intensity constraints. behavior. Appendix A Conclusion Data generation Light intensity and image motion are two important parameters that partly determine the spatial and temporal Given that the natural image data can be specified by sampling properties of visual receptive fields. In addition, an array of intensity values g , we first define a function our model predicts that the average image anisotropy of k,l ge(u, v) with real-valued variables u and v in the following the scene being viewed by an array of visual channels has way: a strong influence on the spatialVand therefore also temporalVsampling properties. Spatiotemporal summa- eð ; Þ :¼ ; ðA1Þ tion improves the signal-to-noise ratio at low frequencies g u v g)u2;)v2

Downloaded from jov.arvojournals.org on 04/06/2019 Journal of Vision (2009) 9(4):18, 1–16 Klaus & Warrant 12

where )I2 denotes the floor function. The sequence of k, l = jM/2 + 1, I, M/2. The sum of all entries in Gk,l is images with motion in a horizontal direction is then normalized to unity. For the temporal response function calculated by (i.e., Equation 3) we proceed in a similar way: Z b ð!Þ Vðt þ t0 j CpÞ g ; ; ¼ wðJÞgeðk; JÞdJ; ðA2Þ ¼ ; ðB2Þ k l t Vt j a XT 1 Vði þ t0 j CpÞ where a = !t defines the left “edge” of a visual channel at i¼0 time t =0,I, T j 1 and b = !t + ! + 1 defines the right “edge” after moving (t + 1). t =0,I, T j 1. In this case we shift the peak value of Vt For motion with a constant velocity the function w = to the t0th time step (t0 = )T/32). In addition to Equation 3 w(u) is defined as we assume V(t) = 0 for t e 0. The convolution of an image sequence g = gk,l,t with a 8 w Gaussian kernel G = G in Equation 5 is calculated as > max ðÞu j a a e u G u k,l > u j a 1 follows: > 1 > < e G wmax u1 u u2 XM=2 wuðÞ¼ ðA3Þ > wmax ðG*gÞ ; ; ¼ G ; g þ ; þ ; ; ðB3Þ > ðÞu j b u e u G b k l t i j k i l j t > j 2 ; ¼j = þ > u2 b i j M 2 1 :> 0 otherwise; and yields a spatially filtered image sequence of the same size as the input sequence. For the calculation of where u1 = min{!t +1,!t + !}, u2 = max{!t +1,!t + !}, Equation B3 we used zero-padded images g , t =0,1, ! k,l,t and wmax = 1/max{1, }. It follows that 2, I (i.e., gk,l,t := 0 for k, l G 0 and k, l Q M) and choose M Z Z to be a power of 2. This allows a computationally efficient V b calculation of Equation B3 by means of the fast Fourier wðuÞdu ¼ wðuÞdu ¼ 1: ðA4Þ transform (see, e.g., Press et al., 2007). jV a

For all calculations, image sequences are normalized with their mean intensity to unity: Appendix C

gk;l;t g^ ; ; ¼ ; ðA5Þ Power spectrum calculation k l t MXj1 XTj1 gi;j;n For an image gk,l, k, l =0,I, M j 1, of size M, we want i;j¼0 n¼0 to assume that g^m,n is its Fourier transform. The Fourier transform g^ is a periodic function of which we will I j I j m,n for k, l =0, , M 1 and t =0, , T 1. M is the size consider the range M/2 e m, n G M/2 (M is a power of 2). ^ of each image in the sequence gk,l,t. If not otherwise This means that the zero-frequency component is at m = mentioned we used M = 128 and T =32+N = 42 (cf. n = 0. The two-dimensional power spectrum of the image Equation 6) in our simulations. 2 gk,l is then given by ªg^m,nª (Figure C1B). To avoid padding artifacts a Hann window of size M M is applied before the calculation of the Fourier transform (Press et al., Appendix B 2007). To reduce the two frequency components m and n to a single spatial frequency fs, values of the power spectrum P I j Spatial and temporal filtering ( fs) are averaged on circles of radius fs =0, , M/2 1 around the zero frequency at m = n = 0 (cf. the light blue The discrete version of the Gaussian kernel G(u, v) circle in Figure C1B;[I] returns the nearest integer): (Equation 2) is given by X 2 Pð fsÞ¼ kg^ ; k : ðC1Þ pffiffiffiffiffiffiffiffiffiffi m n ð ; Þ 2 2 G k l fs¼½ m þn Gk;l ¼ ; ðB1Þ XM=2 ð ; Þ G i j To include values of the power spectrum only in the i;j¼jM=2þ1 horizontal or vertical direction, the average is taken from

Downloaded from jov.arvojournals.org on 04/06/2019 Journal of Vision (2009) 9(4):18, 1–16 Klaus & Warrant 13 ^ where [I] denotes the nearest integer function. Sm,n is the Fourier transform of

XN ¼ 1 ðiÞ ; ðD2Þ Sk;l hk;l;0 N i¼1

(i) and hk,l,n denotes the ith instance of a filtered image sequence (cf. Equation 5). In the same way we calculate the noise N (fs), that is, as the power spectrum of the standard deviation: vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u u XN 2 ¼ t 1 j ðiÞ : ðD3Þ Nk;l j Sk;l hk;l;0 N 1 i¼1

In all examples we used N = 10 samples for the calculation of the signal-to-noise ratio SNR(fs)=S(fs)/ N (fs).

Acknowledgments

Figure C1. Calculation of the one-dimensional power spectrum. We thank Dr. Jamie Carroll Theobald for valuable and (A) A grayscale image with Hann window. (B) The corresponding inspiring discussions. A.K. is grateful for the financial two-dimensional, logarithmically scaled power spectrum with zero support given by the German Academic Exchange Service frequency at the center pixel. To calculate the power at a certain (DAAD). E.J.W. is grateful for the ongoing support of the frequency f the (unscaled) two-dimensional power is averaged Swedish Research Council (Vetenskapsra˚det), the Royal s Physiographic Society of Lund, and the Air Force Office around a circle with radius fs (360-, light blue). To calculate the power in horizontal and vertical directions the two-dimensional of Scientific Research (AFOSR). power is averaged only for 23- sectors of a circle (light green and red arcs, respectively). Note that the two-dimensional power Commercial relationships: none. Corresponding author: Andreas Klaus. spectrum is logarithmically scaled for visualization only and ) averages are calculated from the unscaled power spectrum. (C) Email: Andreas.Klaus ki.se. Address: Department of Neuroscience, Karolinska Institute, The one-dimensional power spectrum with zero frequency (fs =0) scaled to unity. Asterisks indicate the frequencies as used in (B) Retzius va¨g 8, 17177 Stockholm, Sweden. for illustration. References an arc of a circle only (cf. the light green and red arcs in Figure C1B). The corresponding one-dimensional power Balboa, R. M., & Grzywacz, N. M. (2003). Power spectra spectrum averages are shown in Figure C1C. and distribution of contrasts of natural images from different habitats. Vision Research, 43, 2527–2537. [PubMed] Appendix D Barlow, H. (2001). Redundancy reduction revisited. Net- work, 12, 241–253. [PubMed] Image analysis Barlow, H. B., Fitzhugh, R., & Kuffler, S. W. (1957). Change of organization in the receptive fields of the cat’s retina during dark adaptation. The Journal of Here, we defined the signal S(f ) as the power spectrum s Physiology, 137, 338–354. [PubMed][Article] of the average of N filtered images: Barlow, R. B., Jr., Bolanowski, S. J., Jr., & Brachman, M. L. X ^ 2 (1977). Efferent optic nerve fibers mediate circadian SðfsÞ¼ kS^m;nk ; ðD1Þ pffiffiffiffiffiffiffiffiffiffi rhythms in the Limulus eye. Science, 197, 86–89. ¼½ 2þ 2 fs m n [PubMed]

Downloaded from jov.arvojournals.org on 04/06/2019 Journal of Vision (2009) 9(4):18, 1–16 Klaus & Warrant 14

Barlow, R. B., Jr., Kaplan, E., Renninger, G. H., & Saito, T. and the diurnal blue morpho Morpho peleides: A clue (1987). Circadian rhythms in Limulus photoreceptors. to explain the evolution of nocturnal apposition eyes? I. Intracellular studies. The Journal of General Journal of Experimental Biology, 211, 844–851. Physiology, 89, 353–378. [PubMed][Article] [PubMed][Article] Barlow, R. B., & Powers, M. K. (2003). Seeing at night Frederiksen, R., Wcislo, W. T., & Warrant, E. J. (2008). and finding mates: The role of vision. In C. N. Visual reliability and information rate in the retina of Shuster, R. B. Barlow, & H. J. Brockmann (Eds.), The a nocturnal bee. Current Biology, 18, 349–353. American horseshoe crab (pp. 83–102). Cambridge, [PubMed][Article] Massachusetts, and London, England: Harvard Greiner, B. (2006). Adaptations for nocturnal vision in University Press. insect apposition eyes. International Review of Cytol- Batra, R., & Barlow, R. B., Jr., (1982). Efferent control of ogy, 250, 1–46. [PubMed] pattern vision in Limulus lateral eye. Society for Greiner, B., Ribi, W. A., & Warrant, E. J. (2004). Retinal Neuroscience, 8, 49. and optical adaptations for nocturnal vision in the Batra, R., & Barlow, R. B., Jr., (1990). Efferent control of halictid bee Megalopta genalis. Cell and Tissue temporal response properties of the Limulus lateral Research, 316, 377–390. [PubMed] eye. The Journal of General Physiology, 95, 229–244. Greiner, B., Ribi, W. A., & Warrant, E. J. (2005). A [PubMed][Article] neural network to improve dim-light vision? Den- Bloomfield, S. A. (1994). Orientation-sensitive amacrine dritic fields of first-order interneurons in the nocturnal and ganglion cells in the rabbit retina. Journal of bee Megalopta genalis. Cell and Tissue Research, Neurophysiology, 71, 1672–1691. [PubMed] 322, 313–320. [PubMed] Borghuis, B. G., Ratliff, C. P., Smith, R. G., Sterling, P., Greiner, B., Ribi, W. A., Wcislo, W. T., & Warrant, E. J. & Balasubramanian, V. (2008). Design of a neuronal (2004). Neural organisation in the first optic ganglion array. Journal of Neuroscience, 28, 3178–3189. of the nocturnal bee Megalopta genalis. Cell and [PubMed][Article] Tissue Research, 318, 429–437. [PubMed] Coppola, D. M., Purves, H. R., McCoy, A. N., & Hansen, B. C., & Essock, E. A. (2004). A horizontal bias in Purves, D. (1998). The distribution of oriented human visual processing of orientation and its corre- contours in the real world. Proceedings of the spondence to the structural components of natural National Academy of Sciences of the United States scenes. Journal of Vision, 4(12):5, 1044–1060, http:// of America, 95, 4002–4006. [PubMed][Article] journalofvision.org/4/12/5/, doi:10.1167/4.12.5. [PubMed][Article] de Ruyter van Steveninck, R. R., Bialek, W., Potters, M., & Carlson, R. H. (1994). Statistical adaptation and Hartline, H. K., Wagner, H. G., & Ratliff, F. (1956). optimal estimation in movement computation by the Inhibition in the eye of limulus. The Journal of General blowfly visual system. In IEEE International Confer- Physiology, 39, 651–673. [PubMed][Article] ence on Systems, Man, and Cybernetics: ‘Humans, Hemila¨, S., Lerber, T., & Donner, K. (1998). Noise- information and technology’ (vol. 1, pp. 302–307). equivalent and signal equivalent visual summation of San Antonio, TX, USA: IEEE. quantal events in space and time. Visual Neuro- de Vries, H. L. (1943). The quantum character of light and science, 15, 731–742. [PubMed] its bearing upon threshold of vision, the differential Hosoya, T., Baccus, S. A., & Meister, M. (2005). sensitivity and visual acuity of the eye. Physica, 10, Dynamic predictive coding by the retina. Nature, 553–564. 436, 71–77. [PubMed] de Vries, S. H., & Baylor, D. A. (1997). Mosaic arrange- Howard, J. (1981). Temporal resolving power of the ment of ganglion cell receptive fields in rabbit retina. photoreceptors of Locusta migratoria. Journal of Journal of Neurophysiology, 78, 2048–2060. Comparative Physiology A: Sensory, Neural, and [PubMed][Article] Behavioral Physiology, 144, 61–66. Dong, D. W., & Atick, J. J. (1995). Statistics of natural Howard, J., Dubs, A., & Payne, R. (1984). The dynamics of time-varying images. Network: Computation in Neu- phototransduction in insectsVA comparative study. ral Systems, 6, 345–358. Journal of Comparative Physiology A: Sensory, Field, D. J. (1987). Relations between the statistics of Neural, and Behavioral Physiology, 154, 707–718. natural images and the response properties of cortical Jander, U., & Jander, R. (2002). Allometry and resolution cells. Journal of the Optical Society of America A, of bee eyes (Apoidea). Arthropod Structure & Optics and Image Science 4, 2379–2394. [PubMed] Development, 30, 179–193. [PubMed] Frederiksen, R., & Warrant, E. J. (2008). Visual sensi- Juusola, M., & French, A. S. (1997). Visual acuity for tivity in the crepuscular owl butterfly Caligo memnon moving objects in first- and second-order neurons of

Downloaded from jov.arvojournals.org on 04/06/2019 Journal of Vision (2009) 9(4):18, 1–16 Klaus & Warrant 15

the fly compound eye. Journal of Neurophysiology, Roubik, D. W. (1989). Ecology and natural history of 77, 1487–1495. [PubMed][Article] tropical bees. Cambridge: Cambridge University Kirschfeld, K. (1974). The absolute sensitivity of lens and Press. .. compound eyes. Zeitschrift fur Naturforschung C: Simoncelli, E. P., & Olshausen, B. A. (2001). Natural Biosciences, 29, 592–596. [PubMed] image statistics and neural representation. Annual Land, M. F. (1981). Optics and vision in invertebrates. In Review of Neuroscience, 24, 1193–1216. [PubMed] H. J. Autrum (Ed.), Handbook of sensory physiology, Smirnakis, S. M., Berry, M. J., Warland, D. K., Bialek, W., VII-6B (pp. 471–592). Berlin, Germany: Springer. & Meister, M. (1997). Adaptation of retinal process- Land, M. F. (1997). Visual acuity in insects. Annual ing to image contrast and spatial scale. Nature, 386, Review of Entomology, 42, 147–177. [PubMed] 69–73. [PubMed] Land, M. F., & Eckert, H. (1985). Maps of the acute zones Smith, R. G. (1995). Simulation of an anatomically of fly eyes. Journal of Comparative Physiology A: defined local circuit: The cone-horizontal cell net- Sensory, Neural, and Behavioral Physiology, 156, work in cat retina. Visual Neuroscience, 12, 545–561. 525–538. [PubMed] Land, M. F., & Nilsson, D.-E. (2002). Animal eyes. New Snyder, A. W. (1979). Physics of vision in compound York: Oxford University Press. eyes. Handbook of Sensory Physiology, 7, 225–313. Laughlin, S. B. (1994). Matching coding, circuits, cells Srinivasan, M. V., & Bernard, G. D. (1975). The effect of and molecules to signals: General principles of retinal motion on visual acuity of compound eye: A theoretical design in the fly’s eye. Progress in Retinal and Eye analysis. Vision Research, 15, 515–525. [PubMed] Research, 13, 165–196. Srinivasan, M. V., Laughlin, S. B., & Dubs, A. (1982). Laughlin, S. B., de Ruyter van Steveninck, R. R., & Predictive coding: A fresh view of inhibition in the Anderson, J. C. (1998). The metabolic cost of neural retina. Proceedings of the Royal Society of London B: information. Nature Neuroscience, 1, 36–41. Biological Sciences, 216, 427–459. [PubMed] [PubMed] Stavenga, D. G., & Kuiper, J. W. (1977). Insect pupil Nilsson, D.-E. (1989). Optics and evolution of the mechanisms. I. On the pigment migration in the compound eye. In D. G. Stavenga & R. C. Hardie retinula cells of Hymenoptera (suborder Apocrita). (Eds.), Facets of vision (pp. 30–73). Berlin, Germany: Journal of Comparative Physiology, 113, 55–72. Springer. Sterling, P., & Demb, J. B. (2004). Retina. In G. M. Niven, J. E., Andersson, J. C., & Laughlin, S. B. (2007). Shepherd (Ed.), The synaptic organization of the Fly photoreceptors demonstrate energy-information brain (pp. 217–269). New York: Oxford University trade-offs in neural coding. PLoS Biology, 5, e116. Press. [PubMed][Article] Packer, O., Hendrickson, A. E., & Curcio, C. A. (1989). Theobald, J. C., Greiner, B., Wcislo, W. T., & Warrant, E. J. Photoreceptor topography of the retina in the adult (2006). Visual summation in night-flying sweat bees: A pigtail macaque (Macaca nemestrina). Journal of theoretical study. Vision Research, 46, 2298–2309. Comparative Neurology, 288, 165–183. [PubMed] [PubMed] Payne, R., & Howard, J. (1981). Response of an insect Tsukamoto, Y., Smith, R. G., & Sterling, P. (1990). photoreceptor: A simple log-normal model. Nature, “Collective coding” of correlated cone signals in the 290, 415–416. retinal ganglion cell. Proceedings of the National Academy of Sciences of the United States of America, Press, W. H., Teukolsky, S. A., Vetterling, W. T., & 87, 1860–1864. [PubMed][Article] Flannery, B. P. (2007). Numerical recipes: The art of scientific computing. New York: Cambridge Univer- van der Schaaf, A., & van Hateren, J. H. (1996). sity Press. Modelling the power spectra of natural images: Statistics and information. Vision Research, 36, Renninger, G. H., & Barlow, R. B. (1979). Lateral 2759–2770. [PubMed] inhibition, excitation, and the circadian rhythm of the Limulus compound eye. Society for Neuroscience, van Hateren, J. H. (1992a). A theory of maximizing sensory 5, 804. information. Biological Cybernetics, 68, 23–29. [PubMed] Rose, A. (1942). The relative sensitivities of television pickup tubes, photographic film, and the human eye. van Hateren, J. H. (1992b). Real and optimal neural Proceedings of the IRE, 30, 293–300. images in early vision. Nature, 360, 68–70. [PubMed]

Downloaded from jov.arvojournals.org on 04/06/2019 Journal of Vision (2009) 9(4):18, 1–16 Klaus & Warrant 16

van Hateren, J. H. (1992c). Theoretical predictions of Warrant, E. J. (2008a). Nocturnal vision. In T. Albright & spatiotemporal receptive fields of fly LMCs, and R. H. Masland (Eds.), The Senses: A comprehensive experimental validation. Journal of Comparative reference (vol. 2, pp. 53–86). Oxford: Academic Press. Physiology A, 171, 157–170. Warrant, E. J. (2008b). Seeing in the dark: Vision and van Hateren, J. H., & van der Schaaf, A. (1998). visual behaviour in nocturnal bees and wasps. Journal Independent component filters of natural images of Experimental Biology, 211, 1737–1746. [PubMed] compared with simple cells in primary visual cortex. [Article] Proceedings of the Royal Society of London B: Warrant, E. J., Kelber, A., Gisleń,A.,Greiner,B., Biological Sciences, 265, 359–366. [PubMed] Ribi, W., & Wcislo, W. T. (2004). Nocturnal vision [Article] and landmark orientation in a tropical halictid bee. Current Biology, 14, 1309–1318. [PubMed][Article] Warrant, E. J. (1999). Seeing better at night: Life style, eye design and the optimum strategy of spatial and temporal Warrant, E. J., Porombka, T., & Kirchner, W. H. (1996). summation. Vision Research, 39, 1611–1630. Neural image enhancement allows honeybees to see [PubMed] at night. Proceedings of the Royal Society of London B: Biological Sciences, 263, 1521–1526. Warrant, E. J. (2004). Vision in the dimmest habitats on earth. Journal of Comparative Physiology A: Neuro- Wcislo, W. T., Arneson, L., Roesch, K., Gonzalez, V., ethology, Sensory, Neural, and Behavioral Physiology, Smith, A., & Fernandez, H. (2004). The evolution of 190, 765–789. [PubMed] nocturnal behaviour in sweat bees, Megalopta genalis and M. ecuadoria (Hymenoptera: Halictidae): An Warrant, E. J. (2007). Nocturnal bees. Current Biology, escape from competitors and enemies? Biological 17, R991–R992. [PubMed][Article] Journal of the Linnean Society, 83, 377–387.

Downloaded from jov.arvojournals.org on 04/06/2019