ORIGINAL RESEARCH ARTICLE published: xx February 2012 doi: 10.3389/fpsyg.2012.00043

001 Free-energy and illusions: the Cornsweet effect 058 002 059 003 060 004 Harriet Brown* and Karl J. Friston 061 005 The Wellcome Trust Centre for Neuroimaging, University College London Queen Square, London, UK 062 006 063 007 064 Edited by: 008 In this paper, we review the nature of illusions using the free-energy formulation of Bayesian 065 Lars Muckli, University of Glasgow, perception. We reiterate the notion that illusory percepts are, in fact, Bayes-optimal and 009 UK 066 010 represent the most likely explanation for ambiguous sensory input. This point is illustrated 067 Reviewed by: 011 Gerrit W. Maus, University of using perhaps the simplest of visual illusions; namely, the Cornsweet effect. By using 068 012 California at Berkeley, USA plausible prior beliefs about the spatial gradients of illuminance and reflectance in visual 069 013 Peter De Weerd, Maastricht scenes, we show that the Cornsweet effect emerges as a natural consequence of Bayes- 070 University, Netherlands 014 optimal perception. Furthermore, we were able to simulate the appearance of secondary 071 015 *Correspondence: illusory percepts () as a function of stimulus . The contrast-dependent 072 Harriet Brown, Wellcome Trust Centre 016 073 for Neuroimaging, Institute of emergence of the Cornsweet effect and subsequent appearance of Mach bands were 017 Neurology Queen Square, London simulated using a simple but plausible generative model. Because our generative model 074 018 WC1N 3BG, UK. was inverted using a neurobiologically plausible scheme, we could use the inversion as a 075 019 e-mail: [email protected] simulation of neuronal processing and implicit inference. Finally, we were able to verify the 076 020 qualitative and quantitative predictions of this Bayes-optimal simulation psychophysically, 077 021 using stimuli presented briefly to normal subjects at different contrast levels, in the context 078 022 of a fixed alternative forced choice paradigm. 079 023 080 024 Keywords: free-energy, perception, Bayesian inference, illusions, Cornsweet effect, perceptual priors 081 025 082 026 INTRODUCTION about the way that visual information is generated at different 083 027 Illusions are often regarded as “failures” of perception; however, spatial and temporal scales. 084 028 Bayesian considerations often provide a principled explanation for 085 029 apparent failures of inference in terms of prior beliefs. This paper THE CORNSWEET EFFECT AND THE NATURE OF ILLUSIONS 086 030 is about the nature of illusions and their relationship to Bayes- Figure 1 provides an illustration of the . The 087 031 optimal perception. The main point made by this work is that illusion is the false percept that the peripheral regions of a stimulus 088 032 illusory percepts are optimal in the sense of explaining sensations have a different brightness, despite the fact they are physically iso- 089 033 in terms of their most likely cause. In brief,illusions occur when the luminant. This illusion is induced by a biphasic luminance “edge” 090 034 experimenter generates stimuli in an implausible or unlikely way. in the centre of the field of view (shown in the right hand col- 091 035 From the subject’s perspective, these stimuli are ambiguous and umn of Figure 1). The four rows of Figure 1 show the Cornsweet 092 036 could be explained by different underlying causes. This ambiguity effect increasing in magnitude as we increase the contrast of the 093 037 is resolved in a Bayesian setting, by choosing the most likely expla- stimulus. Interestingly, at high levels of contrast, secondary illu- 094 038 nation, given prior beliefs about the hidden causes of the percept. sions – Mach bands (Mach, 1865; Lotto et al., 1999) – appear at 095 039 This key point has been made by many authors (e.g., Purves et al., the para-central points of inflection of the true luminance profile. 096 040 1999). Here, we develop it under biologically realistic simulations It is this contrast-dependent emergence of the Cornsweet effect 097 041 of Bayes-optimal perception and try to make some quantitative and subsequent Mach bands that we wanted to simulate, under 098 042 predictions about how subjects should make perceptual decisions. the assumption that perception is Bayes-optimal. 099 043 We then try to establish the scheme’s validity by showing that these The Bayesian aspect of perception becomes crucial when we 100 044 predictions are largely verified by experimental data from normal consider the nature of illusions. Bayesian theories of perception 101 045 subjects viewing the same stimuli. describe how sensory data (that have a particular likelihood) are 102 046 The example we have chosen is the Cornsweet illusion, which combined with prior beliefs (a prior distribution) to create a per- 103 047 has a long history, dating back to the days of Helmholtz (Mach, cept (a posterior distribution). One can regard illusory percepts 104 048 1865; O’Brien, 1959; Craik, 1966; Cornsweet, 1970). This is partic- as those that are induced by ambiguous stimuli, which can be 105 049 ularly relevant given our formulation of the Bayesian brain is based caused in different ways – in other words, the probability of the 106 050 upon the idea that the brain is a Helmholtz or inference machine data given different causes or explanations is the same. When faced 107 051 (von Helmholtz, 1866; Barlow, 1974; Dayan et al., 1995; Friston with these stimuli, the prior distribution can be used to create a 108 052 et al., 2006). In other words, the brain is trying to infer the hidden unimodal posterior and an unambiguous percept. If the percept 109 053 causes and states of the world generating sensory information, or inference about the hidden causes of sensory information (the 110 054 using predictions based upon a generative model that includes posterior distribution) is different from the true causes used to 111 055 prior beliefs. We hoped to show that the Cornsweet effect can generate stimuli, the inference is said to be illusory or false. How- 112 056 be explained in a parsimonious way by some simple prior beliefs ever, with illusory stimuli the mapping of hidden causes to their 113 057 114

www.frontiersin.org February 2012 | Volume 3 | Article 43 | 1 Brown and Friston Free-energy and illusions: the Cornsweet effect

115 172 116 173 117 174 118 175 119 176 120 177 121 178 122 179 123 180 124 181 125 182 126 183 127 184 128 185 129 186 130 187 131 188 132 189 133 190 134 191 135 FIGURE 2 | Contributions to luminance. Luminance values reaching the 192 are modeled as a multiplication of illuminance from light sources and 136 193 reflectance from the surfaces in the environment. The factorization of 137 194 luminance is thus an ill-posed problem. One toy example of this degeneracy 138 is shown here; the same stimulus can be produced by (at least) two 195 139 possible combinations of illuminance and reflectance. Prior beliefs about 196 140 the likelihood of these causes can be used to pick the most likely percept. 197 141 198 142 199 143 an infinite number of ways. We have shown two plausible causes 200 144 by assuming the stimulus is the product of (non-negative) illu- 201 145 FIGURE 1 |The Cornsweet illusion and Mach bands. The Cornsweet minance and reflectance profiles. The lower two panels show the 202 146 illusion is the false perception that the peripheral regions of a Cornsweet “true” causes generating stimuli for the Cornsweet illusion. Here, 203 147 stimulus have different reflectance values. The magnitude of the effect the stimulus has a reflectance profile that reproduces the Corn- 204 increases as the contrast of the stimulus increases. At higher levels of 148 sweet stimulus and is illuminated with a uniform illuminant. An 205 contrast, the secondary illusion – Mach bands – appear. The Mach bands 149 alternative explanation for exactly the same stimulus is provided 206 are situated at the point of inflection of the luminance gradient. 150 in the upper two panels, in which two isoreflectant surfaces are 207 151 viewed under a smooth gradient of ambient illumination. In this 208 152 sensory consequences is ill-posed (degenerate or many to one), example, we have ensured that both the illuminant and reflectance 209 153 such that a stimulus can have more than one cause. Thus, from are non-negative by applying an exponential transform before 210 154 the point of view of the observer, there can be no “false” infer- multiplying them to generate the stimulus. 211 155 ence unless the true causes are known. The perceptual inference The key point made by Figure 2 is that there are many pos- 212 156 can be optimal in a Bayesian sense, but is still illusory. However, sible gradients of illuminance and reflectance that can produce 213 157 not all possible causes of sensory input will be equally likely, so the same pattern of sensory input (luminance). These differ- 214 158 there will be an optimal inference in relation to prior beliefs about ent explanations for a particular stimulus can only be distin- 215 159 their causes. Prior beliefs can be learnt or innate: priors that are guished by priors on the spatial and temporal characteristics of 216 160 learnt depend upon experience while innate priors can be asso- the reflectance and illuminance. In this example, the ambiguity 217 161 ciated with architectural features of the visual brain, such as the about what caused the stimulus can be resolved if we believe, 218 162 complex arrangement of blobs, interblobs, and stripes in V1, that a priori, that the visual world is composed of isoreflectant sur- 219 163 may reflect priors on the statistical structure of visual information faces, as opposed to surfaces that (implausibly) get brighter or 220 164 selected by evolutionary pressure. darker nearer their edges or occlusions (as in the lower panels). 221 165 Prior beliefs are essential when resolving ambiguity or the ill- Under this prior assumption, an observer who infers the pres- 222 166 posed nature of perceptual inverse problems. Put simply, there will ence of spatially extensive isoreflectant surfaces, and explains the 223 167 always be an optimal posterior estimate of what caused a sensation edge at the centre with an illuminance gradient, would be infer- 224 168 that rests upon prior beliefs. The example in Figure 2 illustrates ring its most likely cause. The Cornsweet “illusion” is thus only an 225 169 this: The central panel shows an ambiguous stimulus (luminance illusion because the experimenter has chosen an unlikely combi- 226 170 profile) that is formally similar to the sort of stimulus that induces nation of illuminance and reflectance profiles. In what follows, we 227 171 the Cornsweet illusion. However, this stimulus can be caused in will exploit priors on the spatial composition and generation of 228

Frontiers in Psychology | Perception Science February 2012 | Volume 3 | Article 43 | 2 Brown and Friston Free-energy and illusions: the Cornsweet effect

229 visual input to simulate the Cornsweet effect and the emergence to produce sensory inputs. In brief, once we know the agent’s gen- 286 230 of Mach bands. erative model, one can use the free-energy principle to predict its 287 231 The Bayesian approach to has been exploited behavior and perception. In the present context, our focus will 288 232 in previous work (Yuille et al., 1991; Knill and Pouget, 2004). be on perception and the role of prior beliefs that are an inher- 289 233 In addition, several other visual illusions have been explained ent part of the generative model. In what follows, we describe 290 234 using Bayesian principles, including motion illusions (Weiss et al., the model and then use it to simulate perceptual inference and 291 235 2002), the sound-induced flash illusion (Shams et al., 2005), and electrophysiological responses. 292 236 the (Lotto and Purves, 2001). Additionally, Purves 293 237 et al. (1999) demonstrated the Bayesian nature of the Cornsweet THE GENERATIVE MODEL 294 238 illusion: when presented in a context implying an illuminance gra- The generative model we used is straightforward: sensory input 295 239 dient and reflectance step, the Cornsweet illusion is elicited more is the product of reflectance and illuminance, where illuminance 296 240 easily. varies smoothly over space but can fluctuate with a high frequency 297 241 In terms of the neuronal systems mediating the Cornsweet over time. Conversely, the reflectance profile of the visual world is 298 242 illusion; some authors have implicated subcortical structures: for caused by isoreflectant fields or surfaces that fluctuate smoothly 299 243 example, Anderson et al. (2009) found that BOLD signal in the in time. Crucially, the spatial scales over which these fluctua- 300 244 lateral geniculate nuclei (LGN) best correlated with perception tions occur have a scale-free nature, of the sort found in natural 301 245 of the Cornsweet illusion, although correlations were also seen images (Burton and Moorhead, 1987; Field, 1987; Tolhurst et al., 302 246 in . Furthermore, the illusion could be abolished if 1992; Ruderman and Bialek, 1994; Ruderman, 1997). To ensure 303 247 the stimulus was not presented binocularly, suggesting an origin positivity of the illuminant and reflectance we apply an expo- 304 248 before V1. Mach bands similarly have been attributed to retinal nential transform to the two factors before multiplying them (as 305 249 mechanisms (e.g., Ratliff, 1965); however, Lotto et al. (1999) have in Figure 2). Equivalently, we can imagine the underlying causes 306 250 suggested a high-level contextual explanation for their appearance. (reflectance and illuminance) as being composed additively in log- 307 251 Irrespective of the cortical or subcortical systems involved, we will space. This model is shown schematically in Figure 3,interms 308 252 assume, in this paper that the same Bayesian principles operate of hidden causes and states. Mathematically, this model can be 309 253 and, crucially, rest on a hierarchical generative model that neces- expressed as: 310 254 sarily implicates distributed neuronal processing at the subcortical 311 = ( ) = ( · + · ) + ω 255 and cortical levels. s g x, v exp R x I vI s 312 (1) 256 x˙ = f (x, v) = vR − x + ωx 313 257 OVERVIEW 314 258 This paper comprises three sections. The first describes a simple Here, s(t) are sensory signals generated from hidden states 315 259 generative model of visual input that entails prior beliefs about x(t) and causes, v(t) plus some random fluctuations ω(t). The 316 260 how visual stimuli are generated and can be used to infer their difference between hidden causes and states is that states evolve 317 261 causes. This model is used in the second section to simulate dynamically, in response to perturbations by hidden causes – 318 262 the perception of the Cornsweet illusion and contrast-dependent these dynamics are described by the equation of motion in the 319 263 emergence of Mach bands. In the third section, we test the pre- second equality above. Hidden causes v(t) = (vI, vR)havebeen 320 264 dictions of the simulations using a psychophysics study of normal divided into those causing changes in luminance and those causing 321 265 subjects. changes in hidden states that produce changes in reflectance. These 322 266 hidden variables control the amplitude of spatial basis functions 323 267 A GENERATIVE MODEL FOR THE CORNSWEET EFFECT (R, I) encoding formal beliefs about the spatial scales of illumi- 324 268 Our simulations are based upon the free-energy formulation of nance and reflectance. For the illuminant (I) we use a low-order 325 269 Bayes-optimal perception. Put briefly, this is based upon the discrete cosine transform, while for the reflectant (R) we use a 326 270 notion that self-organizing agents minimize the average surprise low-order discrete wavelet transform. 327 271 (entropy) of sensory inputs through minimizing a free-energy The particular wavelet transform used here is a Haar wavelet 328 272 bound on surprise. Here, surprise is just the improbability of set that has been thinned by removing high-order wavelets (with 329 273 sampling some sensory information, in relation to a (generative) high spatial frequency) from the periphery of the visual field: This 330 274 model of how those sensations were produced. By adjusting the respects, roughly, the increasing size of classical receptive fields 331 275 free parameters of the model, the sensory information can be with retinotopic eccentricity. For simplicity (and ease of report- 332 276 explained or predicted and surprise minimized. Mathematically, ing the results), we restrict the simulations to a one-dimensional 333 277 surprise is the (negative) log evidence for a model of the world visual field. Because Haar wavelets afford local linear approxima- 334 278 that comprises hidden variables (causes and states) that generate tions to continuous reflectance profiles, the resulting reflectance 335 279 sensory information.Wehave described in many previous publica- has to be a mixture of isoreflectant surfaces at different spatial 336 280 tions how this principle leads to active inference and Bayes-optimal scales. To impose the scale-free aspect, we decrease the variance 337 281 perception (Friston et al., 2006; Friston, 2009; Feldman and Fris- or, equivalently, increase the precision of the reflectance wavelet 338 282 ton, 2010). Free-energy is a function of sensory samples and a coefficients or hidden states in proportion to the order or spatial 339 283 probabilistic representation of what caused those samples. This scale of their wavelet. This is implemented by placing a prior on 340 −3k 284 representation can be cast in terms of the most likely or expected the wavelet coefficients with the form p(xk) = N (0, e ), where k 341 285 states of the world, under a generative model of how they conspire is the order of the wavelet. Neuronally, these basis functions could 342

www.frontiersin.org February 2012 | Volume 3 | Article 43 | 3 Brown and Friston Free-energy and illusions: the Cornsweet effect

343 400 344 401 345 402 346 403 347 404 348 405 349 406 350 407 351 408 352 409 353 410 354 411 355 412 356 413 357 414 358 415 359 416 360 417 361 418 362 419 363 420 364 421 365 422 366 423 367 FIGURE 3 |The generative model: the generative model employed in this the reflectance basis functions hidden states, which accumulate hidden 424 paper models illuminance as a discrete cosine function and reflectance as a causes to generate changes in reflectance. Inversion of the model provides 368 425 Harr wavelet function with peripheral high frequency wavelets removed. In conditional estimates of the hidden causes and states responsible for 369 426 addition, illumination is allowed to change quickly over time, whereas sensory input as a function of time. See main text for an explanation of the 370 reflectance varies more slowly. This is achieved by making the coefficients on variables in this figure. 427 371 428 372 429 373 stand in for a filling-in process such as that described by Gross- 430 374 berg and Hong (2006). Conversely, the illuminant is modeled as 431 375 a mixture of smoothly varying cosine functions with a low spatial 432 376 frequency. This is easily motivated by the fact that most sources 433 377 of illumination are point sources, which results in smooth illu- 434 378 minance profiles. These were modeled here with the first three 435 379 components of a discrete cosine transform (see Figure 4 for a 436 380 graphical representation of the basis functions and how they are 437 381 used to generate a stimulus). 438 382 By construction, this generative model of visual signals sep- 439 383 arates the spatial scales or frequencies of the illuminance and 440 384 reflectance such that all the high frequency components are in 441 385 the reflectance profile, while the low frequency components are in 442 386 illuminance profile. Temporal persistence of reflectance is assured 443 ∈16 × 1 387 because the reflectance coefficients x(t) are hidden states 444 ∈16 × 1 388 that accumulate hidden causes vR(t) . This persistence 445 389 reflects the prior belief that surfaces move in a continuous fashion. 446 390 For simplicity, we mapped the hidden causes controlling illumi- 447 ∈3 × 1 391 nance vI(t) directly to the stimulus (although this is not an 448 392 important feature of our model). This can be thought of as accom- FIGURE 4 |Two-dimension example. The top panels show 2-D examples 449 of luminance (left) and reflectance (right), created form the generative 393 modating rapid changes in illuminance of the sort that might be 450 model. The resulting stimulus is the product of the two (bottom panel). 394 produced by a flickering candle. 451 395 Equation 1 defines our generative model in terms of the joint 452 396 probability over sensory information and the hidden variables 453 397 producing fluctuations in reflectance and illuminance. The fluc- motion of the hidden states are assumed to have a log-precision 454 398 tuations in the hidden causes are assumed to be Gaussian with a of 12. Finally, we assume sensory fluctuations or noise with a 455 399 precision (inverse variance) of one, while the fluctuations in the log-precision of six. In the next section, we will manipulate the 456

Frontiers in Psychology | Perception Science February 2012 | Volume 3 | Article 43 | 4 Brown and Friston Free-energy and illusions: the Cornsweet effect

457 log-precision of the sensory noise as a proxy for changing the to inferring those variables by mapping back from the sensory 514 458 contrast of the stimulus. consequences to the hidden causes and states. This can be imple- 515 459 Figure 4 shows a snapshot of the sort of visual signals this gen- mented in a biologically plausible fashion using a generalized 516 460 erative model produces. Here, we have used the outer product of gradient descent on variational free-energy F(s˜, μ˜ );whichisa 517   461 the discrete transforms above to generate a two-dimensional stim- function of (generalized) sensory states s˜(t) = (s, s , s , ...)and 518 462 ulus. We are not pretending that this is a veridical model of the the expected values μ(t) = (μx, μv) of hidden variables (see Fris- 519 463 real visual world. However, it is sufficient to explain the Cornsweet ton, 2008 for details). In brief, this gradient descent corresponds 520 464 illusion and related effects by incorporating simple and plausible to a Bayesian filtering, in which expected states of the world are 521 465 priors on the spatial scales over which illuminance and reflectance continuously optimized using a prediction term and an update 522 466 change. In the next section, we use this generative model to sim- term: 523 467 ulate perceptual and physiological responses to a stimulus, under 524 ˙ ∂ ∂ε˜T 468 the free-energy principle. This reduces to a Bayesian deconvolu- μ˜ = Dμ˜ − F(s˜, μ˜ ) = Dμ˜ − ξ (2) 525 469 tion of sensory input that tries to discover the most likely hidden ∂μ˜ ∂μ˜ 526 470 causes and states generating that input. 527 Under the simplifying assumption that probabilities are repre- 471 528 PERCEPTION AND PREDICTIVE CODING sented as Gaussian densities, this can be regarded as a generalized 472 529 This perceptual deconvolution can be regarded as the inversion of form of Kalman filtering, where the second (update or gradient) 473 530 a generative model that maps from hidden causes (variables in the term can be expressed as a mixture of prediction errors (see the 474 531 world) to sensory consequences. The inverse mapping corresponds equations in Figure 5). This means that the generalized filtering 475 532 476 533 477 534 478 535 479 536 480 537 481 538 482 539 483 540 484 541 485 542 486 543 487 544 488 545 489 546 490 547 491 548 492 549 493 550 494 551 495 552 496 553 497 554 498 555 499 FIGURE 5 | Hierarchical message-passing in the brain. The schematic states, and the hidden causes. The ensuing equations suggest two neuronal 556 500 details a neuronal architecture that optimizes the conditional expectations of populations that exchange messages; with state-units encoding conditional 557 501 hidden variables in hierarchical models of sensory input of the sort predictions and error-units encoding prediction error. Under hierarchical 558 illustrated in Figure 3. It shows the putative cells of origin of forward driving models, error-units receive messages from the state-units in the same level 502 559 connections that convey prediction error from a lower area to a higher area and the level above; whereas state-units are driven by error-units in the 503 (red arrows) and non-linear backward connections (black arrows) that same level and the level below. These provide bottom-up messages that 560 504 construct predictions (Mumford, 1992; Friston, 2008). These predictions try drive conditional expectations μ(i):i = x, v toward better predictions to 561 505 to explain away (inhibit) prediction error in lower levels. In this scheme, the explain away prediction error. These top-down predictions correspond to 562 ˜ = ˜ (μ˜ (x) μ˜ (v)) 506 sources of forward and backward connections are superficial and deep g : g , that are specified by the generative model. This scheme 563 pyramidal cells (triangles) respectively, where state-units are black and suggests the only connections that link levels are forward connections 507 564 error-units are red. The equations represent a generalized gradient descent conveying prediction error to state-units and reciprocal backward 508 on free-energy (see main text) using the generative model of the previous connections that mediate predictions. Note that the prediction errors that 565 509 figure. If we assume that synaptic activity encodes the conditional are passed forward are weighted by their precisions: Π(i):i = s, x, v. 566 510 expectation of states, then recognition can be formulated as a gradient Technically, this corresponds to generalized predictive coding because it is a 567 ∼ 511 descent on free-energy. Under Gaussian assumptions, these recognition function of generalized variables, which are denoted by a ( ), such that 568 dynamics can be expressed compactly in terms of precision weighted every variable is represented in generalized coordinates of motion: for 512 569 prediction errors: ξ(i):i = s, x, v on the sensory input, motion of hidden example: x˜ = (x, x , x , ...). See Friston (2008) for further details. 513 570

www.frontiersin.org February 2012 | Volume 3 | Article 43 | 5 Brown and Friston Free-energy and illusions: the Cornsweet effect

571 in Eq. 2 to corresponds to a generalized form of predictive coding. 628 572 Predictive coding has become a popular metaphor for understand- 629 573 ing perceptual inference in the . For example, Rao 630 574 and Ballard (1999) used predictive coding to provide a compelling 631 575 explanation for extraclassical receptive field effects in striate cortex. 632 576 Put simply, in these simulations we assume that neural activ- 633 577 ity corresponds to the brain’s representation of the most likely 634 578 values of the hidden causes and states (hidden variables) and 635 579 that these are continuously updated to minimize free-energy. The 636 580 ensuing scheme has been discussed in terms of recurrent message- 637 581 passing among different cell populations in hierarchical sensory 638 582 cortex: see Figure 5 and Mumford (1992). This scheme rests upon 639 583 the use of bottom-up prediction errors to optimize conditional 640 584 estimates of hidden variables. These estimates are then used to 641 585 produce top-down predictions that are compared with sensory 642 586 input to form a bottom-up prediction error. In this context, the 643 587 sum of squared prediction error can be regarded as free-energy. 644 588 The recursive message-passing used in these schemes tries to min- 645 589 imize prediction error, such that the predictions approximate the 646 590 true conditional or posterior estimates of the underlying hidden 647 591 causes. It is this message-passing that we will stimulate in the 648 592 next section and associate with neuronal responses, while using 649 593 what they represent to predict how real subjects would respond 650 594 behaviorally, in terms of their perceptual decisions or inference. 651 595 To simulate these responses we simply integrate or solve Eq. 2, 652 596 using the functions g(x, v) and f(x, v) specified by a generative FIGURE6|Predictions of the model. (C) Shows the estimates of the 653 597 model in Eq. 1. These functions map hidden variables to sen- hidden states (coefficients of the reflectance basis functions) over time. 654 The hidden state controlling the amplitude of the lowest-frequency basis 598 sory input and encode prior beliefs about the dynamics of hidden 655 function, which corresponds to the Cornsweet percept, contributes 599 states. In short, by plugging the equations of our generative model substantially to the overall perception of the stimulus (green line). The 656 600 in Figure 3 into the predictive coding scheme of Figure 5, we can estimates for the hidden causes are shown in (B). The gray areas are 90% 657 601 simulate Bayes-optimal inference about the causes of sensations. confidence intervals. (A) Shows predictions (solid lines) of sensory input 658 based on the estimated hidden causes and states and the resulting 602 Crucially, we can then reconstitute the posterior or conditional 659 prediction error (dotted red lines). The insert on the upper left shows the 603 beliefs about these causes and associate these with percepts. In 660 time-dependent luminance profile used in this simulation. Please see main 604 particular, we can take any mixture of the hidden variables and text for further details. 661 605 assess the posterior belief about that mixture. We will use this 662 606 to quantify the Cornsweet and Mach band percepts, in terms of 663 607 reflectance differences among different parts of the visual field. it does. The remaining causes that deviate from zero (Figure 6C) 664 608 Note that the predictive coding scheme in Figure 5 weights the are the perturbations to the hidden states explaining or predicting 665 609 prediction errors by precision matrices. For example, the preci- changes in reflectance. These drive increases or decreases in the 666 (s) 610 sion of sensory signals is Π = I·exp(γ). These precisions are conditional expectations of the hidden states shown on the right. 667 611 functions of log-precisions γ that encode the expected amplitude The green line is the coefficient of the second-order basis function 668 612 of random fluctuations. splitting the visual field into an area of brightness on the left and 669 613 darkness on the right. It can be seen that at the point of maximum 670 614 SIMULATED RESPONSES illumination, there is an extremely high degree of confidence that 671 615 The simulated responses in Figure 6 were obtained by presenting this hidden state is bigger than zero. This is the Cornsweet percept. 672 616 the Cornsweet stimulus under uniform illumination. Here, the The corresponding percepts in sensory space are shown in 673 617 stimulus was presented transiently by modulating the illumination Figure 7 as a function of peristimulus time. The upper panels 674 618 with a Gaussian envelope over time (see image inset). The result- show the implicit reflectance and illuminance profiles encoded 675 619 ing predictions are shown in Figure 6A as solid lines, while the red by the conditional expectations of hidden variables respectively. 676 620 dotted lines correspond to the prediction error. These predictions After an exponential transform (and multiplication) these pro- 677 621 are based upon the inferred hidden states and causes shown on the duce the sensory predictions shown on the lower left. By taking a 678 622 right and lower left respectively. The lines correspond to the poste- weighted mixture of the perceived reflectance in different regions 679 623 rior expectations and the gray regions correspond to 90% Bayesian of the visual field (shown by the white circles) one can estimate the 680 624 confidence intervals. In terms of the underlying causes, the blue conditional certainty about both the Cornsweet effect (differences 681 625 curve in Figure 6B is an estimate of the (log) amplitude of uniform in perceived reflectance on different sides) and the appearance 682 626 illumination. This should have a roughly quadratic form (given of Mach bands (differences in perceived reflectance on the same 683 627 the Gaussian envelope), peaking at around bin 30, which indeed side). The weights used to evaluate these mixtures are denoted as 684

Frontiers in Psychology | Perception Science February 2012 | Volume 3 | Article 43 | 6 Brown and Friston Free-energy and illusions: the Cornsweet effect

685 simulations, we changed explicitly. In more realistic simulations, 742 686 the log-precision would be itself optimized with respect to free- 743 687 energy (see Feldman and Friston, 2010 for an example of this in 744 688 the modeling of attention). 745 689 Figure 8 shows the results of perceptual inference under the 746 690 Bayesian scheme described above. The only thing that we changed 747 691 was the log-precision of the sensory input, from minus two (low) 748 692 through to intermediate levels and ending with a very high log- 749 693 precision of 16. The two graphs show the conditional expectation 750 694 and 90% confidence intervals for the Cornsweet effect (upper 751 695 panel) and Mach bands (lower panel) respectively, at the point 752 696 of maximum illumination. It can be seen in both instances that 753 697 under low levels of contrast (sensory precision) both effects are 754 698 very small and inferred with a large degree of uncertainty. How- 755 699 ever, as contrast increases, conditional uncertainty reduces and, 756 700 at a critical level, produces a confident inference that the effect is 757 701 greater than zero (or some small threshold). Crucially, the point 758 702 at which this happens for the Cornsweet effect is at a lower level of 759 703 contrast than for the Mach bands. In other words, the Cornsweet 760 704 illusion occurs first and then the Mach bands appear as contrast 761 705 continues to rise. The explanation for this is straightforward; the 762 706 FIGURE 7 |The model’s “perceptions.”The upper panels show the Mach band illusion rests upon higher spatial frequencies in the 763 predicted illuminance (left) and reflectance profiles (right), reconstructed 707 generative model, which have a higher prior precision (encoding 764 from the coefficients of the basis functions estimated from the model 708 inversion shown in the previous figure. An inferred reflectance profile prior beliefs about the statistical – scale-free – structure of natural 765 709 demonstrating the Cornsweet illusion is apparent, but at this level of visual scenes). This means that there needs to be precise sensory 766 710 contrast, Mach bands have not yet appeared. Please see main text further evidence to change them from their prior expectation of zero. In 767 711 details. short, at high levels of contrast or sensory precision, more and 768 712 more fine detail in the posterior percept is recruited to provide 769 713 the optimum explanation for the stimulus. Interestingly, as the 770 W corn and W mac for the Cornsweet and Mach band effects respec- 714 contrast or sensory precision reaches very high levels, the veridical 771 tively. The conditional expectation of these mixtures or effects 715 (x) reflectance and illuminant profiles are inferred and, quantitatively, 772 μmac = W mac·μ and their confidence intervals are shown on 716 both the Cornsweet and mach band effects disappear. The three 773 the lower right. At this level of visual contrast or precision (a 717 images show exemplar percepts, at low, intermediate and high 774 log-precision of six), the Cornsweet effect is clearly evident with 718 levels of contrast respectively. The key difference in the spatial 775 a high degree of certainty, while the confidence interval for the 719 banding that underlies the Cornsweet and Mach band effects is 776 Mach band effect always contains zero. In other words, at this con- 720 evident in the difference between the intermediate and high levels 777 trast (sensory precision) there is a Cornsweet effect but no Mach 721 of contrast. 778 band effect. In the next section, we repeat the simulation above 722 The key prediction of these simulations is that we would expect 779 and record the conditional expectations (and confidences) about 723 subjects to categorize their percepts, following a brief exposure 780 illusory effects at the point of maximum illumination for different 724 to a Cornsweet stimulus, differently at different levels of contrast. 781 levels of contrast. 725 At low levels of contrast, we would expect them to categorize the 782 726 stimulus as uniformly flat. At intermediate levels of contrast, we 783 727 CONTRAST OR PRECISION-DEPENDENT ILLUSORY would expect them to categorize the stimulus as a Cornsweet per- 784 728 PERCEPTS cept, with isoluminant and uniform differences in the right and 785 729 Using the generative model and inversion scheme described above, left hand parts of the visual field; while at higher levels of contrast 786 730 we repeated the simulations over different levels of sensory pre- one would expect the Mach bands to dominate and the stimuli 787 731 cision. This can be regarded as a manipulation of contrast in would be categorized as possessing para-central bands. In princi- 788 732 the following sense: If we assume that the brain uses divisive ple, at very high levels of contrast, the subject should perceive the 789 733 normalization (Weber, 1846; Fechner, 1860; Craik, 1938; Geisler veridical stimulus. However, whether this level of contrast can be 790 734 and Albrecht, 1992; Carandini and Heeger, 1994), the key change attained empirically is an open question. In the next section, we 791 735 in sensory information, following an increase in contrast, is an test these hypotheses psychophysically. Weconclude this section by 792 736 increase in signal to noise; in other words, its precision increases looking not at behavioral responses but at the neuronal responses 793 737 (see the appendix of Feldman and Friston, 2010 for details). We implicit in the simulations. 794 738 use therefore a manipulation of the log-precision of sensory noise 795 739 to emulate changes in visual contrast. It should be noted that we SIMULATING NEURONAL RESPONSES 796 740 did not actually add sensory noise to the stimuli. The key quantity Figure 9 shows the prediction errors at low, intermediate and high 797 741 here is the level of precision assumed by the agent which, in these levels of contrast. These are shown at the sensory level (upper row) 798

www.frontiersin.org February 2012 | Volume 3 | Article 43 | 7 Brown and Friston Free-energy and illusions: the Cornsweet effect

799 856 800 857 801 858 802 859 803 860 804 861 805 862 806 863 807 864 808 865 809 866 810 867 811 868 812 869 813 870 814 871 815 872 816 873 817 874 818 875 819 876 820 877 821 878 822 879 823 880 824 881 825 882 826 883 827 884 828 885 829 886 830 887 831 888 832 889 833 890 834 891 835 892 836 893 837 894 838 895 839 896 840 897 841 898 842 899 843 900 844 901 845 902 846 903 847 FIGURE 8 |The effect of contrast. In the results presented here the appear at higher levels of contrast than the Cornsweet illusion. The inserts in 904 848 precision of the sensory data was used as a proxy for stimulus contrast. As the lower panels show the inferred reflectance is at different levels of contrast 905 precision increases, the strength of both the Cornsweet and Mach effects (indicated by the blue dots in the lower graph). The prediction errors 849 906 increases until, at very high levels of precision (contrast) the true luminance associated with the processing of stimuli at these three levels are shown in 850 profile is perceived and the illusory percepts fade. Crucially, the Mach bands the next figure. Please see main text for further details. 907 851 908 852 909 853 and at the higher levels of the hidden causes and states (lower row). prediction error falls. This is at the expense of inducing predic- 910 854 The key thing to note here is that as contrast increases and the tion errors at the higher level, which increase in proportion to the 911 855 spatial detail of the posterior predictions increases, the sensory precision of sensory information. These higher prediction errors 912

Frontiers in Psychology | Perception Science February 2012 | Volume 3 | Article 43 | 8 Brown and Friston Free-energy and illusions: the Cornsweet effect

913 A PSYCHOPHYSICAL TEST OF THEORETICAL PREDICTIONS 970 914 In this section, we report a psychophysics study of normal subjects 971 915 exposed to the same stimuli used in the simulations above. We 972 916 depart from the normal procedures for assessing illusions (which 973 917 usually involve matching intensity differences) by using a forced 974 918 choice paradigm. This is because we wanted to present stimuli 975 919 briefly, for several reasons: First, brief presentation avoids the con- 976 920 founding effects of saccadic eye movements. Second, it allows us 977 921 to prototype the paradigm for future use in electrophysiological 978 922 (event related potential) studies, which require transient stimuli 979 923 for trial averaging. Finally, a forced choice paradigm places con- 980 924 straints on the subject’s choices that map directly to the model 981 925 predictions. In what follows, we describe the paradigm and inter- 982 926 pret our results quantitatively, in relation to the predictions of the 983 927 simulations above. 984 928 985 929 SUBJECTS AND EXPERIMENTAL PARADIGM 986 930 We studied normal young subjects in accord with guidelines estab- 987 931 lished by the local ethical committee and after obtaining informed 988 932 consent. Eight participants (4 female) completed the Mach band 989 933 paradigm; 19 (12 female) completed the Cornsweet paradigm. 990 934 991 935 EXPERIMENT 1 (CORNSWEET PARADIGM) 992 FIGURE 9 | Contrast and prediction error. As stimulus contrast increases, 936 prediction error is redistributed from sensory input to hidden variables. At The Cornsweet illusion was assessed using a two-interval forced 993 937 high levels of precision, sensory information induces prediction errors at choice procedure. Subjects were presented with a (set contrast) 994 938 higher levels, which in turn explain away prediction error at the sensory Cornsweet stimulus and real luminance step for 200 ms (a Gauss- 995 level. The higher level prediction errors at high precision reflect increasing 939 ian temporal envelope was not used), separated by an interval of 996 confidence that the reflectance is different from the prior expectation of 940 zero. Please see main text for further details. 200 ms. One stimulus appeared to the left of fixation and one to 997 941 the right; this was randomized across trials, as was the order of 998 942 the stimuli. Subjects were asked to report the side on which the 999 943 are simply the difference between the posterior and prior expecta- stimulus with the greatest contrast had appeared (Figure 10). 1000 944 tions and reflect an increasing departure from a prior expectation Six blocks were completed,using Cornsweet stimuli with Weber 1001 945 of zero as contrast (the log-precision of sensory noise) increases. contrasts of 0.0073–0.734. A Quest procedure (Watson and Pelli, 1002 946 Although these results are interesting in themselves, they can also 1983) was used to select each step stimulus for comparison. The 1003 947 be regarded as a simulation of event related potentials. The reason mean of the psychometric function was taken as the point of 1004 948 that we can associate prediction error with observed electromag- subjective equality between the Cornsweet stimulus and a real 1005 949 netic brain responses is that it is usually assumed that prediction luminance. There were 200 presentations per block. 1006 950 errors are encoded by the activity of superficial pyramidal cells (see 1007 951 Figure 5). It is these cells that are thought to contribute primarily EXPERIMENT 2 (MACH BAND PARADIGM) 1008 952 to local field potentials and non-invasive EEG signals. The same method could not be used to identify the strength of the 1009 953 In the high-contrast condition, the prediction errors at the Mach bands percept, as there is no non-illusory stimulus that can 1010 954 lower level are suppressed by the prediction of the presence of be used for matching. Consequently, we used a two-alternative 1011 955 a Cornsweet stimulus. This sort of phenomenon has been demon- forced choice paradigm: A single Cornsweet stimulus was dis- 1012 956 strated using fMRI (Alink et al., 2010; den Ouden et al., 2010); played for 200 ms to the left or right of fixation and participants 1013 957 predictable stimuli cause less activation in stimulus-specific areas were asked to report if the stimulus contained Mach bands or not. 1014 958 than unpredictable stimuli. However, the process simulated here Each participant completed six runs of 200 presentations at 10 1015 959 is likely to produce more complicated neurophysiological corre- Weber contrast levels from 0.0204 to 0.2038. The probability of 1016 960 lates because of the confounding effect of precision; increased reporting a Mach band was assessed as the relative frequency of 1017 961 predictability (through increasing conditional confidence about reporting its presence over trials, within subject (Figure 10). 1018 962 the stimulus) will also increase estimates of precision. Since we In both experiments, stimuli were displayed on an LCD mon- 1019 963 believe that the prediction errors reported by superficial pyra- itor under ambient room lighting. Subjects were seated 60 cm 1020 964 midal cells are precision weighted, decreasing prediction error away from the monitor, such that stimuli subtended an angle of 1021 965 in lower sensory areas may be masked by the increasing preci- 14.21˚ vertically and 6.10˚ horizontally, at 2.96˚–7.06˚ eccentricity. 1022 966 sion of those errors. We will return to this and related issues The luminance ramp of the Cornsweet stimulus profile occupied 1023 967 in a subsequent paper looking at the neurophysiological corre- 2.42˚. Only the lowest levels of contrast the monitor was able to 1024 968 lates of contrast-dependent illusory effects. Here, we focus on produce were employed; thus, luminance values were linearized 1025 969 psychophysical correlates: post hoc. 1026

www.frontiersin.org February 2012 | Volume 3 | Article 43 | 9 Brown and Friston Free-energy and illusions: the Cornsweet effect

1027 RESULTS AND DISCUSSION SE. The reported Cornsweet effect (as indexed by the point of sub- 1084 1028 The results of the psychophysics experiments are shown in jective equality) peaked, on average over subjects, at a contrast of 1085 1029 Figure 11,as a function of empirical (Weber)contrast levels. These about 0.0025. At higher levels of contrast, as in the simulations, 1086 1030 results are expressed as the mean over all subjects and associated the effect fell quantitatively, plateauing at the highest contrast 1087 1031 used in Experiment 1. Conversely, the probability of reporting a 1088 1032 Mach band increased monotonically as a function of the empirical 1089 1033 contrast, reaching about 75% at a Weber contrast of about 0.15. 1090 1034 Qualitatively, these empirical results compare well with the the- 1091 1035 oretical predictions shown in Figure 9: that is, the subjective or 1092 1036 inferred Cornsweet effect emerged before the Mach bands, as con- 1093 1037 trast increases. We also see the characteristic “inverted U” depen- 1094 1038 dency of the Cornsweet effect on contrast levels. The empirical 1095 1039 profile is somewhat compromised by the small range of contrasts 1096 1040 employed, however, this range was sufficient to disclose an unam- 1097 1041 biguous peak. Clearly, it would be nice to relate these empirical 1098 1042 results quantitatively to the simulations shown in Figure 9. This 1099 1043 presents an interesting challenge because the psychophysical data 1100 1044 consist of reported levels of an effect and the probability of an 1101 1045 effect for the Cornsweet and Mach Band illusions respectively. 1102 1046 However, because our simulations provide a conditional or pos- 1103 1047 terior probability over the effects reported, we can simulate both 1104 1048 sorts of reports and see how well they explain the psychophysical 1105 1049 FIGURE 10 |Time courses of trials. Experiment 1: before the start of each data. This quantitative analysis is now considered in more detail. 1106 1050 trial, participants fixated a central cross. One Cornsweet and one real 1107 1051 luminance step stimulus appeared for 200 ms each, with a 200 ms interval A FORMAL BEHAVIORAL ANALYSIS 1108 between them. The order of the stimuli, their orientation and the side on 1052 1109 which each appeared were randomized (although they were constrained to The simulations provide conditional expectations (and precisions) 1053 appear on opposite sides within each trial). Participants then had 1750 ms of both the Cornsweet and Mach band effects over a number of 1110 1054 to report the side on which the stimulus with the greatest contrast had simulated (Weber) contrast levels, as modeled with the precision 1111 1055 appeared (using the arrow keys of the keyboard). Experiment 2: each trial of sensory noise. This means that one can compute a psychome- 1112 stared with fixation. A Cornsweet stimulus then appeared for 200 ms with a 1056 tric function of contrast c that returns the behavioral predictions 1113 random orientation to the left or right of fixation. Participants had 1750 ms 1057 1114 to report, with the “Y” and “N” keys, if they perceived Mach bands. of both illusions respectively; namely, the level of the illusion and 1058 the probability of inferring a Mach band. To predict the reported 1115 1059 1116 1060 1117 1061 1118 1062 1119 1063 1120 1064 1121 1065 1122 1066 1123 1067 1124 1068 1125 1069 1126 1070 1127 1071 1128 1072 1129 1073 1130 1074 1131 1075 1132 1076 1133 1077 1134

1078 FIGURE 11 | experimental results. This figure shows the results of the quantified in terms of probability that the illusion is reported to be 1135 1079 empirical psychophysical study for the Cornsweet illusion (left panel) and present. Both results are shown as functions of empirical (Weber) 1136 1080 the Mach band illusion (Right panel). Both report the average effect over stimulus contrast levels. We have used the same range for both graphs 1137 1081 subjects and the SE (bars). The Cornsweet illusion is measured in terms so that the dependency on contrast levels can be compared. The key 1138 of the subjective contrast level (quantified in terms of subjective thing to note here is that the Cornsweet illusion peaks before the Mach 1082 1139 equivalence using psychometric functions). The Mach band illusion is band illusion (vertical line). 1083 1140

Frontiers in Psychology | Perception Science February 2012 | Volume 3 | Article 43 | 10 Brown and Friston Free-energy and illusions: the Cornsweet effect

1141 level of the Cornsweet illusion we can simply use the conditional single (unknown) slope coefficient β5. These relationships provide 1198 1142 expectation μcorn(c) scaled by some (unknown) coefficient β1.To a mapping between the results of the simulations and the observed 1199 1143 predict the probability of reporting the presence of Mach bands, responses averaged over subjects (under the assumptions of addi- 1200 1144 one can integrate the conditional probability distribution over the tive prediction errors). This is known as a response model and is 1201 1145 Mach band effect above some (unknown) threshold β2.However, detailed schematically in Figure 11. The predictions are based on 1202 1146 to do this, we need to know the relationship between the simulated the simulated responses in Figure 9, assuming a smooth psycho- 1203 1147 and empirical contrasts: metric function of contrast that was modeled as a linear mixture of 1204 1148 As noted above, we used the log-precision of sensory noise cosine functions: Xk(γ) = cos(πγk) for k = 1, ..., 6). The coeffi- 1205 1149 γ to model log-contrast in accord with Weber’s law. This means cients of this discrete cosine set were estimated with ordinary least 1206 1150 we can assume a linear relationship between the empirical log- squares, using the responses of the model (μcorn(γ), μmac(γ)) over 1207 1151 contrast and simulated log-precision. This induces two further different precision levels, at the time of maximum luminance. 1208 1152 unknown coefficients – the slope and intercept (β3 and β4) that Given the form of the relationships between the simulated 1209 1153 parameterise the relationship between the simulated and empiri- and empirical contrasts and between the report probability and 1210 1154 cal contrasts. Finally, we need to relate the conditional probability conditional probabilities for Mach bands, one can use the psy- 1211 1155 of a suprathreshold Mach band effect to the probability of report- chophysical data to estimate the unknown coefficients of these 1212 1156 ing its presence. Here, we assumed a simple, monotonic sigmoid relationships: βi for = 1,..., 5. The results of this computation- 1213 1157 relationship, under the constraint that when the conditional prob- ally informed response modeling are shown in the right panel of 1214 1158 ability was 50:50, the report probability was also 50:50. The precise Figure 12. The upper panels show the same data as in Figure 10 1215 1159 form of this mapping is provided in Figure 11 (left panel) and has a placed over the theoretical psychometric functions based on the 1216 1160 1217 1161 1218 1162 1219 1163 1220 1164 1221 1165 1222 1166 1223 1167 1224 1168 1225 1169 1226 1170 1227 1171 1228 1172 1229 1173 1230 1174 1231 1175 1232 1176 1233 1177 1234 1178 1235 1179 1236 1180 1237 1181 1238 1182 1239 1183 1240 1184 1241 1185 1242 1186 1243 FIGURE 12 | Predicted and experimental results. Theoretical predictions band P(c), the coefficients βi can be estimated under the assumption of 1187 of the empirical results reported in the previous figure: These predictions are additive prediction errors ε. The predicted responses following this 1244 1188 based upon a response model that maps from the conditional expectations estimation are shown in the graphs on the right hand side. The upper panels 1245 1189 and precisions in the simulations to the behavioural responses of subjects. show the empirical data superimposed upon conditional predictions from 1246 β = μ 1190 This mapping rests on some unknown parameters or coefficients i:i 1, the model. The gray lines are the predicted psychometric functions, corn(c) 1247 ..., 5 that control the relationship between the simulated γ(c) and empirical and σ(p (c)). The red dots correspond to the predictions at levels of 1191 mac 1248 contrast levels c used for stimuli and the relationship between the contrast used in the simulations (as shown in Figure 9), while the black dots 1192 1249 probabilities of reporting a Mach band σ(pmac(c)) and the conditional correspond to the empirical responses: M(c) and P(c). The lower left panel

1193 probability that it exceeds some threshold pmac(c) at contrast level c. These show the relationship between the empirical and simulated contrast levels 1250 1194 relationships form the basis of a response model, whose equations are (as a semi-log plot of the empirical contrast against the log-precision of 1251 1195 provided in the left panel (for simplicity, we have omitted the expressions for sensory noise). The lower right panel shows the relationship between the 1252 conditional variance of the Mach band contrast). Given empirical responses probability of reporting a Mach band is present and the underlying 1196 1253 for the mean Cornsweet effect M(c) and probability of reporting a Mach conditional probability that it is above threshold. 1197 1254

www.frontiersin.org February 2012 | Volume 3 | Article 43 | 11 Brown and Friston Free-energy and illusions: the Cornsweet effect

1255 simulations of the previous section. These predictions are based simulations. This means that it may be difficult to demonstrate the 1312 1256 on the mapping from simulated to empirical contrast levels (lower “inverted U” behavior for the Cornsweet illusion seen in Figure 9. 1313 1257 left) and the relationship between the probability of reporting a This is because it may be difficult to present stimuli at the ultra 1314 1258 Mach band and the conditional confidence that it is present (lower high levels of contrast required. Note that the empirical proba- 1315 1259 right). bility of reporting a Mach band does not decrease as a function 1316 1260 By construction, the relationship between the simulated and of contrast level. This is to be anticipated from the theoretical 1317 1261 empirical contrasts is linear when plotted on a log-log scale. The predictions: increasing the contrast level increases the conditional 1318 1262 slope of this plot suggests that the higher contrasts used in the sim- precision about the inferred level of the Mach band effect, which 1319 1263 ulations are, practically, not realizable in an empirical setting. This means that the probability that is above threshold can still increase 1320 1264 is because as the simulated contrast increases the corresponding even if the conditional expectation decreases (as in Figure 9). 1321 1265 empirical contrast increases much more quickly. The implication 1322 1266 of this is that the contrasts employed in the psychophysics study CONCLUSION 1323 1267 correspond to the first few levels of the simulated contrasts. This This paper has reviewed the nature of illusions, in the context 1324 1268 means that it may be difficult to demonstrate the theoretically pre- of Bayes-optimal perception. We reiterate the notion that illusory 1325 1269 dicted attenuation of the Cornsweet illusion at very high levels of percepts are optimal in that they may represent the most likely 1326 1270 contrast. explanation for ambiguous sensory input. We have illustrated this 1327 1271 The relationship between the report and conditional probabil- using the Cornsweet illusion. By using simple and plausible prior 1328 1272 ities suggests that subjects have a tendency to “all or nothing” expectations about the spatial deployment of illuminance and 1329 1273 reporting; in the sense that a conditional confidence that the reflectance,we have shown show that the Cornsweet effect emerges 1330 1274 probability of reporting a Mach band is slightly greater than the as a natural consequence of Bayes-optimal perception. Further- 1331 1275 conditional confidence it is above threshold. Conversely, subjects more, we were able to simulate a contrast-dependent emergence 1332 1276 appear to report the absence of Mach bands with a probabil- of the Cornsweet effect and subsequent appearance of Mach bands 1333 1277 ity that is slightly greater than the conditional probability it is that was verified psychophysically using a forced choice paradigm. 1334 1278 below threshold. The resulting psychometric predictions (in the 1335 1279 upper panels) show a remarkable agreement between the predicted SOFTWARE NOTE 1336 1280 and observed probabilities of reporting a Mach band. The corre- The simulations and graphics presented in this paper can be 1337 1281 spondence between the predicted and empirical results for the reproduced with the DEM toolbox distributed with the academic 1338 1282 Cornsweet illusion are less convincing but show that both asymp- freeware SPM from http://www.fil.ion.ucl.ac.uk/spm/. The anno- 1339 1283 tote to a peak level much more quickly than the probability of tated files that implement the Cornsweet illusion simulations and 1340 1284 reporting a Mach band. the more general routines used for model inversion are provided 1341 1285 In summary, this analysis suggests that there is a reasonable as Matlab code. 1342 1286 quantitative agreement between the theoretical predictions and 1343 1287 empirical results. Furthermore, in practical terms, it appears that ACKNOWLEDGMENTS 1344 1288 the normal range of Weber contrasts that can be usefully employed This work was funded by the Wellcome Trust. We thank Marcia 1345 1289 corresponds to a relatively low level of sensory precision in the Bennett for helping prepare this manuscript. 1346 1290 1347 1291 1348 1292 REFERENCES Cornsweet, T. (1970). Visual Perception. Feldman, H., and Friston, K. J. (2010). Grossberg, S., and Hong, S. (2006). 1349 1293 Alink, A., Schwiedrzik, C. M., Kohler, New York: Academic Press. Attention, uncertainty, and free- A neural model of surface per- 1350 A., Singer, W., and Muckli, L. (2010). Craik, K. J. (1938). The effect of adap- energy. Front. Hum. Neurosci. 4:215. ception: lightness, anchoring, 1294 1351 Stimulus predictability reduces tation on differential brightness dis- doi:10.3389/fnhum.2010.00215 and filling-in. Spat. Vis. 19, 1295 responses in primary visual cortex. crimination. J. Physiol. (Lond.) 92, Field,D. J. (1987). Relations between the 263–321. 1352 1296 J. Neurosci. 30, 2960–2966. 406–421. statistics of natural images and the Knill, D. C., and Pouget, A. (2004). 1353 1297 Anderson, E. J., Dakin, S. C., and Craik, K. J. W. (1966). The Nature response properties of cortical cells. The Bayesian brain: the role of 1354 1298 Rees, G. (2009). Monocular sig- of Psychology, ed. S. L. Sherwood. J. Opt. Soc. Am. A 12, 2379–2394. uncertainty in neural coding and 1355 nals in human lateral genicu- (Cambridge: Cambridge University Friston, K. (2008). Hierarchical mod- computation. Trends Neurosci. 27, 1299 1356 late nucleus reflect the Craik- Press). els in the brain. PLoS Comput. 712–719. 1300 Cornsweet-O’Brien effect. J. Vis. 9, Dayan, P., Hinton, G. E., and Neal, R. Biol. 4, e1000211. doi:10.1371/jour- Lotto, R. B., and Purves, D. (2001). 1357 1301 1–18. M. (1995). The Helmholtz machine. nal.pcbi.1000211 An empirical explanation of the 1358 1302 Barlow, H. B. (1974). Inductive infer- Neural Comput. 7, 889–904. Friston, K. (2009). The free-energy Chubb illusion. J. Cogn. Neurosci. 13, 1359 ence, coding, perception, and lan- den Ouden, H. E., Daunizeau, J., Roiser, principle: a rough guide to the brain? 547–555. 1303 1360 guage. Perception 3, 123–134. J., Friston, K. J., and Stephan, K. Trends Cogn Sci. (Regul. Ed.) 13, Lotto, R. B., Williams, S. M., and 1304 1361 Burton, G. J., and Moorhead, I. R. E. (2010). Striatal prediction error 293–301. Purves, D. (1999). Mach bands 1305 (1987). Color and spatial structure modulates cortical coupling. J. Neu- Friston, K., Kilner, J., and Harrison, L. as empirically derived associations. 1362 1306 in natural scenes. Appl. Opt. 26, rosci. 30, 3210–3219. (2006). A free energy principle for Proc. Natl. Acad. Sci. U.S.A. 96, 1363 1307 157–170. Fechner, G. T. (1860). “Specielles zur the brain. J. Physiol. Paris 100,70–87. 5245–5250. 1364 Carandini, M., and Heeger, D. J. (1994). Methode richtiger und falscher Fälle, Geisler, W. S., and Albrecht, D. G. Mach, E. (1865). Über die Wirkung 1308 1365 Summation and division by neurons in Anwendung auf die Gewichtsver- (1992). Cortical neurons: isolation der raumlichen Verteilung des 1309 in primate visual cortex. Science 264, suche,” in Elemente der Psychophysik of contrast gain control. Vision Res. Lichtreizes auf die Netzhaut. 1366 1310 1333–1336. (Leipzig: Breitkopf and Härtel). 32, 1409–1410. Sitzungsber. Heidelb. Akad. 1367 1311 1368

Frontiers in Psychology | Perception Science February 2012 | Volume 3 | Article 43 | 12 Brown and Friston Free-energy and illusions: the Cornsweet effect

1369 Wiss. Math. Naturwiss. Kl. 52, the Retina. San Francisco: Holden- Watson, A. B., and Pelli, D. G. could be construed as a potential con- 1426 1370 303–322. Day. (1983). QUEST: a Bayesian flict of interest. 1427 Mumford, D. (1992). On the com- Ruderman, D. L. (1997). Origins of scal- adaptive psychometric method. 1371 1428 putational architecture of the neo- ing in natural images. Vision Res. 37, Percept. Psychophys. 33, Received: 30 November 2011; paper 1372 cortex. II. The role of cortico- 3385–3398. 113–120. pending published: 21 December 2011; 1429 1373 cortical loops. Biol. Cybern. 66, Ruderman, D. L., and Bialek, W. (1994). Weber, E. (1846). “Der Tastsinn and das accepted: 07 February 2012; published 1430 1374 241–251. Statistics of natural images: Scaling Gemeingefuhl,” in Handwörterbuch online: xx February 2012. 1431 1375 O’Brien,V.(1959). Contrast by contour- in the woods. Phys.Rev.Lett.73, der Physiologie, ed. R. Wagner Citation: Brown H and Friston KJ (2012) 1432 enhancement. Am.J.Psychol.72, 814–817. (Leipzig: Wilhelm Engelmann). Free-energy and illusions: the Corn- 1376 1433 299–300. Shams, L., Ma, W. J., and Beierholm, U. Weiss, Y., Simoncelli, E. P., and Adel- sweet effect. Front. Psychology 3:43. doi: 1377 Purves, D., Shimpi, A., and Lotto, R. B. (2005). Sound-induced flash illusion son, E. H. (2002). Motion illusions 10.3389/fpsyg.2012.00043 1434 1378 (1999). An empirical explanation of as an optimal percept. Neuroreport as optimal percepts. Nat. Neurosci. This article was submitted to Frontiers in 1435 1379 the Cornsweet effect. J. Neurosci. 19, 16, 1923–1927. 5, 598–604. Perception Science,a specialty of Frontiers 1436 1380 8542–8551. Tolhurst, D. J., Tadmor, Y., and Chao, T. Yuille, A., Geiger, D., and Bulthoff, in Psychology. 1437 Rao, R. P., and Ballard, D. H. (1999). (1992). Amplitude spectra of natural H. (1991). Stereo integration, mean Copyright © 2012 Brown and Friston. 1381 1438 Predictive coding in the visual images. Ophthalmic Physiol. Opt. 12, field theory and psychophysics. Net- This is an open-access article distributed 1382 cortex: a functional interpretation 229–232. work 2, 423–442. under the terms of the Creative Commons 1439 1383 of some extra-classical receptive- von Helmholtz, H. (1866). Treatise on Attribution Non Commercial License, 1440 1384 field effects. Nat. Neurosci. 2, Physiological Optics, Vol. III, 3rd Edn Conflict of Interest Statement: The which permits non-commercial use, dis- 1441 79–87. (trans. by J. P. C. Southall 1925 Opt. authors declare that the research was tribution, and reproduction in other 1385 1442 Ratliff, F. (1965). Mach Bands: Quanti- Soc. Am. Section 26). New York: conducted in the absence of any com- forums, provided the original authors and 1386 1443 tative Studies on Neural Networks in Dover. mercial or financial relationships that source are credited. 1387 1444 1388 1445 1389 1446 1390 1447 1391 1448 1392 1449 1393 1450 1394 1451 1395 1452 1396 1453 1397 1454 1398 1455 1399 1456 1400 1457 1401 1458 1402 1459 1403 1460 1404 1461 1405 1462 1406 1463 1407 1464 1408 1465 1409 1466 1410 1467 1411 1468 1412 1469 1413 1470 1414 1471 1415 1472 1416 1473 1417 1474 1418 1475 1419 1476 1420 1477 1421 1478 1422 1479 1423 1480 1424 1481 1425 1482

www.frontiersin.org February 2012 | Volume 3 | Article 43 | 13