Progress in Neurobiology 68 (2002) 113–143 Functional integration and inference in the brain Karl Friston∗ The Wellcome Department of Imaging Neuroscience, Institute of Neurology, University College London, 12 Queen Square, London WC1N 3BG, UK Received 17 October 2001; accepted 16 August 2002 Abstract Self-supervised models of how the brain represents and categorises the causes of its sensory input can be divided into two classes: those that minimise the mutual information (i.e. redundancy) among evoked responses and those that minimise the prediction error. Although these models have similar goals, the way they are attained, and the functional architectures employed, can be fundamentally different. This review describes the two classes of models and their implications for the functional anatomy of sensory cortical hierarchies in the brain. We then consider how empirical evidence can be used to disambiguate between architectures that are sufficient for perceptual learning and synthesis. Most models of representational learning require prior assumptions about the distribution of sensory causes. Using the notion of empirical Bayes, we show that these assumptions are not necessary and that priors can be learned in a hierarchical context. Furthermore, we try to show that learning can be implemented in a biologically plausible way. The main point made in this review is that backward connections, mediating internal or generative models of how sensory inputs are caused, are essential if the process generating inputs cannot be inverted. Because these processes are dynamical in nature, sensory inputs correspond to a non-invertible nonlinear convolution of causes. This enforces an explicit parameterisation of generative models (i.e. backward connections) to enable approximate recognition and suggests that feedforward architectures, on their own, are not sufficient. Moreover, nonlinearities in generative models, that induce a dependence on backward connections, require these connections to be modulatory; so that estimated causes in higher cortical levels can interact to predict responses in lower levels. This is important in relation to functional asymmetries in forward and backward connections that have been demonstrated empirically. To ascertain whether backward influences are expressed functionally requires measurements of functional integration among brain systems. This review summarises approaches to integration in terms of effective connectivity and proceeds to address the question posed by the theoretical considerations above. In short, it will be shown that functional neuroimaging can be used to test for interactions between bottom–up and top–down inputs to an area. The conclusion of these studies points toward the prevalence of top–down influences and the plausibility of generative models of sensory brain function. © 2002 Elsevier Science Ltd. All rights reserved. Contents 1. Introduction ................................................................................. 114 2. Functional specialisation and integration ....................................................... 115 2.1. Background ............................................................................. 115 2.2. Functional specialisation and segregation .................................................. 115 2.3. The anatomy and physiology of cortico-cortical connections................................. 115 2.4. Functional integration and effective connectivity............................................ 117 3. Representational learning ..................................................................... 117 3.1. The nature of representations ............................................................. 117 3.2. Supervised models ....................................................................... 120 3.2.1. Category specificity and connectionism .............................................. 120 3.2.2. Implementation .................................................................... 121 3.3. Information theoretic approaches .......................................................... 122 3.3.1. Efficiency, redundancy and information.............................................. 122 3.3.2. Implementation .................................................................... 123 ∗ Tel.: +44-207-833-7454; fax: +44-207-813-1445. E-mail address: k.friston@fil.ion.ucl.ac.uk (K. Friston). 0301-0082/02/$ – see front matter © 2002 Elsevier Science Ltd. All rights reserved. PII: S0301-0082(02)00076-X 114 K. Friston / Progress in Neurobiology 68 (2002) 113–143 3.4. Predictive coding and the inverse problem ................................................. 123 3.4.1. Implementation .................................................................... 124 3.4.2. Predictive coding and Bayesian inference ............................................ 125 3.5. Cortical hierarchies and empirical Bayes................................................... 125 3.5.1. Empirical Bayes in the brain ....................................................... 127 3.6. Generative models and representational learning............................................ 128 3.6.1. Density estimation and EM......................................................... 129 3.6.2. Supervised representational learning................................................. 130 3.6.3. Information theory ................................................................. 130 3.6.4. Predictive coding .................................................................. 131 3.7. Summary ............................................................................... 131 4. Generative models and the brain .............................................................. 132 4.1. Context, causes and representations ....................................................... 133 4.2. Neuronal responses and representations .................................................... 133 4.2.1. Examples from electrophysiology ................................................... 134 5. Functional architectures assessed with brain imaging ............................................ 134 5.1. Context-sensitive specialisation ........................................................... 135 5.1.1. Categorical designs ................................................................ 135 5.1.2. Multifactorial designs .............................................................. 135 5.1.3. Psychophysiological interactions .................................................... 136 5.2. Effective connectivity .................................................................... 136 5.2.1. Effective connectivity and Volterra kernels ........................................... 137 5.2.2. Nonlinear coupling among brain areas............................................... 139 6. Functional integration and neuropsychology .................................................... 139 6.1. Dynamic diaschisis ...................................................................... 139 6.1.1. An empirical demonstration ........................................................ 140 7. Conclusion .................................................................................. 141 Acknowledgements .............................................................................. 141 References...................................................................................... 141 1. Introduction The key focus of this section is on the functional architec- tures implied by each model of representational learning. In concert with the growing interest in contextual and Information theory can, in principle, proceed using only extra-classical receptive field effects in electrophysiology forward connections. However, it turns out that this is only (i.e. how the receptive fields of sensory neurons change ac- possible when processes generating sensory inputs are in- cording to the context a stimulus is presented in), a sim- vertible and independent. Invertibility is precluded when ilar paradigm shift is emerging in imaging neuroscience. the cause of a percept and the context in which it is en- Namely, the appreciation that functional specialisation ex- gendered interact. These interactions create a problem of hibits similar extra-classical phenomena in which a cortical contextual invariance that can only be solved using internal area may be specialised for one thing in one context but or generative models. Contextual invariance is necessary something else in another. These extra-classical phenom- for categorisation of sensory input (e.g. category-specific ena have implications for theoretical ideas about how the responses) and represents a fundamental problem in per- brain might work. This review uses the relationship among ceptual synthesis. Generative models based on predictive theoretical models of representational learning as a vehicle coding solve this problem with hierarchies of backward and to illustrate how imaging can be used to address important lateral projections that prevail in the real brain. In short, questions about functional brain architectures. generative models of representational learning are a natural We start by reviewing two fundamental principles of choice for understanding real functional architectures and, brain organisation, namely functional specialisation and critically, confer a necessary role on backward connections. functional integration and how they rest upon the anatomy Empirical evidence, from electrophysiological studies and physiology of cortico-cortical connections