Active Inference: Building a New Bridge Between Control Theory and Embodied Cognitive Science
Total Page:16
File Type:pdf, Size:1020Kb
A University of Sussex PhD thesis Available online via Sussex Research Online: http://sro.sussex.ac.uk/ This thesis is protected by copyright which belongs to the author. This thesis cannot be reproduced or quoted extensively from without first obtaining permission in writing from the Author The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the Author When referring to this work, full bibliographic details including the author, title, awarding institution and date of the thesis must be given Please visit Sussex Research Online for more information and further details Active inference: building a new bridge between control theory and embodied cognitive science Manuel BALTIERI A thesis submitted in fulfilment of the requirements for the degree of Doctor of Philosophy in the School of Engineering and Informatics University of Sussex January, 2019 i Declaration of Authorship I hereby declare that this thesis has not been and will not be, submitted in whole or in part to another University for the award of any other degree. Signed: Date: ii “[T]he rule “collect truth for truth’s sake” may be justified when the truth is unchanging; but when the system is not completely isolated from its surroundings, and is undergoing secular changes, the collection of truth is futile, for it will not keep. ” Ashby (1958) iii Abstract The application of Bayesian techniques to the study and computational modelling of biological systems is one of the most remarkable advances in the natural and cognitive sciences over the last 50 years. More recently, it has been proposed that Bayesian frameworks are not only useful for building descriptive models of bio- logical functions, but that living systems themselves can be seen as Bayesian (in- ference) machines. On this view, the statistical tools more traditionally used to ac- count for data in biology, neuroscience and psychology, are now used to model the mechanisms underlying functions and properties of living systems as if the systems themselves were the ones “calculating” those probabilities following Bayesian infer- ence schemes. The free energy principle (FEP) is a framework proposed in light of this paradigm shift, advocating the minimisation of variational free energy, a proxy for sensory surprisal, as a general computational principle for biological systems. More intuitively and under some simplifying assumptions, the minimisation of vari- ational free energy reduces, for an agent, to the minimisation of prediction errors on sensory input. Initially proposed as a candidate unifying theory of brain function- ing, the FEP was later extended to encompass hypotheses on the origins of life, and is nowadays discussed in the cognitive science community for its possible implica- tions for theories of the mind. In particular, one of the most popular process theories derived from the FEP, active inference, describes a biologically plausible algorithmic implementation of this principle with several repercussions on our understanding of cognition. In this thesis, I will focus on the role of this process theory for action and per- ception. In active inference, the two of them are combined in a closed sensorimotor loop as co-dependent processes of minimisation of a single loss function, variational free energy, with respect to different sets of variables. Building on this, I will sug- gest that some of the core ideas of active inference are best seen in terms of enactive, embodied, extended and embedded (4E) theories, in contrast to the majority of the literature emphasising its apparent connections to more traditional, computational, accounts of the mind. In particular, I will develop this argument by focusing on some proposals central to 4E approaches: (a) the non-brain-centric nature of cogni- tive processes, (b) the lack of explicit representations of the world, (c) the coupling of agent-environment systems and (d) the necessity of real-time feedback signals from the environment. Under the FEP formulation, I will present a series of case studies with mainly two objectives in mind: 1) to conceptually analyse and reframe these 4E ideas in the context of active inference, arguing for the advantages of their for- malisation in a more general probabilistic (Bayesian) framework and, 2) to present new mathematical models and agent-based implementations of some of the concep- tual connections between Bayesian inference frameworks and 4E proposals, largely missing in the literature. iv Acknowledgements I am grateful to my supervisor, Christopher Buckley, for his guidance over the last three years, inspiring me when it wasn’t clear where this project was going and providing me with accurate feedback when things finally began to work. He showed me how to be more critical and more constructive. Much of what I am as a scientist now, I owe to him. I want to thank Thomas Nowotny, my second supervisor, who lent an ear on several occasions. He regularly helped me with the most disparate issues and made sure I always had a backup plan when the exact direction of this thesis was still unclear. A special mention to my examiners, Andy Clark and Daniel Polani, whose input surely improved the presentation of this work. I thank my colleagues at Sussex: Simon McGregor for our discussions on infor- mation theory, probability theory and cognitive science and Chris Thornton for his careful comments on some of my caricatured descriptions of different ideas in cog- nitive science. I am also greatly indebted to colleagues from the Sackler Centre for Consciousness Science: Keisuke Suzuki for our conversations on embodied cogni- tive science, dynamical systems and physics, Warrick Roseboom for pointing out some of my naive beliefs and assumptions regarding empirical (neuro)science, Li- onel Barnett for the invaluable time he spent covering different aspects of stochastic processes and time series analysis, Anil Seth for sparking my initial interest in the theories presented in this thesis, for encouraging me to interact with his group and for providing support in different ways during this journey. The time spent in Tokyo at EON/ELSI between 2017-18 during an intermission period was invaluable and for this opportunity I must thank Olaf Witkowski for believing in me in the first place. Thanks to him and to Nathaniel Virgo, I had the chance to work on some incredibly interesting research questions while sur- rounded by wonderful colleagues. This experience was one of the most rewarding in my (short) career, widening my perspectives on fundamental questions of science. I wish to thank in particular Martin Biehl, Nicholas Guttenberg, Takuya Isomura and Lana Sinapayen for the conversations we had in Tokyo and around the world. For these conversations I also want to thank Taro Toyoizumi, Hideaki Shimazaki, Takashi Ikegami and Ryota Kanai who hosted me in their groups on different oc- casions to discuss my ideas, anything and everything. Taro and Hideaki especially helped me while I was trying to sort out some of the mathematical details presented in later chapters, thanks to them I came back to Brighton knowing that my ideas could work. My office-mates also played an important role at different stages of this project. Esin Yavuz helped me when I first settled in the office and listened to the crazy ideas of a first year PhD student, giving me good advice on several important matters. Mario Pannunzi forced me to consider ideas beyond maths and suggested ways to clarify some of the points presented in early chapters. To my family and to the friends who supported me during this long journey, you have my deepest gratitude. v Contents Declaration of Authorship i Abstract iii Acknowledgements iv 1 Introduction 1 1.1 Thesis contributions . .6 1.1.1 Published work . .7 1.1.2 Limitations . .8 2 Background 10 2.1 Embodied, enactive, extended and embedded (4E) cognition . 10 2.1.1 The cybernetics roots of 4E . 12 2.1.2 4E cognition and models of the environment . 13 2.2 Perception as inference (estimation) . 14 2.2.1 The Bayesian Brain hypothesis . 16 2.2.2 Predictive coding . 20 2.2.3 Predictive Processing . 21 Conservative predictive processing . 23 Radical Predictive Processing (RPP) . 23 2.3 Action as control . 24 2.3.1 Classical control . 25 2.3.2 Optimal control . 26 2.3.3 Stochastic optimal control . 27 2.3.4 Reinforcement learning . 28 2.3.5 Other relevant approaches to control . 30 2.4 The Free Energy Principle (FEP) . 32 2.4.1 Active inference . 33 2.4.2 Active inference agents . 33 The mountain car problem . 34 The linebot . 37 The infotropic machine . 38 Other models . 39 2.5 Conclusion . 40 vi 3 Methods 42 3.1 A mathematical formulation of the free energy principle . 42 3.1.1 The generative density . 45 3.1.2 The variational density . 47 Dynamic Expectation Maximisation (DEM) . 48 Variational filtering (VF) . 49 Generalised filtering (GF) . 50 3.1.3 The Laplace assumption . 50 3.1.4 The Laplace-encoded variational free energy . 53 3.2 The minimisation of variational free energy . 55 3.2.1 Perception . 56 3.2.2 Action . 57 3.2.3 Learning . 58 3.2.4 Attention . 59 4 A simple action-perception loop in active inference 62 4.1 Action in an active inference context . 62 4.2 A Bayesian cruise controller . 64 4.2.1 Just observing, the passive tracker . 68 4.2.2 In a delusional state, the passive dreamer . 71 4.2.3 Acting with no reason, the active tracker . 74 4.2.4 Chasing one’s dreams, the active dreamer . 75 4.3 Discussion . 77 4.4 Conclusion . 84 5 Generative models of sensorimotor contingencies 85 5.1 Background . 86 5.2 A minimal generative model of phototaxis .