DOCSLIB.ORG

Journal of Mathematical Psychology Active Inference on Discrete State

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Journal of Mathematical Psychology Active Inference on Discrete State Journal of Mathematical Psychology 99 (2020) 102447 Contents lists available at ScienceDirect Journal of Mathematical Psychology journal homepage: www.elsevier.com/locate/jmp Review Active inference on discrete state-spaces: A synthesis ∗ Lancelot Da Costa a,b, , Thomas Parr b, Noor Sajid b, Sebastijan Veselic b, Victorita Neacsu b, Karl Friston b a Department of Mathematics, Imperial College London, London, SW7 2RH, United Kingdom b Wellcome Centre for Human Neuroimaging, University College London, London, WC1N 3AR, United Kingdom article info a b s t r a c t Article history: Active inference is a normative principle underwriting perception, action, planning, decision-making Received 17 April 2020 and learning in biological or artificial agents. From its inception, its associated process theory has Received in revised form 23 July 2020 grown to incorporate complex generative models, enabling simulation of a wide range of complex Accepted 3 September 2020 behaviours. Due to successive developments in active inference, it is often difficult to see how its Available online xxxx underlying principle relates to process theories and practical implementation. In this paper, we try to Keywords: bridge this gap by providing a complete mathematical synthesis of active inference on discrete state- Active inference space models. This technical summary provides an overview of the theory, derives neuronal dynamics Free energy principle from first principles and relates this dynamics to biological processes. Furthermore, this paper provides Process theory a fundamental building block needed to understand active inference for mixed generative models; Variational Bayesian inference allowing continuous sensations to inform discrete representations. This paper may be used as follows: Markov decision process to guide research towards outstanding challenges, a practical guide on how to implement active Mathematical review inference to simulate experimental behaviour, or a pointer towards various in-silico neurophysiological responses that may be used to make empirical predictions. ' 2020 The Author(s). Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Contents 1. Introduction.........................................................................................................................................................................................................................3 2. Active inference..................................................................................................................................................................................................................3 3. Discrete state-space generative models...........................................................................................................................................................................6 4. Variational Bayesian inference..........................................................................................................................................................................................6 4.1. Free energy and model evidence.........................................................................................................................................................................6 4.2. On the family of approximate posteriors ...........................................................................................................................................................7 4.3. Computing the variational free energy...............................................................................................................................................................8 5. Perception............................................................................................................................................................................................................................9 5.1. Plausibility of neuronal dynamics........................................................................................................................................................................ 10 6. Planning, decision-making and action selection............................................................................................................................................................. 10 6.1. Planning and decision-making ............................................................................................................................................................................. 10 6.2. Action selection, policy-independent state-estimation ..................................................................................................................................... 11 6.3. Biological plausibility ............................................................................................................................................................................................ 11 6.4. Pruning of policy trees.......................................................................................................................................................................................... 11 6.5. Discussion of the action–perception cycle ......................................................................................................................................................... 11 7. Properties of the expected free energy ........................................................................................................................................................................... 12 8. Learning............................................................................................................................................................................................................................... 12 9. Structure learning............................................................................................................................................................................................................... 14 9.1. Bayesian model reduction .................................................................................................................................................................................... 14 9.2. Bayesian model expansion ................................................................................................................................................................................... 15 10. Discussion............................................................................................................................................................................................................................ 16 11. Conclusion ........................................................................................................................................................................................................................... 17 Declaration of competing interest.................................................................................................................................................................................... 17 ∗ Corresponding author at: Department of Mathematics, Imperial College London, London, SW7 2RH, United Kingdom. E-mail address: [email protected] (L. Da Costa). https://doi.org/10.1016/j.jmp.2020.102447 0022-2496/' 2020 The Author(s). Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by- nc-nd/4.0/). L. Da Costa, T. Parr, N. Sajid et al. Journal of Mathematical Psychology 99 (2020) 102447 .............................................................................................................................................................................................................................................. 17 Appendix A. More complex generative models.............................................................................................................................................................. 17 A.1. Learning B and D ................................................................................................................................................................................................... 17 A.2. Complexifying the prior over policies................................................................................................................................................................. 17 A.3. Multiple state and outcome modalities .............................................................................................................................................................. 18 A.4. Deep temporal models.......................................................................................................................................................................................... 18 Appendix B. Computation of dynamics underlying perception.................................................................................................................................... 18 B.1. Free energy conditioned upon a policy .............................................................................................................................................................. 18 B.2. Free energy gradients...........................................................................................................................................................................................
Load more

Recommended publications
  • The Digamma Function and Explicit Permutations of the Alternating Harmonic Series
    The digamma function and explicit permutations of the alternating harmonic series. Maxim Gilula February 20, 2015 Abstract The main goal is to present a countable family of permutations of the natural numbers that provide explicit rearrangements of the alternating harmonic series and that we can easily define by some closed expression. The digamma function presents its ubiquity in mathematics once more by being the key tool in computing explicitly the simple rearrangements presented in this paper. The permutations are simple in the sense that composing one with itself will give the identity. We show that the count- able set of rearrangements presented are dense in the reals. Then, slight generalizations are presented. Finally, we reprove a result given originally by J.H. Smith in 1975 that for any conditionally convergent real series guarantees permutations of infinite cycle type give all rearrangements of the series [4]. This result provides a refinement of the well known theorem by Riemann (see e.g. Rudin [3] Theorem 3.54). 1 Introduction A permutation of order n of a conditionally convergent series is a bijection φ of the positive integers N with the property that φn = φ ◦ · · · ◦ φ is the identity on N and n is the least such. Given a conditionally convergent series, a nat- ural question to ask is whether for any real number L there is a permutation of order 2 (or n > 1) such that the rearrangement induced by the permutation equals L. This turns out to be an easy corollary of [4], and is reproved below with elementary methods. Other \simple"rearrangements have been considered elsewhere, such as in Stout [5] and the comprehensive references therein.
    [Show full text]
  • The Riemann and Hurwitz Zeta Functions, Apery's Constant and New
    The Riemann and Hurwitz zeta functions, Apery’s constant and new rational series representations involving ζ(2k) Cezar Lupu1 1Department of Mathematics University of Pittsburgh Pittsburgh, PA, USA Algebra, Combinatorics and Geometry Graduate Student Research Seminar, February 2, 2017, Pittsburgh, PA A quick overview of the Riemann zeta function. The Riemann zeta function is defined by 1 X 1 ζ(s) = ; Re s > 1: ns n=1 Originally, Riemann zeta function was defined for real arguments. Also, Euler found another formula which relates the Riemann zeta function with prime numbrs, namely Y 1 ζ(s) = ; 1 p 1 − ps where p runs through all primes p = 2; 3; 5;:::. A quick overview of the Riemann zeta function. Moreover, Riemann proved that the following ζ(s) satisfies the following integral representation formula: 1 Z 1 us−1 ζ(s) = u du; Re s > 1; Γ(s) 0 e − 1 Z 1 where Γ(s) = ts−1e−t dt, Re s > 0 is the Euler gamma 0 function. Also, another important fact is that one can extend ζ(s) from Re s > 1 to Re s > 0. By an easy computation one has 1 X 1 (1 − 21−s )ζ(s) = (−1)n−1 ; ns n=1 and therefore we have A quick overview of the Riemann function. 1 1 X 1 ζ(s) = (−1)n−1 ; Re s > 0; s 6= 1: 1 − 21−s ns n=1 It is well-known that ζ is analytic and it has an analytic continuation at s = 1. At s = 1 it has a simple pole with residue 1.
    [Show full text]
  • Touro Psychology Major Requirements
    Touro Psychology Major Requirements ThedricPragmatismWeather-beaten percuss and hisEnglebertventricous hormone fructified Reinhard repast verymonotonously, bodying acridly so while inflexibly but Freddy derisive that remains Ravi Jerald triumphs curbablenever girnshis and metalanguages. so apocalyptic. initially. Touro University Worldwide offers an online PhD in card and Organizational Psychology. Does psychology have math? All Touro College graduates eligible for certification must apply online with is state in. Explore what you'd experience in an environmental psychology master's degree. Laboring on an online master's in organizational psychology degree could open doors for people. Code you use enter any MAJOR tournament you mention select the delinquent that says PSYCHOLOGY. Psychology is designed to try scope of sequence requirements for fellow single-semester introduction to psychology course this book offers a comprehensive. The hardest part about mathematics in psychology is Statistics While Statistics itself is rather forget and difficult psychometric measurements with its ordinal numbers or qualitative data will make it right more so. Exact major requirements for urge and other UC and CSU campuses can usually found online wwwassistorg Articulation agreements with private institutions can be. Touro University Worldwide offers a response of Arts MA in Psychology. Touro College South a division of a Jewish-sponsored college with fabulous main. Touro College UFT. SUNY New Paltz Welcome to Pre-Health SUNY New Paltz. Fillable Online las touro Touro CollegeAdvisement and. Best Online Master's in Psychology TheBestSchoolsorg. Most careers in psychology require a minimum of a masters degree will it's no. Touro University Worldwide offers one help the few PsyD degree programs.
    [Show full text]
  • COMPLETE MONOTONICITY for a NEW RATIO of FINITE MANY GAMMA FUNCTIONS Feng Qi
    COMPLETE MONOTONICITY FOR A NEW RATIO OF FINITE MANY GAMMA FUNCTIONS Feng Qi To cite this version: Feng Qi. COMPLETE MONOTONICITY FOR A NEW RATIO OF FINITE MANY GAMMA FUNCTIONS. 2020. hal-02511909 HAL Id: hal-02511909 https://hal.archives-ouvertes.fr/hal-02511909 Preprint submitted on 19 Mar 2020 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. COMPLETE MONOTONICITY FOR A NEW RATIO OF FINITE MANY GAMMA FUNCTIONS FENG QI Dedicated to people facing and fighting COVID-19 Abstract. In the paper, by deriving an inequality involving the generating function of the Bernoulli numbers, the author introduces a new ratio of finite many gamma functions, finds complete monotonicity of the second logarithmic derivative of the ratio, and simply reviews complete monotonicity of several linear combinations of finite many digamma or trigamma functions. Contents 1. Preliminaries and motivations 1 2. A lemma 3 3. Complete monotonicity 4 4. A simple review 5 References 7 1. Preliminaries and motivations Let f(x) be an infinite differentiable function on (0; 1). If (−1)kf (k)(x) ≥ 0 for all k ≥ 0 and x 2 (0; 1), then we call f(x) a completely monotonic function on (0; 1).
    [Show full text]
  • General Psychology
    mathematics HEALTH ENGINEERING DESIGN MEDIA management GEOGRAPHY EDUCA E MUSIC C PHYSICS law O ART L agriculture O BIOTECHNOLOGY G Y LANGU CHEMISTRY TION history AGE M E C H A N I C S psychology General Psychology Subject: GENERAL PSYCHOLOGY Credits: 4 SYLLABUS A definition of Psychology Practical problems, Methods of Psychology, Work of Psychologists, Schools of psychology, Attention & Perception - Conscious clarity, determinants of Attention, Distraction, Sensory deprivation, Perceptual constancies, perception of fundamental physical dimensions, Illusions, Organizational factors of perception. Principles of learning Classical conditioning, Operant Conditioning, Principles of reinforcement, Cognitive Learning, Individualized learning, Learner & learning memory - kinds of memory, processes of memory, stages of memory, forgetting. Thinking and language - Thinking process, Concepts. Intelligence & Motivation Theories - Measurement of Intelligence; Determinants; Testing for special aptitudes, Motivation - Motives as inferences, Explanations and predictors, Biological motivation, Social motives, Motives to know and to be effective. Emotions Physiology of emotion, Expression of emotions, Theories of emotions; Frustration and conflict, Personality - Determinants of Personality, Theories of personality Psychodynamic, Trait, Type, Learning, Behavioural & Self: Measurement of personality Suggested Readings: 1. Morgan, Clifford. T., King, Richard. A., Weisz, John.R., Schopler, John, Introduction to Psychology, TataMcGraw Hill. 2. Marx, Melvin H.
    [Show full text]
  • Euler-Maclaurin and Euler-Boole Formulas
    Appendix A Euler-MacLaurin and Euler-Boole Formulas A.1 A Taylor Formula The classical Taylor formula Z Xm xk x .x t/m f .x/ D @kf .0/ C @mC1f .t/dt kŠ 0 mŠ kD0 xk can be generalized if we replace the polynomial kŠ by other polynomials (Viskov 1988; Bourbaki 1959). Definition If is a linear form on C0.R/ such that .1/ D 1,wedefinethe polynomials .Pn/ by: P0 D 1 @Pn D Pn1 , .Pn/ D 0 for n 1 P . / k The Generating Function k0 Pk x z We have formally X X X k k k @x. Pk.x/z / D Pk1.x/z D z. Pk.x/z / k0 k1 k0 © Springer International Publishing AG 2017 175 B. Candelpergher, Ramanujan Summation of Divergent Series, Lecture Notes in Mathematics 2185, DOI 10.1007/978-3-319-63630-6 176 A Euler-MacLaurin and Euler-Boole Formulas thus X k xz Pk.x/z D C.z/e k0 To evaluate C.z/ we use the notation x for and by definition of .Pn/ we can write X X k k x. Pk.x/z / D x.Pk.x//z D 1 k0 k0 X k xz xz x. Pk.x/z / D x.C.z/e / D C.z/x.e / k0 1 this gives C.z/ D xz . Thus the generating function of the sequence .Pn/ is x.e / X n xz Pn.x/z D e =M.z/ n xz where the function M is defined by M.z/ D x.e /: Examples P xn n xz .
    [Show full text]
  • How a Cognitive Psychologist Came to Seek Universal Laws
    Psychonomic Bulletin & Review 2004, 11 (1), 1-23 PSYCHONOMIC SOCIETY KEYNOTE ADDRESS How a cognitive psychologist came to seek universal laws ROGERN. SHEPARD Stanford University, Stanford, California and Arizona Senior Academy, Tucson, Arizona Myearlyfascination with geometry and physics and, later, withperception and imagination inspired a hope that fundamental phenomena ofpsychology, like those ofphysics, might approximate univer­ sal laws. Ensuing research led me to the following candidates, formulated in terms of distances along shortestpaths in abstractrepresentational spaces: Generalization probability decreases exponentially and discrimination time reciprocally with distance. Time to determine the identity of shapes and, pro­ visionally, relation between musical tones or keys increases linearly with distance. Invariance of the laws is achieved by constructing the representational spaces from psychological rather than physical data (using multidimensional scaling) and from considerations of geometry, group theory, and sym­ metry. Universality of the laws is suggested by their behavioral approximation in cognitively advanced species and by theoretical considerations of optimality. Just possibly, not only physics but also psy­ chology can aspire to laws that ultimately reflect mathematical constraints, such as those ofgroup the­ ory and symmetry, and, so, are both universal and nonarbitrary. The object ofall science, whether natural science or psy­ three principal respects. The first is my predilection for chology, is to coordinate our experiences and to bring them applying mathematical ideas that are often ofa more geo­ into a logical system. metrical or spatial character than are the probabilistic or (Einstein, 1923a, p. 2) statistical concepts that have generally been deemed [But] the initial hypotheses become steadily more abstract most appropriate for the behavioral and social sciences.
    [Show full text]
  • On Some Series Representations of the Hurwitz Zeta Function Mark W
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Elsevier - Publisher Connector Journal of Computational and Applied Mathematics 216 (2008) 297–305 www.elsevier.com/locate/cam On some series representations of the Hurwitz zeta function Mark W. Coffey Department of Physics, Colorado School of Mines, Golden, CO 80401, USA Received 21 November 2006; received in revised form 3 May 2007 Abstract A variety of infinite series representations for the Hurwitz zeta function are obtained. Particular cases recover known results, while others are new. Specialization of the series representations apply to the Riemann zeta function, leading to additional results. The method is briefly extended to the Lerch zeta function. Most of the series representations exhibit fast convergence, making them attractive for the computation of special functions and fundamental constants. © 2007 Elsevier B.V. All rights reserved. MSC: 11M06; 11M35; 33B15 Keywords: Hurwitz zeta function; Riemann zeta function; Polygamma function; Lerch zeta function; Series representation; Integral representation; Generalized harmonic numbers 1. Introduction (s, a)= ∞ (n+a)−s s> a> The Hurwitz zeta function, defined by n=0 for Re 1 and Re 0, extends to a meromorphic function in the entire complex s-plane. This analytic continuation to C has a simple pole of residue one. This is reflected in the Laurent expansion ∞ n 1 (−1) n (s, a) = + n(a)(s − 1) , (1) s − 1 n! n=0 (a) (a)=−(a) =/ wherein k are designated the Stieltjes constants [3,4,9,13,18,20] and 0 , where is the digamma a a= 1 function.
    [Show full text]
  • A New Entire Factorial Function
    A new entire factorial function Matthew D. Klimek Laboratory for Elementary Particle Physics, Cornell University, Ithaca, NY, 14853, USA Department of Physics, Korea University, Seoul 02841, Republic of Korea Abstract We introduce a new factorial function which agrees with the usual Euler gamma function at both the positive integers and at all half-integers, but which is also entire. We describe the basic features of this function. The canonical extension of the factorial to the complex plane is given by Euler's gamma function Γ(z). It is the only such extension that satisfies the recurrence relation zΓ(z) = Γ(z + 1) and is logarithmically convex for all positive real z [1, 2]. (For an alternative function theoretic characterization of Γ(z), see [3].) As a consequence of the recurrence relation, Γ(z) is a meromorphic function when continued onto − the complex plane. It has poles at all non-positive integer values of z 2 Z0 . However, dropping the two conditions above, there are infinitely many other extensions of the factorial that could be constructed. Perhaps the second best known factorial function after Euler's is that of Hadamard [4] which can be expressed as sin πz z z + 1 H(z) = Γ(z) 1 + − ≡ Γ(z)[1 + Q(z)] (1) 2π 2 2 where (z) is the digamma function d Γ0(z) (z) = log Γ(z) = : (2) dz Γ(z) The second term in brackets is given the name Q(z) for later convenience. We see that the Hadamard gamma is a certain multiplicative modification to the Euler gamma.
    [Show full text]
  • The Psychology of Mathematics and Mathematics for Psychologists
    The Psychology of Mathematics and Mathematics for Psychologists Luc Delbeke Katholieke Universiteit Leuven, Department of Psychology Tiensestraat 102 B3000 Leuven, Belgium [email protected] The International Academy of Education (IAE: a scientific association that promotes educational research) published a booklet on improving students’ achievements in mathematics (Grouw, D.A. and Cebulla, K.J, 2000). It focuses on aspects of learning that appear to be universal in much formal schooling, although the research it is based on mainly comes from primary and middle school education. Nevertheless, the practices that are recommended in the booklet seem likely to be generally applicable. Even so, the authors caution that the principles they advocate should be assessed with reference to local conditions, and adapted accordingly. It will be argued that these principles can be generalized to the teaching of psychometrics, statistics and mathematical modeling to psychology students, but that indeed adaptation to the context of teaching mathematics-related subjects to psychology students will be in order. According to Meier (1993) the results of Aiken et al.’s (1990) survey of U.S. graduate departments of psychology amply documented the lack of emphasis placed by those departments on psychological measurement. Only one fourth rated their graduate students as skilled with psychometric methods and concepts. Only 13 % offered test construction courses. Rated competencies declined with newer approaches in psychometrics: students are judged as most proficient with classical methods of reliability and validity measurement, but less so with factor and item analysis methods, and nearly ignorant of item response and generalizability theories. It led him to urge for a “revitalizing” of the measurement curriculum.
    [Show full text]
  • The American Journal of Psychology, 105, 1992, 626-631
    626 BOOK REVIEWS Chi, M. T. H., Glaser, R., & Farr, M. J. (1988). The nature ofexpertise. Hillsdale, NJ: Erlbaum. Gardner, H. (1985). The mind's new science: A history ofthe cognitive revolution. New York: Basic Books. Horowitz, M. (1988). Introduction to psychodynamics: A new synthesis. New York: Basic Books. LeDoux, J. E. (1989). Cognitive-emotional interactions in the brain. Cognition & Emo- tion, 3, 267-289. Mamelak, A. N., & Hobson, J. A. (1989). Dream bizarreness as the cognitive correlate of altered neuronal behavior in REM sleep. Journal of Cognitive Neuroscience, I, 201-222. Neisser, U. (1976). Cognition and reality. New York: Freeman. Ojemann, C. A. (1983). Brain organization for language from the perspective of electrical stimulation mapping. Behavioral and Brain Sciences, 6, 189-230. Petersen, S. E., Fox, P. T., Snyder, A. Z., & Raichle, M. E. (1990). Activation of extrastriate areas by visual words and word-like stimuli. Science, 248, 1041-1043. Posner, M. I. (1990). Foundations of cognitive science. Cambridge, MA: MIT Press/ Bradford Books. Rummelhart, D. E., & McClelland, J. L. (1986). Parallel distributed processing. Cam- bridge, MA: MIT Press. Treisman, A. (1988). Features and objects. Quarterly Journal ofExperimenta1 Psychology, 40(A), 201-237. Frontiers of Mathematical Psychology: Essays in Honor of Clyde Coombs Edited by Donald R. Brown and J. E. Keith Smith. New York: Springer- Verlag, 1991. 202 pp. Paper, $35.00. Titles can be seductive. "Frontiers" seems to promise that mastery of the volume will provide one with a good sense of what is current in that field. Certainly Frontiers of Mathematical Psychology Is easily mastered by any ex- perimental or cognitive psychologist, but he or she will learn little about contemporary mathematical psychology.
    [Show full text]
  • Colleges Offering Masters in Psychology in Kolkata
    Colleges Offering Masters In Psychology In Kolkata If directed or intransitive Dewey usually molts his deoxidiser act inescapably or tuts hand-to-mouth and adhesively, how cleft is Antonin? Telemetered and middling Angel manages her cowages drip-dries or rarefy spontaneously. Is Welch backstair or transferable after approvable Kaspar carnalize so explanatorily? When you find a career that does not support of languages such cases of living will aim to my sights on colleges offering this institute reserves the latest updates The Guardian University Guide 2019 Over 93 of final-year Psychology. Which college in kolkata for offering msc in the university offered to engage in. Admission depends on colleges in schools in psychology is a special needs to address provided by a private university and referenced to read our available to finalise my city. Quote message for relevant topics of flame university offers a versatile way to perform evaluation services and behavior and work? Dual DegreeIntegrated BScMSc Programmes in Kolkata. View 2 colleges offering MA in Psychology in Kolkata Download colleges brochure read questions and student reviews Compare colleges on fees eligibility. Notice for psychological advantages to in kolkata for me after the covid pandemic situation courses? Career As Psychologist Courses Scope Jobs Salary. This is just show up to name a student housing with as well as judges in any admission? European tech in masters was. Of last island for Admission in altogether different policy Graduate courses in Jadavpur University. Is psychology easy gate study? What plate the connection between mathematics and psychology. Unleash your potential with arbitrary-focused degree programs in Architecture.
    [Show full text]