DOCSLIB.ORG

Crash Prediction and Collision Avoidance Using Hidden

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Crash Prediction and Collision Avoidance Using Hidden CRASH PREDICTION AND COLLISION AVOIDANCE USING HIDDEN MARKOV MODEL A Thesis Submitted to the Faculty of Purdue University by Avinash Prabu In Partial Fulfillment of the Requirements for the Degree of Master of Science in Electrical and Computer Engineering August 2019 Purdue University Indianapolis, Indiana ii THE PURDUE UNIVERSITY GRADUATE SCHOOL STATEMENT OF THESIS APPROVAL Dr. Lingxi Li, Chair Department of Electrical and Computer Engineering Dr. Yaobin Chen Department of Electrical and Computer Engineering Dr. Brian King Department of Electrical and Computer Engineering Approved by: Dr. Brian King Head of Graduate Program iii Dedicated to my family, friends and my Professors, for their support and encouragement. iv ACKNOWLEDGMENTS Firstly, I would like to thank Dr. Lingxi Li for his continuous support and en- couragement. He has been a wonderful advisor and an exceptional mentor, without whom, my academic journey will not be where it is now. I would also like to to thank Dr. Yaobin Chen, whose course on Control theory has had a great impact in shaping this thesis. I would like to extend my appreciation to Dr. Brian King, who has been a great resource of guidance throughout the course of my Master's degree. A special thanks to Dr. Renran Tian, who is my principle investigator at TASI, IUPUI, for his guidance and encouragement. I also wish to thank Sherrie Tucker, who has always been available to answer my questions, despite her busy schedule. I would like to acknowledge my parents, Mr. & Mrs. Prabhu and my sister Chathurma for their continuous belief and endless love towards me. I also acknowledge my TASI team members and friends for their support. v TABLE OF CONTENTS Page LIST OF TABLES :::::::::::::::::::::::::::::::::: vii LIST OF FIGURES ::::::::::::::::::::::::::::::::: viii ABSTRACT ::::::::::::::::::::::::::::::::::::: x 1 INTRODUCTION :::::::::::::::::::::::::::::::: 1 1.1 Literature Review :::::::::::::::::::::::::::::: 2 1.2 Thesis Contributions :::::::::::::::::::::::::::: 5 1.3 Thesis Organization ::::::::::::::::::::::::::::: 5 2 MARKOV CHAINS ::::::::::::::::::::::::::::::: 7 2.1 Description of a Markov Chain :::::::::::::::::::::: 7 2.1.1 Model Specifications of a Markov Chain ::::::::::::: 8 2.1.2 States, Transition Matrix and Initial Probability Distribution : 8 2.2 Types of Markov Chains :::::::::::::::::::::::::: 14 2.2.1 Absorbing Markov Chains ::::::::::::::::::::: 15 2.2.2 Ergodic and Regular Markov Chains ::::::::::::::: 17 3 HIDDEN MARKOV MODEL IMPLEMENTATION IN ACTIVE SAFETY : 23 3.1 Implementation and Analysis of HMM :::::::::::::::::: 23 3.1.1 Elements of HMM ::::::::::::::::::::::::: 26 3.1.2 HMM Explained Through a Simple Coin Toss Example ::::: 27 3.2 Design of Active Safety System :::::::::::::::::::::: 29 3.2.1 Markov Chain - Model Specification :::::::::::::::: 29 3.2.2 Model Design :::::::::::::::::::::::::::: 32 3.2.3 Description of Variables and Formulae Used ::::::::::: 33 3.3 Crash Prediction and Avoidance :::::::::::::::::::::: 37 3.3.1 Flow-1 - Calculation for Probable Time of Crash ::::::::: 37 vi Page 3.3.2 Flow 2 - Calculation of Probable Crash Probability ::::::: 40 3.3.3 Flow 3 - Determining the Appropriate Active Safety System :: 42 3.4 Adaptive Cruise Control :::::::::::::::::::::::::: 43 3.5 Lane Keep Assist :::::::::::::::::::::::::::::: 44 4 SIMULATION AND RESULTS ::::::::::::::::::::::::: 46 4.1 Description of Data Obtained ::::::::::::::::::::::: 47 4.2 Simulation :::::::::::::::::::::::::::::::::: 48 4.2.1 Determination of State Transition Matrix ::::::::::::: 49 4.2.2 Adaptive Cruise Control Simulation :::::::::::::::: 56 4.3 Scenarios and Simulation Result :::::::::::::::::::::: 57 4.3.1 Scenario 1: Unsafe Operation with ACC ::::::::::::: 59 4.3.2 Scenario 2: Unsafe Operation with Lane Departure and Steer- ing Assist :::::::::::::::::::::::::::::: 62 4.3.3 Scenario 3: Safe Operation ::::::::::::::::::::: 63 5 CONCLUSION AND FUTURE WORK :::::::::::::::::::: 66 5.1 Conclusion :::::::::::::::::::::::::::::::::: 66 5.2 Future Work ::::::::::::::::::::::::::::::::: 67 REFERENCES :::::::::::::::::::::::::::::::::::: 68 vii LIST OF TABLES Table Page 3.1 Observation Sequence of 1-coin Model :::::::::::::::::::: 24 3.2 Observation Sequence of 2-coin Model :::::::::::::::::::: 25 3.3 Observation Sequence of 3-coin Model :::::::::::::::::::: 25 3.4 Observation Probabilities of 3-coin Model :::::::::::::::::: 26 3.5 First Layer ::::::::::::::::::::::::::::::::::: 30 3.6 Second Layer :::::::::::::::::::::::::::::::::: 30 3.7 Observation Probabilities for Each Lane ::::::::::::::::::: 31 4.1 Speed Range Mapping ::::::::::::::::::::::::::::: 54 viii LIST OF FIGURES Figure Page 2.1 Random Walk of a Gentleman ::::::::::::::::::::::::: 11 2.2 Transition Graph of Random Walk :::::::::::::::::::::: 12 2.3 Transition Graph for a Markov Chain with Three States :::::::::: 13 2.4 Cat's Random Walk :::::::::::::::::::::::::::::: 15 2.5 Markov Chain of a Cat's Random Walk ::::::::::::::::::: 16 3.1 1-coin Model :::::::::::::::::::::::::::::::::: 24 3.2 2-coin Model :::::::::::::::::::::::::::::::::: 25 3.3 3-coin Model :::::::::::::::::::::::::::::::::: 26 3.4 HMM Model :::::::::::::::::::::::::::::::::: 28 3.5 Road Layout :::::::::::::::::::::::::::::::::: 32 3.6 Lane Change Markov Chain :::::::::::::::::::::::::: 33 3.7 Speed Change Markov Chain ::::::::::::::::::::::::: 33 3.8 Lane Change Data ::::::::::::::::::::::::::::::: 34 3.9 Probable Time of Crash :::::::::::::::::::::::::::: 38 3.10 Lidar Principle ::::::::::::::::::::::::::::::::: 40 3.11 Probable Crash Probability :::::::::::::::::::::::::: 41 3.12 Flow 3 :::::::::::::::::::::::::::::::::::::: 42 3.13 Distance Calculation :::::::::::::::::::::::::::::: 43 3.14 Adaptive Cruise Control :::::::::::::::::::::::::::: 44 3.15 Lane Keep Assist :::::::::::::::::::::::::::::::: 45 4.1 Data Obtained from USDOT ::::::::::::::::::::::::: 46 4.2 Filtered Data Used for Simulation ::::::::::::::::::::::: 47 4.3 Instrumented Vehicle :::::::::::::::::::::::::::::: 47 4.4 Lane Change State Transition Matrix Flow :::::::::::::::::: 49 ix Figure Page 4.5 Probability(Lane Change) ::::::::::::::::::::::::::: 50 4.6 State Transition Matrix of Car 1 ::::::::::::::::::::::: 51 4.7 State Transition Matrix of Car 2 ::::::::::::::::::::::: 51 4.8 State Transition graph of Car 1 :::::::::::::::::::::::: 52 4.9 State Transition Graph of Car 2 :::::::::::::::::::::::: 52 4.10 Speed Change State Transition Matrix Flow ::::::::::::::::: 53 4.11 Speed Change State Transition Matrix of Car 1 ::::::::::::::: 55 4.12 Speed Change State Transition Matrix of Car 2 ::::::::::::::: 55 4.13 W, Z and M for Car 1 ::::::::::::::::::::::::::::: 56 4.14 W, Z and M for Car 2 ::::::::::::::::::::::::::::: 56 4.15 Adaptive Cruise Control :::::::::::::::::::::::::::: 58 4.16 Plant Design for Cars :::::::::::::::::::::::::::::: 59 4.17 Lane probabilities of Car 1 at probable time of crash :::::::::::: 60 4.18 Lane Probabilities of Car 2 at Probable Time of Crash ::::::::::: 60 4.19 Crash Probability at Probable Time of Crash :::::::::::::::: 60 4.20 Simulation of ACC ::::::::::::::::::::::::::::::: 61 4.21 Lane Probabilities of Car 1 at Probable Time of Crash ::::::::::: 62 4.22 Lane Probabilities of Car 2 at Probable Time of Crash ::::::::::: 63 4.23 Crash Probability at Probable Time of Crash :::::::::::::::: 63 4.24 Lane Probabilities of Car 1 at Probable Time of Crash ::::::::::: 64 4.25 Lane Probabilities of Car 2 at Probable Time of Crash ::::::::::: 64 4.26 Crash Probability at Probable Time of Crash :::::::::::::::: 64 x ABSTRACT Prabu, Avinash. M.S.E.C.E., Purdue University, August 2019. Crash Prediction and Collision Avoidance using Hidden Markov Model. Major Professor: Lingxi Li. Automotive technology has grown from strength to strength in the recent years. The main focus of research in the near past and the immediate future are autonomous vehicles. Autonomous vehicles range from level 1 to level 5, depending on the percent- age of machine intervention while driving. To make a smooth transition from human driving and machine intervention, the prediction of human driving behavior is critical. This thesis is a subset of driving behavior prediction. The objective of this thesis is to predict the possibility of crash and implement an appropriate active safety system to prevent the same. The prediction of crash requires data of transition between lanes, and speed ranges. This is achieved through a variation of hidden Markov model. With the crash prediction and analysis of the Markov models, the required ADAS system is activated. The above concept is divided into sections and an algorithm was developed. The algorithm is then scripted into MATLAB for simulation. The results of the simulation is recorded and analyzed to prove the idea. 1 1. INTRODUCTION The number of cars have been increasing exponentially every year. This implies the need for Advanced Driver Assisted Systems more than ever. It is the partial duty of the manufacturers to ensure that the occupants of the vehicle are safe and at the same time comply with the safety and driving regulations of the road. Advanced Driver assisted systems are human-machine interface, that increase car and road safety. These systems help in reducing the human error and thereby reducing the probability of a crash. ADAS systems can be classified into active and passive systems. A passive system alerts the driver when there is a possibility of a dangerous situation. An example of a passive systems is the lane departure warning alert. On
Load more

Recommended publications
  • Statistical Pattern Recognition: a Review
    4 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 22, NO. 1, JANUARY 2000 Statistical Pattern Recognition: A Review Anil K. Jain, Fellow, IEEE, Robert P.W. Duin, and Jianchang Mao, Senior Member, IEEE AbstractÐThe primary goal of pattern recognition is supervised or unsupervised classification. Among the various frameworks in which pattern recognition has been traditionally formulated, the statistical approach has been most intensively studied and used in practice. More recently, neural network techniques and methods imported from statistical learning theory have been receiving increasing attention. The design of a recognition system requires careful attention to the following issues: definition of pattern classes, sensing environment, pattern representation, feature extraction and selection, cluster analysis, classifier design and learning, selection of training and test samples, and performance evaluation. In spite of almost 50 years of research and development in this field, the general problem of recognizing complex patterns with arbitrary orientation, location, and scale remains unsolved. New and emerging applications, such as data mining, web searching, retrieval of multimedia data, face recognition, and cursive handwriting recognition, require robust and efficient pattern recognition techniques. The objective of this review paper is to summarize and compare some of the well-known methods used in various stages of a pattern recognition system and identify research topics and applications which are at the forefront of this exciting and challenging field. Index TermsÐStatistical pattern recognition, classification, clustering, feature extraction, feature selection, error estimation, classifier combination, neural networks. æ 1INTRODUCTION Y the time they are five years old, most children can depend in some way on pattern recognition..
    [Show full text]
  • Probabilistic Sequence Models with Speech and Language Applications
    Thesis for the degree of Doctor of Philosophy Probabilistic Sequence Models with Speech and Language Applications Gustav Eje Henter Communication Theory School of Electrical Engineering KTH – Royal Institute of Technology Stockholm 2013 Henter, Gustav Eje Probabilistic Sequence Models with Speech and Language Applications Copyright c 2013 Gustav Eje Henter except where otherwise stated. All rights reserved. ISBN 978-91-7501-932-1 TRITA-EE 2013:042 ISSN 1653-5146 Communication Theory School of Electrical Engineering KTH – Royal Institute of Technology SE-100 44 Stockholm, Sweden Abstract Series data, sequences of measured values, are ubiquitous. Whenever ob- servations are made along a path in space or time, a data sequence results. To comprehend nature and shape it to our will, or to make informed de- cisions based on what we know, we need methods to make sense of such data. Of particular interest are probabilistic descriptions, which enable us to represent uncertainty and random variation inherent to the world around us. This thesis presents and expands upon some tools for creating prob- abilistic models of sequences, with an eye towards applications involving speech and language. Modelling speech and language is not only of use for creating listening, reading, talking, and writing machines—for instance al- lowing human-friendly interfaces to future computational intelligences and smart devices of today—but probabilistic models may also ultimately tell us something about ourselves and the world we occupy. The central theme of the thesis is the creation of new or improved models more appropriate for our intended applications, by weakening limiting and questionable assumptions made by standard modelling techniques.
    [Show full text]
  • Unsupervised Language Acquisition: Theory and Practice
    Unsupervised Language Acquisition: Theory and Practice Alexander Simon Clark Submitted for the degree of D.Phil. University of Sussex September, 2001 Declaration I hereby declare that this thesis has not been submitted, either in the same or different form, to this or any other university for a degree. Signature: Acknowledgements First, I would like to thank Bill Keller, for his supervision over the past three years. I would like to thank my colleagues at ISSCO for making me welcome and especially Susan Armstrong for giving me the job in the first place, and various people for helpful comments and discussions including Chris Manning, Dan Klein, Eric Gaussier, Nicola Cancedda, Franck Thollard, Andrei Popescu-Beilis, Menno van Zaanen and numerous other people including Sonia Halimi for check- ing my Arabic. I would also like to thank Gerald Gazdar and Geoffrey Sampson for their helpful comments as part of my thesis committee. I would also like to thank all of the people that have worked on the various software packages and operating systems that I have used, such as LaTeX, Gnu/Linux, and the gcc compiler, as well as the people who have worked on the preparation of the various corpora I have used: in particular I am grateful to John McCarthy and Ramin Nakisa for allowing me to use their painstakingly collected data sets for Arabic. Finally I would like to thank my wife, Dr Clare Hornsby, for her unfailing support and encouragement (in spite of her complete lack of understanding of the subject) and my children, Lily and Francis, for putting up with my absences both physical and mental, and showing me how language acquisition is really done – over-regularisation of past tense verbs included.
    [Show full text]
  • Simplicity: a Unifying Principle in Cognitive Science?
    Simplicity: A unifying principle in cognitive science? Nick Chater Institute for Applied Cognitive Science Department of Psychology University of Warwick Coventry, CV4 7AL UK Paul Vitányi, Centrum voor Wiskunde en Informatica Kruislaan 413 1098 SJ Amsterdam The Netherlands 1 Summary Much of perception, learning and high-level cognition involves finding patterns in data. But there are always infinitely many patterns compatible with any finite amount of data. How does the cognitive system choose ‘sensible’ patterns? A long tradition in epistemology, philosophy of science, and mathematical and computational theories of learning argues that patterns ‘should’ be chosen according to how simply they explain the data. This article reviews research exploring the idea that simplicity does, indeed, drive a wide range of cognitive processes. We outline mathematical theory, computational results, and empirical data underpinning this viewpoint. Key words: simplicity, Kolmogorov complexity, codes, learning, induction, Bayesian inference 30-word summary:This article outlines the proposal that many aspects of cognition, from perception, to language acquisition, to high-level cognition involve finding patterns that provide the simplest explanation of available data. 2 The cognitive system finds patterns in the data that it receives. Perception involves finding patterns in the external world, from sensory input. Language acquisition involves finding patterns in linguistic input, to determine the structure of the language. High-level cognition involves finding patterns in information, to form categories, and to infer causal relations. Simplicity and the problem of induction A fundamental puzzle is what we term the problem of induction: infinitely many patterns are compatible with any finite set of data (see Box 1).
    [Show full text]
  • CSCI 5521: Pattern Recognition Prof
    CSCI 5521: Pattern Recognition Prof. Paul Schrater Business • Course web page: http://gandalf.psych.umn.edu/~schrater/schrater_lab/courses /PattRecog07/PattRecog.html Prof. Paul Schrater Pattern Recognition CSCI 5521 2 Syllabus • Reading materials: – Pattern Classification, 2nd Ed. Duda, Hart, Stork – Statistical Pattern Recognition, 2nd Ed. Andrew Webb – Elements of Statistical Learning: Data Mining, Inference, Prediction, Trevor Hastie, Robert Tibshirani, and Jerome Friedman. Springer, 2001. – Gaussian Processes for Machine Learning, Carl Rasmussen and Christopher K.I. Williams, MIT Press, 2006. – Papers posted on the web site. – Downloads will be password protected. • Grading • 60% on the homework assignments • 40% on the final project. Prof. Paul Schrater Pattern Recognition CSCI 5521 3 Syllabus cont’d • Final Project 12-15 page paper involving: 1) Simulation or experiments. For example, implement a pattern recognition system for a particular application, e.g. digit classification, document clustering, etc. 2) Literature survey (with critical evaluation) on a given topic. 3) Theoretical work (detailed derivations, extensions of existing work, etc) Important dates: • Sept. 28: Topic selection. One or two pages explaining the project with a list of references. • Nov. 7: Partial report (3 to 5 pages). • Dec. 20: Final report (12 to 15 pages). • Students may work in groups of 2-4. Prof. Paul Schrater Pattern Recognition CSCI 5521 4 Policies/Procedures • DO NOT CHEAT. • Do NOT work in groups for homework (consulting with fellow students permitted, but group submissions are not). • Electronically submit homework. • Homework must be submitted by Midnight on the day it is due. Prof. Paul Schrater Pattern Recognition CSCI 5521 5 Introduction to Pattern Recognition • Syllabus • What are Patterns? • Pattern Recognition • An Example • Pattern Recognition Systems • The Design Cycle • Learning and Adaptation • Conclusion Examples of Patterns Prof.
    [Show full text]
  • Epurdy-Phd-Dissertation.Pdf
    THE UNIVERSITY OF CHICAGO GRAMMATICAL METHODS IN COMPUTER VISION A DISSERTATION SUBMITTED TO THE FACULTY OF THE DIVISION OF THE PHYSICAL SCIENCES IN CANDIDACY FOR THE DEGREE OF DOCTOR OF PHILOSOPHY DEPARTMENT OF COMPUTER SCIENCE BY ERIC PURDY CHICAGO, ILLINOIS MARCH 2013 Copyright c 2013 by Eric Purdy All Rights Reserved To my beloved wife TABLE OF CONTENTS LIST OF FIGURES . vii LIST OF TABLES . xii ABSTRACT . xiii 1 INTRODUCTION . 1 1.1 Examples of Grammatical Approaches . .2 1.1.1 Curve Grammars . .2 1.1.2 Visual Chalkboard . .4 1.1.3 Visual Search Engine . .4 1.1.4 Other Visual Grammars . .4 1.2 Grammatical Vision is Important . .6 1.2.1 Soft Decisions . .6 1.2.2 Modeling Clutter with Object Sub-parts . .7 1.2.3 Whole Scene Parsing . .7 1.3 Hierarchical Decomposition and Rich Description . .8 1.3.1 Training on rich annotations . .9 1.3.2 Some Problems with Rich Description, and Solutions . 10 1.3.3 Rich Description and XML . 12 1.4 Grammars and Statistical Models . 13 1.4.1 Statistical Modules . 13 1.4.2 Independence and the Poverty of Stimulus . 13 1.5 Grammatical Vision is Difficult . 15 1.6 Grammar induction is difficult . 15 1.7 Visual Grammars are hard for additional reasons . 16 1.7.1 Comparisons are Always Inexact . 16 1.7.2 Dealing with Scale . 17 1.7.3 Plane grammars do not naturally admit efficient parsing . 18 1.8 Contributions . 18 2 GRAMMATICAL MODELS OF SHAPE . 20 2.1 Introduction . 20 2.2 Motivating Example: L-Systems .
    [Show full text]
  • Cveigh, E., Kass, D., Miller, M.I., Winslow, R.L
    Curriculum Vitae Michael I. Miller https://en.wikipedia.org/wiki/Michael_I._Miller Bessie Darling Massey Professor and Director, Johns Hopkins Department of Biomedical Engineering Director, Center for Imaging Science Co-Director, Kavli Neuroscience Discovery Institute Gilman Scholar Professor of Electrical and Computer Engineering (410) 516-3826 (office) [email protected] Education Ph.D. Biomedical Engineering, Johns Hopkins University, 1984 M.S. Electrical Engineering, Johns Hopkins University, 1979 B.S. Electrical Engineering, State University of New York at Stony Brook, 1976 Experience • Bessie Darling Massey Professor and Director, Department of Biomedical Engineering. 2017 - Present • Co-Director, Kavli Neuroscience Discovery Institute. 2015 – Present • Director, Center for Imaging Science. 2000 - Present • Gilman Scholar, Johns Hopkins University. 2011 – Present • Herschel and Ruth Seder Chair in Biomedical Engineering, Johns Hopkins University. 2004 - 2017 • Professor, Johns Hopkins University. 1998 - Present • Visiting Professor,École Normale Supérieure de Cachan, France. 2000, May 2002, May 2007, May 2009, May 2011, June 2014, June 2015 • Visiting Professor, Institute Henri Poincaré, France. November, 1998 • Visiting Professor, Brown University. 1990,1994,1995-2001 • Newton R. and Sarah L. Wilson Chair in Biomedical Engineering, Washington University. 1995-1998 • Professor, Washington University St. Louis. 1992 - 1998 • Visiting Professor, Université René Descartes. November. 1997 • Associate Professor, Washington University St.
    [Show full text]
  • Generating Open World Descriptions of Video Using Commonsense
    QUARTERLY OF APPLIED MATHEMATICS VOLUME LXXVII, NUMBER 2 JUNE 2019, PAGES 323–356 https://doi.org/10.1090/qam/1530 Article electronically published on January 11, 2019 GENERATING OPEN WORLD DESCRIPTIONS OF VIDEO USING COMMON SENSE KNOWLEDGE IN A PATTERN THEORY FRAMEWORK By SATHYANARAYANAN N. AAKUR (Department of Computer Science and Engineering, University of South Florida, Tampa, Florida 33620 ), FILLIPE DM DE SOUZA (Department of Computer Science and Engineering, University of South Florida, Tampa, Florida 33620), and SUDEEP SARKAR (Department of Computer Science and Engineering, University of South Florida, Tampa, Florida 33620 ) This paper is dedicated to Professor Ulf Grenander Abstract. The task of interpretation of activities as captured in video extends beyond just the recognition of observed actions and objects. It involves open world reasoning and constructing deep semantic connections that go beyond what is directly observed in the video and annotated in the training data. Prior knowledge plays a big role. Grenander’s canonical pattern theory representation offers an elegant mechanism to capture these semantic connections between what is observed directly in the image and past knowl- edge in large-scale common sense knowledge bases, such as ConceptNet. We represent interpretations using a connected structure of basic detected (grounded) concepts, such as objects and actions, that are bound by semantics with other background concepts not directly observed, i.e., contextualization cues. Concepts are basic generators and the bonds are defined by the semantic relationships between concepts. Local and global regularity constraints govern these bonds and the overall connection structure. We use an inference engine based on energy minimization using an efficient Markov Chain Monte Carlo that uses the ConceptNet in its move proposals to find these structures that de- scribe the image content.
    [Show full text]
  • NEW BOOKS Pattern Theory: the Stochastic Analysis of Real-World
    QUARTERLY OF APPLIED MATHEMATICS Volume 69, Number 2, June 2011, Pages 403–404 S 0033-569X(2011)01270-6 Article electronically published on March 23, 2011 NEW BOOKS Pattern Theory: The stochastic analysis of real-world signals. By David Mumford and Ag- nes Desolneux, A K Peters, Ltd., 407 pp., $79.00 This is a remarkable work. It is full of sophisticated mathematical models dealing with all sorts of phenomena from the physical as well as from the man-made world. Or, to use the author’s parlance, “analysis of real-world signals”. Much work in statistics, pattern recognition, etc. is based on highly simplified models and reflect more the researcher’s statistical expertise than an attempt at real understanding of the phenomena studies. In contrast, this book goes deeper and presents serious attempts to really dig deeply into the subject matter of the signals being analyzed. At the same time it does not avoid the mathematical difficulties in the construction of reality-based models. Indeed, each chapter contains a section called Basics, in which the reader is being prepared to use the mathematical tools suitable for a particular application. These tools are often of an advanced character. Even elementary ideas are presented from a higher and unifying perspective. Or to quote Felix Klein: vom h¨oheren Standpunkt aus. While expertly guiding the reader to pattern theoretic thinking, this book also entertains by presenting additional material in the form of some delightful mathematics in the Basics sections. One of the first types of signals to be studied in an early chapter is English text (written) beginning with Claude Shannon’s thought experiment with simulated text.
    [Show full text]
  • Pattern Theory: the Mathematics of Perception
    ICM 2002 · Vol. III · 1–3 Pattern Theory: the Mathematics of Perception David Mumford∗ 1. Introduction How can we understand intelligent behavior? How can we design intelligent computers? These are questions that have been discussed by scientists and the public at large for over 50 years. As mathematicians, however, the question we want to ask is “is there a mathematical theory underlying intelligence?” I believe the first mathematical attack on these issues was Control Theory, led by Wiener and Pontryagin. They were studying how to design a controller which drives a motor affecting the world and also sits in a feedback loop receiving measurements from the world about the effect of the motor action. The goal was to control the motor so that the world, as measured, did something specific, i.e. move the tiller so that the boat stays on course. The main complication is that nothing is precisely predictable: the motor control is not exact, the world does unexpected things because of its complexities and the measurements you take of it are imprecise. All this led, in the simplest case, to a beautiful analysis known as the Wiener-Kalman-Bucy filter (to be described below). But Control Theory is basically a theory of the output side of intelligence with the measurements modeled in the simplest possible way: e.g. linear functions of the state of the world system being controlled plus additive noise. The real input side of intelligence is perception in a much broader sense, the analysis of all the noisy incomplete signals which you can pick up from the world through natural or artificial senses.
    [Show full text]
  • Discriminative, Descriptive and Generative Models
    QUARTERLY OF APPLIED MATHEMATICS VOLUME , NUMBER 0 XXXX XXXX, PAGES 000–000 A TALE OF THREE PROBABILISTIC FAMILIES: DISCRIMINATIVE, DESCRIPTIVE AND GENERATIVE MODELS BY YING NIAN WU (Department of Statistics, University of California, Los Angeles), RUIQI GAO (Department of Statistics, University of California, Los Angeles), TIAN HAN (Department of Statistics, University of California, Los Angeles), AND SONG-CHUN ZHU (Department of Statistics, University of California, Los Angeles) Abstract. The pattern theory of Grenander is a mathematical framework where patterns are represented by probability models on random variables of algebraic structures. In this paper, we review three families of probability models, namely, the discriminative models, the descriptive models, and the generative models. A discriminative model is in the form of a classifier. It specifies the con- ditional probability of the class label given the input signal. A descriptive model specifies the probability distribution of the signal, based on an energy function defined on the signal. A gen- erative model assumes that the signal is generated by some latent variables via a transformation. We shall review these models within a common framework and explore their connections. We shall also review the recent developments that take advantage of the high approximation capaci- ties of deep neural networks. 1. Introduction. Initially developed by Grenander in the 1970s, the pattern theory [30, 31] is a unified mathematical framework for representing, learning and recognizing patterns that arise in science and engineering. The objects in pattern theory are usually of high complexity or dimensionality, defined in terms of the constituent elements and the bonds between them. The patterns of these objects are characterized by both the algebraic structures governed by local and global rules, as well as the probability distributions of the associated random variables.
    [Show full text]
  • Markov Chain Monte Carlo Methods for Statistical Inference Julian Besag1 Department of Statistics University of Washington, Seattle, USA
    Markov Chain Monte Carlo Methods for Statistical Inference Julian Besag1 Department of Statistics University of Washington, Seattle, USA Spring 2004 SUMMARY These notes provide an introduction to Markov chain Monte Carlo methods and their applications to both Bayesian and frequentist statistical inference. Such methods have revolutionized what can be achieved computationally, es- pecially in the Bayesian paradigm. The account begins by discussing ordi- nary Monte Carlo methods: these have the same goals as the Markov chain versions but can only rarely be implemented. Subsequent sections describe basic Markov chain Monte Carlo, based on the Hastings algorithm and in- cluding both the Metropolis method and the Gibbs sampler as special cases, and go on to discuss some more specialized developments, including adaptive slice sampling, exact goodness{of{¯t tests, maximum likelihood estimation, the Langevin{Hastings algorithm, auxiliary variables techniques, perfect sam- pling via coupling from the past, reversible jumps methods for target spaces of varying dimensions, and simulated annealing. Specimen applications are described throughout the notes. Keywords: Adaptive slice sampling; Autologistic distribution; Auxiliary variables; Bayesian compu- tation; Competing risks; Contingency tables; Darwin's ¯nches; Exact p{values; Gibbs sampler; Hastings algorithm; Hidden Markov models; Importance sampling; Ising model; Langevin di®usion; Markov chain Monte Carlo; Markov random ¯elds; Maximum likelihood estimation; Metropolis method; Mixture mod- els;
    [Show full text]