DOCSLIB.ORG

Machine Learning Classification

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Machine Learning Classification Machine Learning 10-701/15-781, Spring 2008 Naïve Bayes Classifier Eric Xing Lecture 3, January 23, 2006 Reading: Chap. 4 CB and handouts Classification z Representing data: z Choosing hypothesis class z Learning: h:X a Y z X – features z Y – target classes 1 Suppose you know the following … z Classification-specific Dist.: P(X|Y) p(X |Y = 1) r = p1 (X ;µ1 ,Σ1 ) p(X | Y = 2) r = p2 (X ; µ2 ,Σ2 ) Bayes classifier: z Class prior (i.e., "weight"): P(Y) z This is a generative model of the data! Optimal classification z Theorem: Bayes classifier is optimal! z That is z Proof: z Recall last lecture: z How to learn a Bayes classifier? 2 Recall Bayes Rule Which is shorthand for: Equivalently: Recall Bayes Rule Which is shorthand for: Common abbreviation: 3 Learning Bayes Classifier z Training data: z Learning = estimating P(X|Y), and P(Y) z Classification = using Bayes rule to calculate P(Y | Xnew) How hard is it to learn the optimal classifier? z How do we represent these? How many parameters? z Prior, P(Y): z Suppose Y is composed of k classes z Likelihood, P(X|Y): z Suppose X is composed of n binary features z Complex model ! High variance with limited data!!! 4 Naïve Bayes: assuming that Xi and Xj are conditionally independent given Y, for all i≠j Conditional Independence z X is conditionally independent of Y given Z, if the probability distribution governing X is independent of the value of Y, given the value of Z Which we often write z e.g., z Equivalent to: 5 The Naïve Bayes assumption z Naïve Bayes assumption: z Features are independent given class: z More generally: z How many parameters now? z Suppose X is composed of n binary features The Naïve Bayes Classifier z Given: z Prior P(Y) z n conditionally independent features X given the class Y z For each Xi, we have likelihood P(Xi|Y) z Decision rule: z If assumption holds, NB is optimal classifier! 6 Naïve Bayes Algorithm z Train Naïve Bayes (examples) z for each* value yk z estimate z for each* value xij of each attribute Xi z estimate z Classify (Xnew) * probabilities must sum to 1, so need estimate only n-1 parameters... Learning NB: parameter estimation z Maximum Likelihood Estimate (MLE): choose θ that maximizes probability of observed data D z Maximum a Posteriori (MAP) estimate: choose θ that is most probable given prior probability and the data 7 MLE for the parameters of NB Discrete features: z Maximum likelihood estimates (MLE’s): z Given dataset z Count(A=a,B=b) ≡ number of examples where A=a and B=b Subtleties of NB classifier 1 – Violating the NB assumption z Often the Xi are not really conditionally independent z We use Naïve Bayes in many cases anyway, and it often works pretty well z often the right classification, even when not the right probability (see [Domingos&Pazzani, 1996]) z But the resulting probabilities P(Y|Xnew) are biased toward 1 or 0 (why?) 8 Subtleties of NB classifier 2 – Insufficient training data z What if you never see a training instance where X1000=a when Y=b? z e.g., Y={SpamEmail or not}, X1000={‘Rolex’} z P(X1000=T | Y=T) = 0 z Thus, no matter what the values X2,…,Xn take: z P(Y=T | X1,X2,…,X1000=T, …, Xn) = 0 z What now??? MAP for the parameters of NB Discrete features: z Maximum a Posteriori (MAP) estimate: (MAP’s): z Given prior: z Consider binary feature z θ is a Bernoulli rate αT −1 α F −1 Γ(αT +α F ) αT −1 α F −1 θ (1 −θ ) P(θ;αT ,α F ) = θ (1 −θ ) = Γ(αT )Γ(α F ) B(αT ,α F ) z Let βa=Count(X=a) ≡ number of examples where X=a 9 Bayesian learning for NB parameters – a.k.a. smoothing z Posterior distribution of θ z Bernoulli: z Multinomial z MAP estimate: z Beta prior equivalent to extra thumbtack flips z As N → 1, prior is “forgotten” z But, for small sample size, prior is important! MAP for the parameters of NB z Dataset of N examples z Let βiab=Count(Xi=a,Y=b) ≡ number of examples where Xi=a and Y=b z Let γb=Count(Y=b) z Prior Q(Xi|Y) ∝ Multinomial(αi1, …, αiK) or Multinomial(α/K) Q(Y) ∝ Multinomial(τi1, …, τiM) or Multinomial(τ/M) m “virtual” examples z MAP estimate z Now, even if you never observe a feature/class, posterior probability never zero 10 Text classification z Classify e-mails z Y = {Spam,NotSpam} z Classify news articles z Y = {what is the topic of the article?} z Classify webpages z Y = {Student, professor, project, …} z What about the features X? z The text! Features X are entire document – Xi for ith word in article 11 NB for Text classification z P(X|Y) is huge!!! z Article at least 1000 words, X={X1,…,X1000} th z Xi represents i word in document, i.e., the domain of Xi is entire vocabulary, e.g., Webster Dictionary (or more), 10,000 words, etc. z NB assumption helps a lot!!! z P(Xi=xi|Y=y) is just the probability of observing word xi in a document on topic y Bag of words model z Typical additional assumption – Position in document doesn’t matter: P(Xi=xi|Y=y) = P(Xk=xi|Y=y) z “Bag of words” model – order of words on the page ignored z Sounds really silly, but often works very well! When the lecture is over, remember to wake up the person sitting next to you in the lecture room. 12 Bag of words model z Typical additional assumption – Position in document doesn’t matter: P(Xi=xi|Y=y) = P(Xk=xi|Y=y) z “Bag of words” model – order of words on the page ignored z Sounds really silly, but often works very well! in is lecture lecture next over person remember room sitting the the the to to up wake when you Bag of Words Approach aardvark 0 about 2 all 2 Africa 1 apple 0 anxious 0 ... gas 1 ... oil 1 … Zaire 0 13 NB with Bag of Words for text classification z Learning phase: z Prior P(Y) z Count how many documents you have from each topic (+ prior) z P(Xi|Y) z For each topic, count how many times you saw word in documents of this topic (+ prior) z Test phase: z For each document z Use naïve Bayes decision rule Twenty News Groups results 14 Learning curve for Twenty News Groups What if we have continuous Xi ? th z Eg., character recognition: Xi is i pixel z Gaussian Naïve Bayes (GNB): Sometimes assume variance z is independent of Y (i.e., σi), z or independent of Xi (i.e., σk) z or both (i.e., σ) 15 Estimating Parameters: Y discrete, Xi continuous z Maximum likelihood estimates: jth training example δ(x)=1 if x true, else 0 Gaussian Naïve Bayes 16 Example: GNB for classifying mental states [Mitchell et al.] ~1 mm resolution ~2 images per sec. 15,000 voxels/image non-invasive, safe measures Blood Oxygen Level 10 sec Dependent (BOLD) response Typical impulse response Brain scans can track activation with precision and sensitivity [Mitchell et al.] 17 Gaussian Naïve Bayes: Learned µvoxel,word P(BrainActivity | WordCategory = {People,Animal})[Mitchell et al.] Learned Bayes Models – Means for P(BrainActivity | WordCategory) [Mitchell et al.] Pairwise classification accuracy: 85% People words Animal words 18 What you need to know about Naïve Bayes z Optimal decision using Bayes Classifier z Naïve Bayes classifier z What’s the assumption z Why we use it z How do we learn it z Why is Bayesian estimation important z Text classification z Bag of words model z Gaussian NB z Features are still conditionally independent z Each feature has a Gaussian distribution given class 19.
Load more

Recommended publications
  • Practical Statistics for Particle Physics Lecture 1 AEPS2018, Quy Nhon, Vietnam
    Practical Statistics for Particle Physics Lecture 1 AEPS2018, Quy Nhon, Vietnam Roger Barlow The University of Huddersfield August 2018 Roger Barlow ( Huddersfield) Statistics for Particle Physics August 2018 1 / 34 Lecture 1: The Basics 1 Probability What is it? Frequentist Probability Conditional Probability and Bayes' Theorem Bayesian Probability 2 Probability distributions and their properties Expectation Values Binomial, Poisson and Gaussian 3 Hypothesis testing Roger Barlow ( Huddersfield) Statistics for Particle Physics August 2018 2 / 34 Question: What is Probability? Typical exam question Q1 Explain what is meant by the Probability PA of an event A [1] Roger Barlow ( Huddersfield) Statistics for Particle Physics August 2018 3 / 34 Four possible answers PA is number obeying certain mathematical rules. PA is a property of A that determines how often A happens For N trials in which A occurs NA times, PA is the limit of NA=N for large N PA is my belief that A will happen, measurable by seeing what odds I will accept in a bet. Roger Barlow ( Huddersfield) Statistics for Particle Physics August 2018 4 / 34 Mathematical Kolmogorov Axioms: For all A ⊂ S PA ≥ 0 PS = 1 P(A[B) = PA + PB if A \ B = ϕ and A; B ⊂ S From these simple axioms a complete and complicated structure can be − ≤ erected. E.g. show PA = 1 PA, and show PA 1.... But!!! This says nothing about what PA actually means. Kolmogorov had frequentist probability in mind, but these axioms apply to any definition. Roger Barlow ( Huddersfield) Statistics for Particle Physics August 2018 5 / 34 Classical or Real probability Evolved during the 18th-19th century Developed (Pascal, Laplace and others) to serve the gambling industry.
    [Show full text]
  • 3.3 Bayes' Formula
    Ismor Fischer, 5/29/2012 3.3-1 3.3 Bayes’ Formula Suppose that, for a certain population of individuals, we are interested in comparing sleep disorders – in particular, the occurrence of event A = “Apnea” – between M = Males and F = Females. S = Adults under 50 M F A A ∩ M A ∩ F Also assume that we know the following information: P(M) = 0.4 P(A | M) = 0.8 (80% of males have apnea) prior probabilities P(F) = 0.6 P(A | F) = 0.3 (30% of females have apnea) Given here are the conditional probabilities of having apnea within each respective gender, but these are not necessarily the probabilities of interest. We actually wish to calculate the probability of each gender, given A. That is, the posterior probabilities P(M | A) and P(F | A). To do this, we first need to reconstruct P(A) itself from the given information. P(A | M) P(A ∩ M) = P(A | M) P(M) P(M) P(Ac | M) c c P(A ∩ M) = P(A | M) P(M) P(A) = P(A | M) P(M) + P(A | F) P(F) P(A | F) P(A ∩ F) = P(A | F) P(F) P(F) P(Ac | F) c c P(A ∩ F) = P(A | F) P(F) Ismor Fischer, 5/29/2012 3.3-2 So, given A… P(M ∩ A) P(A | M) P(M) P(M | A) = P(A) = P(A | M) P(M) + P(A | F) P(F) (0.8)(0.4) 0.32 = (0.8)(0.4) + (0.3)(0.6) = 0.50 = 0.64 and posterior P(F ∩ A) P(A | F) P(F) P(F | A) = = probabilities P(A) P(A | M) P(M) + P(A | F) P(F) (0.3)(0.6) 0.18 = (0.8)(0.4) + (0.3)(0.6) = 0.50 = 0.36 S Thus, the additional information that a M F randomly selected individual has apnea (an A event with probability 50% – why?) increases the likelihood of being male from a prior probability of 40% to a posterior probability 0.32 0.18 of 64%, and likewise, decreases the likelihood of being female from a prior probability of 60% to a posterior probability of 36%.
    [Show full text]
  • Naïve Bayes Classifier
    CS 60050 Machine Learning Naïve Bayes Classifier Some slides taken from course materials of Tan, Steinbach, Kumar Bayes Classifier ● A probabilistic framework for solving classification problems ● Approach for modeling probabilistic relationships between the attribute set and the class variable – May not be possible to certainly predict class label of a test record even if it has identical attributes to some training records – Reason: noisy data or presence of certain factors that are not included in the analysis Probability Basics ● P(A = a, C = c): joint probability that random variables A and C will take values a and c respectively ● P(A = a | C = c): conditional probability that A will take the value a, given that C has taken value c P(A,C) P(C | A) = P(A) P(A,C) P(A | C) = P(C) Bayes Theorem ● Bayes theorem: P(A | C)P(C) P(C | A) = P(A) ● P(C) known as the prior probability ● P(C | A) known as the posterior probability Example of Bayes Theorem ● Given: – A doctor knows that meningitis causes stiff neck 50% of the time – Prior probability of any patient having meningitis is 1/50,000 – Prior probability of any patient having stiff neck is 1/20 ● If a patient has stiff neck, what’s the probability he/she has meningitis? P(S | M )P(M ) 0.5×1/50000 P(M | S) = = = 0.0002 P(S) 1/ 20 Bayesian Classifiers ● Consider each attribute and class label as random variables ● Given a record with attributes (A1, A2,…,An) – Goal is to predict class C – Specifically, we want to find the value of C that maximizes P(C| A1, A2,…,An ) ● Can we estimate
    [Show full text]
  • Numerical Physics with Probabilities: the Monte Carlo Method and Bayesian Statistics Part I for Assignment 2
    Numerical Physics with Probabilities: The Monte Carlo Method and Bayesian Statistics Part I for Assignment 2 Department of Physics, University of Surrey module: Energy, Entropy and Numerical Physics (PHY2063) 1 Numerical Physics part of Energy, Entropy and Numerical Physics This numerical physics course is part of the second-year Energy, Entropy and Numerical Physics mod- ule. It is online at the EENP module on SurreyLearn. See there for assignments, deadlines etc. The course is about numerically solving ODEs (ordinary differential equations) and PDEs (partial differential equations), and introducing the (large) part of numerical physics where probabilities are used. This assignment is on numerical physics of probabilities, and looks at the Monte Carlo (MC) method, and at the Bayesian statistics approach to data analysis. It covers MC and Bayesian statistics, in that order. MC is a widely used numerical technique, it is used, amongst other things, for modelling many random processes. MC is used in fields from statistical physics, to nuclear and particle physics. Bayesian statistics is a powerful data analysis method, and is used everywhere from particle physics to spam-email filters. Data analysis is fundamental to science. For example, analysis of the data from the Large Hadron Collider was required to extract a most probable value for the mass of the Higgs boson, together with an estimate of the region of masses where the scientists think the mass is. This region is typically expressed as a range of mass values where the they think the true mass lies with high (e.g., 95%) probability. Many of you will be analysing data (physics data, commercial data, etc) for your PTY or RY, or future careers1 .
    [Show full text]
  • The Bayesian Approach to Statistics
    THE BAYESIAN APPROACH TO STATISTICS ANTHONY O’HAGAN INTRODUCTION the true nature of scientific reasoning. The fi- nal section addresses various features of modern By far the most widely taught and used statisti- Bayesian methods that provide some explanation for the rapid increase in their adoption since the cal methods in practice are those of the frequen- 1980s. tist school. The ideas of frequentist inference, as set out in Chapter 5 of this book, rest on the frequency definition of probability (Chapter 2), BAYESIAN INFERENCE and were developed in the first half of the 20th century. This chapter concerns a radically differ- We first present the basic procedures of Bayesian ent approach to statistics, the Bayesian approach, inference. which depends instead on the subjective defini- tion of probability (Chapter 3). In some respects, Bayesian methods are older than frequentist ones, Bayes’s Theorem and the Nature of Learning having been the basis of very early statistical rea- Bayesian inference is a process of learning soning as far back as the 18th century. Bayesian from data. To give substance to this statement, statistics as it is now understood, however, dates we need to identify who is doing the learning and back to the 1950s, with subsequent development what they are learning about. in the second half of the 20th century. Over that time, the Bayesian approach has steadily gained Terms and Notation ground, and is now recognized as a legitimate al- ternative to the frequentist approach. The person doing the learning is an individual This chapter is organized into three sections.
    [Show full text]
  • LINEAR REGRESSION MODELS Likelihood and Reference Bayesian
    { LINEAR REGRESSION MODELS { Likeliho o d and Reference Bayesian Analysis Mike West May 3, 1999 1 Straight Line Regression Mo dels We b egin with the simple straight line regression mo del y = + x + i i i where the design p oints x are xed in advance, and the measurement/sampling i 2 errors are indep endent and normally distributed, N 0; for each i = i i 1;::: ;n: In this context, wehavelooked at general mo delling questions, data and the tting of least squares estimates of and : Now we turn to more formal likeliho o d and Bayesian inference. 1.1 Likeliho o d and MLEs The formal parametric inference problem is a multi-parameter problem: we 2 require inferences on the three parameters ; ; : The likeliho o d function has a simple enough form, as wenow show. Throughout, we do not indicate the design p oints in conditioning statements, though they are implicitly conditioned up on. Write Y = fy ;::: ;y g and X = fx ;::: ;x g: Given X and the mo del 1 n 1 n parameters, each y is the corresp onding zero-mean normal random quantity i plus the term + x ; so that y is normal with this term as its mean and i i i 2 variance : Also, since the are indep endent, so are the y : Thus i i 2 2 y j ; ; N + x ; i i with conditional density function 2 2 2 2 1=2 py j ; ; = expfy x =2 g=2 i i i for each i: Also, by indep endence, the joint density function is n Y 2 2 : py j ; ; = pY j ; ; i i=1 1 Given the observed resp onse values Y this provides the likeliho o d function for the three parameters: the joint density ab ove evaluated at the observed and 2 hence now xed values of Y; now viewed as a function of ; ; as they vary across p ossible parameter values.
    [Show full text]
  • Paradoxes and Priors in Bayesian Regression
    Paradoxes and Priors in Bayesian Regression Dissertation Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the Graduate School of The Ohio State University By Agniva Som, B. Stat., M. Stat. Graduate Program in Statistics The Ohio State University 2014 Dissertation Committee: Dr. Christopher M. Hans, Advisor Dr. Steven N. MacEachern, Co-advisor Dr. Mario Peruggia c Copyright by Agniva Som 2014 Abstract The linear model has been by far the most popular and most attractive choice of a statistical model over the past century, ubiquitous in both frequentist and Bayesian literature. The basic model has been gradually improved over the years to deal with stronger features in the data like multicollinearity, non-linear or functional data pat- terns, violation of underlying model assumptions etc. One valuable direction pursued in the enrichment of the linear model is the use of Bayesian methods, which blend information from the data likelihood and suitable prior distributions placed on the unknown model parameters to carry out inference. This dissertation studies the modeling implications of many common prior distri- butions in linear regression, including the popular g prior and its recent ameliorations. Formalization of desirable characteristics for model comparison and parameter esti- mation has led to the growth of appropriate mixtures of g priors that conform to the seven standard model selection criteria laid out by Bayarri et al. (2012). The existence of some of these properties (or lack thereof) is demonstrated by examining the behavior of the prior under suitable limits on the likelihood or on the prior itself.
    [Show full text]
  • A Widely Applicable Bayesian Information Criterion
    JournalofMachineLearningResearch14(2013)867-897 Submitted 8/12; Revised 2/13; Published 3/13 A Widely Applicable Bayesian Information Criterion Sumio Watanabe [email protected] Department of Computational Intelligence and Systems Science Tokyo Institute of Technology Mailbox G5-19, 4259 Nagatsuta, Midori-ku Yokohama, Japan 226-8502 Editor: Manfred Opper Abstract A statistical model or a learning machine is called regular if the map taking a parameter to a prob- ability distribution is one-to-one and if its Fisher information matrix is always positive definite. If otherwise, it is called singular. In regular statistical models, the Bayes free energy, which is defined by the minus logarithm of Bayes marginal likelihood, can be asymptotically approximated by the Schwarz Bayes information criterion (BIC), whereas in singular models such approximation does not hold. Recently, it was proved that the Bayes free energy of a singular model is asymptotically given by a generalized formula using a birational invariant, the real log canonical threshold (RLCT), instead of half the number of parameters in BIC. Theoretical values of RLCTs in several statistical models are now being discovered based on algebraic geometrical methodology. However, it has been difficult to estimate the Bayes free energy using only training samples, because an RLCT depends on an unknown true distribution. In the present paper, we define a widely applicable Bayesian information criterion (WBIC) by the average log likelihood function over the posterior distribution with the inverse temperature 1/logn, where n is the number of training samples. We mathematically prove that WBIC has the same asymptotic expansion as the Bayes free energy, even if a statistical model is singular for or unrealizable by a statistical model.
    [Show full text]
  • Part IV: Monte Carlo and Nonparametric Bayes Outline
    Part IV: Monte Carlo and nonparametric Bayes Outline Monte Carlo methods Nonparametric Bayesian models Outline Monte Carlo methods Nonparametric Bayesian models The Monte Carlo principle • The expectation of f with respect to P can be approximated by 1 n E P(x)[ f (x)] " # f (xi ) n i=1 where the xi are sampled from P(x) • Example: the average # of spots on a die roll ! The Monte Carlo principle The law of large numbers n E P(x)[ f (x)] " # f (xi ) i=1 Average number of spots ! Number of rolls Two uses of Monte Carlo methods 1. For solving problems of probabilistic inference involved in developing computational models 2. As a source of hypotheses about how the mind might solve problems of probabilistic inference Making Bayesian inference easier P(d | h)P(h) P(h | d) = $P(d | h ") P(h ") h " # H Evaluating the posterior probability of a hypothesis requires considering all hypotheses ! Modern Monte Carlo methods let us avoid this Modern Monte Carlo methods • Sampling schemes for distributions with large state spaces known up to a multiplicative constant • Two approaches: – importance sampling (and particle filters) – Markov chain Monte Carlo Importance sampling Basic idea: generate from the wrong distribution, assign weights to samples to correct for this E p(x)[ f (x)] = " f (x)p(x)dx p(x) = f (x) q(x)dx " q(x) n ! 1 p(xi ) " # f (xi ) for xi ~ q(x) n i=1 q(xi ) ! ! Importance sampling works when sampling from proposal is easy, target is hard An alternative scheme… n 1 p(xi ) E p(x)[ f (x)] " # f (xi ) for xi ~ q(x) n i=1 q(xi ) n p(xi
    [Show full text]
  • Random Forest Classifiers
    Random Forest Classifiers Carlo Tomasi 1 Classification Trees A classification tree represents the probability space P of posterior probabilities p(yjx) of label given feature by a recursive partition of the feature space X, where each partition is performed by a test on the feature x called a split rule. To each set of the partition is assigned a posterior probability distribution, and p(yjx) for a feature x 2 X is then defined as the probability distribution associated with the set of the partition that contains x. A popular split rule called a 1-rule partitions a set S ⊆ X × Y into the two sets L = f(x; y) 2 S j xd ≤ tg and R = f(x; y) 2 S j xd > tg (1) where xd is the d-th component of x and t is a real number. This note considers only binary classification trees1 with 1-rule splits, and the word “binary” is omitted in what follows. Concretely, the split rules are placed on the interior nodes of a binary tree and the probability distri- butions are on the leaves. The tree τ can be defined recursively as either a single (leaf) node with values of the posterior probability p(yjx) collected in a vector τ:p or an (interior) node with a split function with parameters τ:d and τ:t that returns either the left or the right descendant (τ:L or τ:R) of τ depending on the outcome of the split. A classification tree τ then takes a new feature x, looks up its posterior in the tree by following splits in τ down to a leaf, and returns the MAP estimate, as summarized in Algorithm 1.
    [Show full text]
  • Marginal Likelihood
    STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! [email protected]! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 2 Last Class •" In our last class, we looked at: -" Statistical Decision Theory -" Linear Regression Models -" Linear Basis Function Models -" Regularized Linear Regression Models -" Bias-Variance Decomposition •" We will now look at the Bayesian framework and Bayesian Linear Regression Models. Bayesian Approach •" We formulate our knowledge about the world probabilistically: -" We define the model that expresses our knowledge qualitatively (e.g. independence assumptions, forms of distributions). -" Our model will have some unknown parameters. -" We capture our assumptions, or prior beliefs, about unknown parameters (e.g. range of plausible values) by specifying the prior distribution over those parameters before seeing the data. •" We observe the data. •" We compute the posterior probability distribution for the parameters, given observed data. •" We use this posterior distribution to: -" Make predictions by averaging over the posterior distribution -" Examine/Account for uncertainly in the parameter values. -" Make decisions by minimizing expected posterior loss. (See Radford Neal’s NIPS tutorial on ``Bayesian Methods for Machine Learning'’) Posterior Distribution •" The posterior distribution for the model parameters can be found by combining the prior with the likelihood for the parameters given the data. •" This is accomplished using Bayes’
    [Show full text]
  • Bayesian Inference
    Bayesian Inference Thomas Nichols With thanks Lee Harrison Bayesian segmentation Spatial priors Posterior probability Dynamic Causal and normalisation on activation extent maps (PPMs) Modelling Attention to Motion Paradigm Results SPC V3A V5+ Attention – No attention Büchel & Friston 1997, Cereb. Cortex Büchel et al. 1998, Brain - fixation only - observe static dots + photic V1 - observe moving dots + motion V5 - task on moving dots + attention V5 + parietal cortex Attention to Motion Paradigm Dynamic Causal Models Model 1 (forward): Model 2 (backward): attentional modulation attentional modulation of V1→V5: forward of SPC→V5: backward Photic SPC Attention Photic SPC V1 V1 - fixation only V5 - observe static dots V5 - observe moving dots Motion Motion - task on moving dots Attention Bayesian model selection: Which model is optimal? Responses to Uncertainty Long term memory Short term memory Responses to Uncertainty Paradigm Stimuli sequence of randomly sampled discrete events Model simple computational model of an observers response to uncertainty based on the number of past events (extent of memory) 1 2 3 4 Question which regions are best explained by short / long term memory model? … 1 2 40 trials ? ? Overview • Introductory remarks • Some probability densities/distributions • Probabilistic (generative) models • Bayesian inference • A simple example – Bayesian linear regression • SPM applications – Segmentation – Dynamic causal modeling – Spatial models of fMRI time series Probability distributions and densities k=2 Probability distributions
    [Show full text]