Generalized Linear Models & Logistic Regression Jonathan Pillow Mathematical Tools for Neuroscience (NEU 314) Spring, 2016 lecture 17 Example 3: unknown neuron 100 75 50 (spike count) 25 0 -25 0 25 (contrast) What model would you use to fit this neuron? Linear-Nonlinear-Poisson model stimulus filter nonlinearity Poisson spiking λ(t) stimulus k f conditional intensity (“spike rate”) Poisson spiking • example of generalized linear model (GLM) Aside on GLMs: 1. Be careful about terminology: GLM ≠ GLM General Linear Model Generalized Linear Model (Nelder 1972) Linear Linear 2003 interview with John Nelder... Stephen Senn: I must confess to having some confusion when I was a young statistician between general linear models and generalized linear models. Do you regret the terminology? John Nelder: I think probably I do. I suspect we should have found some more fancy name for it that would have stuck and not been confused with the general linear model, although general and generalized are not quite the same. I can see why it might have been better to have thought of something else. Senn, (2003). Statistical Science Moral: Be careful when naming your model! 2. General Linear Model Linear Noise (exponential family) “Dimensionality Reduction” Examples: 1. Gaussian 2. Poisson 3. Generalized Linear Model Linear Nonlinear Noise (exponential family) Examples: 1. Gaussian 2. Poisson Linear-Nonlinear-Poisson stimulus filter exponential Poisson nonlinearity spiking stimulus k f λ(t) conditional intensity (“spike rate”) • output: Poisson process (Truccolo et al 04) GLM with spike-history dependence stimulus filter exponential probabilistic nonlinearity spiking stimulus + post-spike filter conditional intensity (spike rate) • output: product of stimulus and spike-history term GLM dynamic behaviors • irregular spiking (Poisson process) stimulus filter outputs (“currents”) p(spike) post-spike filter h(t) GLM dynamic behaviors • regular spiking stimulus filter outputs (“currents”) p(spike) post-spike filter h(t) GLM dynamic behaviors • bursting stimulus filter outputs (“currents”) p(spike) post-spike filter h(t) GLM dynamic behaviors • adaptation stimulus filter outputs (“currents”) p(spike) post-spike filter h(t) multi-neuron GLM stimulus filter exponential probabilistic nonlinearity spiking + post-spike filter neuron 1 stimulus + neuron 2 multi-neuron GLM stimulus filter exponential probabilistic nonlinearity spiking + post-spike filter neuron 1 stimulus coupling filters + neuron 2 GLM equivalent diagram: ... time t conditional intensity (spike rate) Logistic Regression GLM for binary classification weights sigmoid Bernoulli input “0” or “1” 1. Linear weights 2. sigmoid (“logistic”) function 3. Bernoulli (coin flip) Logistic Regression GLM for binary classification weights sigmoid Bernoulli input “0” or “1” compact expression: (note when y = 1, this is equal to exp(wx)/(1+exp(wx)), which is equal to 1/(1+exp(-wx) ) Logistic Regression GLM for binary classification weights sigmoid Bernoulli input “0” or “1” compact expression: fit w by maximizing log-likelihood: Logistic Regression geometric view p = 0.5 contour (classification boundary) neuron 2spikes neuron 1 spikes Bayesian Estimation three basic ingredients: 1. Likelihood jointly determine the posterior 2. Prior “cost” of making an estimate 3. Loss function L(✓ˆ, ✓) if the true value is • fully specifies how to generate an estimate from the data Bayesian estimator is defined as: ✓ˆ(m) = arg min L(✓ˆ, ✓)p(✓ m)d✓ “Bayes’ risk” ˆ | ✓ Z Typical Loss functions and Bayesian estimators 2 1. L(✓ˆ, ✓)=(✓ˆ ✓) squared error loss − 0 need to find minimizing the expected loss: Differentiate with respect to and set to zero: “posterior mean” also known as Bayes’ Least Squares (BLS) estimator Typical Loss functions and Bayesian estimators ˆ ˆ “zero-one” loss 2. L(✓, ✓)=1 δ(✓ ✓) (1 unless ) − − 0 expected loss: which is minimized by: • posterior maximum (or “mode”). • known as maximum a posteriori (MAP) estimate..