Introduction to Neural Computation Prof. Michale Fee MIT BCS 9.40 — 2018 Lecture 9 — Receptive fields 1 Spatial receptive fields Video of visual neurons of a cat from Hubel & Wiesel’s experiments. LIGHT Ali Moeeny. “ Hubel & Wiesel – LGN Neuron.” April 23, 2011. YouTube. Figure (annotated) from Introduction to Visual Prostheses on Webvision. LicenseC C BY-NC. 2 Spatial receptive fields Video of visual neurons of a cat – simple and complex cells - from Hubel & Wiesel’s experiments. LIGHT Ali Moeeny. “ Hubel & Wiesel – Cortical Neuron – V1.” April 23, 2011. YouTube. Figure (annotated) from Introduction to Visual Prostheses on Webvision. LicenseC C BY-NC. 3 Learning objectives for Lecture 9 • To be able to mathematically describe a neural response as a linear filter followed by a nonlinear function. – A correlation of a spatial receptive field with the stimulus – A convolution of a temporal receptive field with the stimulus • To understand the concept of a Spatio-temporal Receptive Field (STRF) and the concept of ‘separability’ • To understand the idea of a Spike Triggered Average and how to use it to compute a Spatio-temporal Receptive Field and a Spectro-temporal Receptive Field (STRF). 4 Learning objectives for Lecture 9 • To be able to mathematically describe a neural response as a linear filter followed by a nonlinear function. – A correlation of a spatial receptive field with the stimulus – A convolution of a temporal receptive field with the stimulus • To understand the concept of a Spatio-temporal Receptive Field (STRF) and the concept of ‘separability’ • To understand the idea of a Spike Triggered Average and how to use it to compute a Spatio-temporal Receptive Field and a Spectro-temporal Receptive Field (STRF). 5 Spatial receptive fields • How do we represent receptive fields mathematically? • At the simplest level, we think of the receptive field (RF) as the region of visual space that causes the neuron to spike. • But a visual neuron doesn’t respond to any stimulus within this RF. It responds selectively to certain ‘features’ in the stimulus. • We can think of a neuron as having a filter (G) that passes certain features in both space and time. • The better the stimulus ‘overlaps’ with the filter, the more the neuron will spike. 6 Spatial receptive fields • How do we represent receptive fields mathematically? Start by describing the spatial part of this filter. output spike filter x nonlinearity generator y r r L I(x, y) L = GI[ (x, y)] = [ 0 + ]+ PT [n] stimulus Poisson process response filter stimulus firing rate Image of mouse in public domain. 7 Spatial receptive fields • How do we represent receptive fields mathematically? We are going to consider the simplest case in which the response of a neuron is given by a linear filter acting on the stimulus. y G(x, y) r = r + G(x, y)I(x, y)dx dy 0 ∫∫ x Let’s look at this in one dimension G(x) r = r + G(x)I(x)dx 0 ∫ x Like a correlation ∑Gi Ii i 8 Spatial receptive fields • How do we represent receptive fields mathematically? G(x) G(x) x I(x) I(x) x G(x)I(x) G(x)I(x) x ∫ G(x)I(x)dx big ∫ G(x)I(x)dx small 9 Linearity • Response varies linearly with overlap G(x) G(x) x I(x) I(x) x G(x)I(x) G(x)I(x) x ∫ G(x)I(x)dx big ∫ G(x)I(x)dx half as big 10 Temporal receptive fields • We can also think of the response of a neuron as some function of the temporal variations in the stimulus. r(t) = r0 + D [ S(t) ] Time-dependent Spontaneous Filter Stimulus firing rate firing rate 11 Temporal receptive fields • We can think of ‘overlap’ in the time domain! That there is a particular ‘temporal profile’ of a stimulus that makes a neuron spike. S(t) time Does this look familiar? 12 Temporal receptive fields ∞ r(t) = r + D(τ ) (t −τ )dτ Convolution!! 0 ∫ S −∞ D(τ ) τ Linear temporal response kernel. (Or ‘temporal kernel’) It is linear in the sense that if we make the stimulus partial, or weaker, the response changes linearly. 13 Learning objectives for Lecture 9 • To be able to mathematically describe a neural response as a linear filter followed by a nonlinear function. – A correlation of a spatial receptive field with the stimulus – A convolution of a temporal receptive field with the stimulus • To understand the concept of a Spatio-temporal Receptive Field (STRF) and the concept of ‘separability’ • To understand the idea of a Spike Triggered Average and how to use it to compute a Spatio-temporal Receptive Field and a Spectro-temporal Receptive Field (STRF). 14 Spatio-temporal receptive fields • Now we are going to put the temporal receptive field and the spatial receptive field together in a single object. • This is called the spatio-temporal receptive field (STRF). • Let’s imagine a stimulus that is a function of space and time, like the light falling on a retina: I(x, y,t) • But now we are going to simplify things by considering only one spatial dimension: I(x,t) ∞ r(t) = r + dx dτ D(x,τ )I(x,t −τ ) 0 ∫ −∞ 15 Spatio-temporal receptive fields Here we are doing a correlation and a convolution at the same time! Correlation in the integral over space and a convolution in the integral over time! r(t) = r + dx dτ D(x,τ )I(x,t −τ ) 0 ∫∫ correlation ∞ ∞ r(t) = r + dτ dx D(x,τ )I(x,t −τ ) 0 ∫ ∫ −∞ −∞ convolution 16 Separability • If a receptive field is separable in space and time, then we can decompose it into a spatial receptive field and a temporal receptive field. Separable Inseparable x x D(x,τ ) D(x,τ ) GS (x) - + τ τ DT (τ ) 17 Separability • If a receptive field is separable in space and time, then we can decompose it into a spatial receptive field and a temporal receptive field: ∞ ∞ r(t) = r + dx dτ D(x,τ )I(x,t −τ ) 0 ∫ ∫ −∞ −∞ D(x,τ ) =GS (x)DT (τ ) ∞ S(t) = ∫ dxG(x)I(x,t) Correlation −∞ where ∞ r(t) = r + dτ D (τ ) (t −τ ) Convolution 0 ∫ T S −∞ 18 Representing stimulus and receptive fields in space and time Suppose our stimulus is a bar of light extending from x=2 to x=4 and that is turned on for times from t=1 to t=6 We represent it on a space-time plot as follows: space r.f. 4 light 2 time 1 6 firing rate time 19 Representing stimulus and receptive fields in space and time Suppose we now consider a receptive field D(t,x) with spatial as well as temporal structure (but ‘space-time separable’: D(t,x) = D(t)D(x) ) space 4 light 2 time 1 6 firing rate time 20 Response to a moving bar of light Now suppose our stimulus is moving: light space 4 2 time 1 6 firing rate time 21 Response to a moving bar of light Now suppose our stimulus is moving in opposite direction: light space 4 2 time 1 6 firing rate time 22 Learning objectives for Lecture 9 • To be able to mathematically describe a neural response as a linear filter followed by a nonlinear function. – A correlation of a spatial receptive field with the stimulus – A convolution of a temporal receptive field with the stimulus • To understand the concept of a Spatio-temporal Receptive Field (STRF) and the concept of ‘separability’ • To understand the idea of a Spike Triggered Average and how to use it to compute a Spatio-temporal Receptive Field and a Spectro-temporal Receptive Field (STRF). 23 Spike-Triggered Average (t) S spikes + Sum over n spikes + 1 n K(τ ) = ∑S(ti −τ ) n i=1 Average over trials = n K(τ ) 1 τ K(τ ) = S(ti −τ ) n ∑ i=1 Trials 24 Measuring STRFs in the retina Marcus Meister Slides pp 25-31 © Marcus Meister. All rights reserved. This content is excluded from our Creative Commons license. 25 For more information, see https://ocw.mit.edu/help/faq-fair-use/. Simultaneous Recording from Retinal Ganglion Cells V) µ Voltage ( 200 0 0.0 Time (s) 0.2 Slides pp 25-31 © Marcus Meister. All rights reserved. This content is excluded from our Creative Commons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/. 26 60 Trial Rabbit ganglion cells responding to a natural movie “Trees swaying in the breeze” 0 12 14 16 18 20 160 0 80 0 300 0 Firing rate (Hz) 400 0 12 14 16 18 20 Time (s) Marcus Meister Slides pp 25-31 © Marcus Meister. All rights reserved. This content is excluded from our Creative Commons license. For more information, see https://ocw.mit.edu/help/faq-fair-use/. 27 Measuring STRFs in the retina Random flicker stimulus Marcus Meister Slides pp 25-31 © Marcus Meister. All rights reserved. This content is excluded from our Creative Commons license. 28 For more information, see https://ocw.mit.edu/help/faq-fair-use/. Reverse-Correlation to a Random Flicker Stimulus Response Stimulus Spike-triggered average Marcus Meister Slides pp 25-31 © Marcus Meister. All rights reserved. This content is excluded from our Creative Commons license. 29 For more information, see https://ocw.mit.edu/help/faq-fair-use/. Spatio-Temporal Receptive Fields (STRF) See Lecture 9 video to view the above clips. Marcus Meister Slides pp 25-31 © Marcus Meister. All rights reserved.