Signal Detection Theory Direct Comparison of Perception And

Signal detection theory

yes no

NeurometricNeurometric functionfunction PsychometricPsychometric f ufunctionnction

Neuronal Behavioral response judgement Choice probability?

Direct comparison of perception and neural activity

Record neuron that prefers downward motion while monkeys make up/down motion judgments for stimuli that are barely discriminable.

Newsome, Movshon, Britten, Shadlen, Salzman Visual area MT

Record

LIP, VIP 7a

DP FEF

V1 M V4 T MST V 2 FST

PITd S T VP PITv P VOT CITd CITv AITd Visual AITv input

Visual area MT

MST MT V3a

Electrophysiology (action potential)

microelectrode amplifier Voltage (mV) Time (msec)

Cortical pyramidal cell (Golgi stain) MT neurons are direction-selective

MT neurons are direction-selective

Maunsell and Van Essen, 1983

Behavioral Receptive field Pref target protocol Dots Aperture

Null target

10 deg Fixation Point

Fix Pt

Dots

Targets

1 sec Stimulus manipulation: motion coherence

0% coherence 50% coherence 100% coherence

Psychometric function

Britten, Shadlen, Newsome & Movshon, 1992

Motion coherence and MT neurons

Motion stimulus no coherence 50% coherence 100% coherence

Responses of MT neurons direction Preferred Motion coherence and MT neurons

Neural responses are noisy

Each tick is an action potential

Each line corresponds to

a stimulus Trial number presentation

Average across (sp/sec) all trials Firing rate Time (msec)

Perceptual decision

Decision rule: choose ‘pref’ if pref response > non-pref response.

fn fp

fn (r) Probability fp (r)

r Neuronal response

fp (r) : response PDF for pref direction

fn (r) : response PDF for non-pref direction Perceptual decision

rp : response to pref direction

rn : response to non-pref direction

fn fp fp (r) : response PDF for pref direction

fn (r) : response PDF for non-pref direction

Fn (r) : response CDF for non-pref direction Probability

r Neuronal response

∞ r ⎡ ′ ′⎤ P(correct) = P(rp > rn ) = fp (r) fn (r )dr dr ∫ ⎣∫0 ⎦ 0 r fn (r′)dr′ = Fn (r) ∫0 ∞ P(correct) = f (r)F (r)dr ∫ p n 0

6.4% Neurometric function

(correct) Response distributions P = ∑ fp[r]Fn[r] r

Response (spikes/trial) Neurometric function

6.4%

I 1 1 ““‘I ’ I ’ ,'"I 0.2 0.4 0.6 0.8 7.0 1 .o 10.0 100 P (null > crit) Correlation (%) Response (spikes/trial) Figure 5. Analysis of physiological data. A, This three-dimensional plot illustrates frequency histograms of responses obtained from a single MT neuron at five different correlation levels. The horizontal axis shows the amplitude of the neuronal response, and the vertical axis indicates the number of trials on which a particular response was obtained. The depth axis shows the correlation of motion signal used to elicit the response distributions. Open bars depict responses obtained for motion in the neuron’s preferred direction, while solid bars illustrate responses for null direction motion. For this neuron, each distribution is based on 60 trials. B, ROCs for the five pairs of preferred-null response distributions illustrated in A. Each point on an ROC depicts the proportion of trials on which the preferred direction response exceeded a criterion level plotted against the proportion of trials on which the null direction response exceededBritten, criterion. E aShadlen,ch ROC was g eNewsomenerated by incre a&sin gMovshon, the criterion le1992vel from 0 to 120 spikes in one-spike increments. Increased separation of the preferred and null response distributions in A leads to an increased deflection of the ROC away from the diagonal. C, A neurometric function that describes the sensitivity of an MT neuron to motion signals of increasing strength. The curve shows the performance of a simple decision model that bases judgements of motion direction on responses like those illustrated in A. The proportion of correct choices made by the model is plotted against the strength of the motion signal. The proportion of correct choices at a particular correlation level is simply the normalized area under the corresponding ROC curve in B. For this neuron, data were obtained at seven correlation levels; response distributions and ROC curves are illustrated for five of these levels in A and B. The neurometric function was fitted with sigmoidal curves of the form given in Equation 1. In this experiment, neuronal threshold (a) was 4.41 correlation and the unitless siope parameter for the neurometric function (p) was 1.30.

I 1 1 ““‘I ’ I ’ ,'"I 0.2 0.4 0.6 0.8 7.0 1 .o 10.0 100 NeurometricP (null > crit) vs. psychometricCorrelation (%) functions Figure 5. Analysis of4 7p5h2y sioloBgrititcena l deat taa.l . A,l MThTi s Ntehurroenes- dimenansdi onaPls ychopplhoyts icailll ustrates Pefrfeorqmuaenncec y histograms of responses obtained from a single MT neuron at five different correlation levels. The horizontal axis shows the amplitude of the neuronal response, and the vertical axis indicates the number of trials on which a particular response was obtained. The depth axis shows the correlation of motion signal used to elicit the response distributions. Open bars depict responses obtained for motion in the neuron’s preferred direction, while solid bars illustrate responses for null direction motion. For this neuron, each distribution is based on 60 trials. B, ROCs for the five pairs of preferred-nautll roruegspholnys e ediqsturibault iornas tes as the criterion response increased from illustrated in A. Each point on an ROC depicts the proportion of trials on which the preferred direction response excee1de dto a c1rit0er0io ni mlepvuell spelost/tetrdi al. In general, the curvature of the ROC against the proportion of trials on which the null direction response exceeded criterion. Each ROC was generated by ianwcreaays ingfr otmhe crtihteerio nd ialegvoeln al indicates the separation of the preferred from 0 to 120 spikes in one-spike increments. Increased separation of the preferred and null response distributions in A leadpsychometrics to an increased deflection of the ROC away from the diagonal. C, A neurometric function that describes the sensitivity of an MT naenurdo n ntou llm orteiosnp osnigsnea ls doifs tributions (Bamber, 1975). increasing strength. The curve shows the performance of a simple decision model that bases judgements of motion direction on responses like those illustrated in A. The proportion of correct choices made by the model is plotted against the strength of the motion signal. TGher eperonneurometricp orationnd Sofw ceortsre ct( 1966) showed formally that the normalized choices at a particular correlation level is simply the normalized area under the corresponding ROC curve in B. For thisa nreuaro nu, ndaetar wetrhee o bRtaiOneCd corresponds to the performance expected at seven correlation levels; response distributions and ROC curves are illustrated for five of these levels in A and B. The neurometric function was The Journal of Neuroscience, December 1992, fZ(l2) 4755 fitted with sigmoidal curves of the form given in Equation 1. In this experiment, neuronal threshold (a) was 4.41 correloatfi ona n aindd ethael uonbitlsesesr vseiorp e in a two-alternative, forced-choice psycho- parameter for the neurometric function (p) was 1.30. physical paradigm like the one employed in the present study. Again, one can intuit that this is reasonable. At 12.8% corre- A lation, 99% of the area of the unit square in Figure 5B falls monkey J --f/- beneath the ROC, corresponding to the near-perfect perfor- / mance we would expect based on the response distributions for 12.8% correlation in Figure 5.4. In contrast, only 51% of the / unit square falls beneath the ROC for 0.8% correlation, corre- / sponding as expected to random performance. For each correlation level tested, we used this method to compute the probability that the decision rule would yield a -k monkey E correct response, and the results are shown in Figure 5C. These / data capture the sensitivity of the neuron to directional signals

E 1 m,onkey YT , , , in the same manner that the psychometric function captures perceptual sensitivity to directional signals. As for the psychometric data, we fitted the neurometric data with smooth curves IO 15 20 of the form given by Equation 1. This function provided an Mean neuronal threshold excellent description of the neurometric data; the fit could be rejected for only 2 of the 2 16 neurometric functions in our data set (x2 test, p < 0.05). Application of Equation 2 resulted in a significantly improved fit for only one neuron. For the example in Figure 5C, the threshold parameter, (Y,w as 4.4% correlation, and the slope parameter, N&e uwraosn a1l .3t0h. rFeosrh oeladc h (n%eu)r ometric function, these parameters can be compared to the equivalent parameters Figuorbet ai1n0e. d A frcoomm partishoen cofr reasbpsoluntdei ngn euronpasl ychaondm eptsriycc hophysical function. threshold for the 2 16 experiments in our sample. Solid circles indicate 1 10 100 1 10 100 experiments in which neuronal and psychophysical threshold were sta- Compartiistoicnalo lyf pinsdyisctihngoumisheatbrliec; anodp enne cuirrcolems eilltursictraf uten cetxiopenrism.e Fntisg - in which ure 6A shothwe sttw hoe mpesayscurheos mwerter ics iagnidfic annetluy rodmifferternict. fuThnec tiboronksen obdi-a gonal is Correlation (%) tained fromth et hlinee eBritten,oxnp ewhriicmh enal tl Shadlen,ipllouinsttsr awteoudld inNewsomef aFll igif unreeusro 4na la &nthd rMovshon, e5s.h oTldh ep redict e1992d psychophysical threshold perfectly. The frequency histogram at the up- Figure 6. Psychometrica nd neurometricf unctionso btainedi n six two functiopners raigrhet r ewmas afrokrmaebdly bsyim suilmamr binog tha cinro sths ethier losccaattetiroplnot alwoitnhign diago- experimentsT. he open symbols and broken lines depict psychometric the abscisnsaalaly n dor iienn tethd eibr inosv, eTrhae ll rsehsualtipnge .T hhiseto garapmp arise na t ssciamledil arreiptylic a of the data,w hile the solid symbols ands olid lines represennte urometricd ata. of the twod isfturibnuctitoion nso wf tahrses rheofldle crtaetiods ind epaic tceldo sine cFoigrurre s7.po ndence The six exampleisll ustratet he rangeo f relationshippsr esenitn ourd ata. L I betweent he threshold parameters,o (,a nd the slopep arameters, A, Resultso f the experiment illustrated I in Figures I 4 and 5. PsIy cho- physical and neuronal data were statist1ic.2a lly indistin1g.u4i shable 1in.6 this p, obtained from the Weibull fits (Eq. 1) to the two data sets. The neuroamgoentraicl l itnher edsehpoicldts tohfe 4li.n4e% o fc eoqrureallaittyio onn wcohmichp aallre pdo fian-t s would experiment. Thresholds and slope parameters are given in the captions fall if neuronal threshold perfectly predicted psychophysical for Figures 4 and 5. B, A second experiMmeantn in nwehuicrho napsl ychomseltoripce and vorably to the psychometric threshold of 6.1% correlation, and neurometric data were statistically indistinguishable. Psychometric 01= threshold. Summing within a diagonally arranged set of bins Figure 9. A comparison of average neuronal and psychophysicatlh ep esrl-o pep laeraadms etot etrhsew efrreeq suiemniclay rh aisstw ogerlla (mne inu rothme eutpripc epr =ri g1h.t3 0c;o mer of 17.8% correlation, p = 1.20; neurometric o( = 23.0% correlation, (3 = psychometric p = 1.17). This similarity of psychometric and 1.31. C, An exformanceperiment aicnr oswsh iacnhi maplss.y cAh,o pThey sigceaol methtrrice shmoelda n woaf sn esuurobn- al threshold Figure 10; this is simply a scaledr eplica of the distribution of is plotted against the geometric mean of psychophysical threshold for stantially lower than neuronal threshold. Psychometric 01 = 3.7% cor- neurometrtihcr deashtao wlda rsaa ti coos mshmoownn f eina tuFrigeu orfe o 7u.r Ddaetsap, iaten tdh Fe igsuyrmem etrical each of the three animals in the study. The vertical error bars in6dBica itlelu stratesa seconde xample.A lthough the absolutet hreshold relation, p = 1.S6E8M; nefourr onmeeurtroinc al (Yth =re s1h4o.l8d%, cwohrilree latthioen ,h ori@zo =n ta1l .49e.r rDor, bAanr s show SEM distribution of the ratios about unity, the scatterplot reveals experiment in fwohr icphs ycnheouprhoynsaicl al thrthersehsohldol d. waTsh es ubbrsotkaennt iallilnye ilso wtheer bethsat-nfi tting lreevgreesls- wereo hnliyg hae mr uonddeesrtc tohrere claotniodnit iboentsw eoef ntht hise etwxop emriemaesnutr, etsh(r e = 0.29), psychophysical threshold. Psychometric (Y = 13.0% correlation, (3 = sion through the data points. B, The geometric mean of neuromnetruicr ometraicn adn mdo pssty ocfh tohmis ecotrricre dlatiao ns eists wa cecroeu angteadinf oq ru bitye sthime inlater ranimal 2.15; neurometriscl opeL Yi=s p4lo.7tt%ed coargraeilnastito nth, e gfel=o me1t.r5ic8 . mEe, aAnn ofe xppseyrcihmoemnet tricin slope(. nEerruorr onal: CY= 23.0%, p = 1.3 1; psychophysical: (Y= 17.8%, p which thresholdsb arsw earned stihmei larre grbeusst iosnlo pleinse waerree adsi sdseimscilraibre. d Pins yAch. ometric differencesi llustrated in Figure 9A. Thus, psychophysical and LY= 3.9% correlation, B = 1.36; neurometric 01 = 4.0% correlation, p = = 1.20). Hingehuerro anbasl tohluretesth horledssh aoreld nsot ytp sictraollnyg loy cccourrrreeladt wedh eon tahne cell-by- 0.79. F, An experiment in which threshold and slope were dissimilar. propertiesc oef ll tbhaes nise, aulrtohno uugnhd tehre ys taurdey c rloesqeulyir eredla ate pds oync hthoep havyesrai-g e. The Psychometric (oYf =t he3 .1v%a riacnocrree latiionn ,p sy(c3h =o ph0y.9s1ic;a ln eurocmx (ertzri,c t he LaY=m o2u7n.t0 % of vartianllcye nonoapbtismenacl epo rfe as setnrotantgio cne llo-tfo t-hce lld cioscrrreimlatinioann ids an o(et .sgu.,r purnis- ings ince correlation, 0 = 1.81. accounted for, = 0.438). Adding neuronalt hreshold to the musoudaell y smnaellu rerocneaplt isven fiseitlidvsity,l a rtog eme coctieonnt ridciirteiecsti,o nr hvigahrie ssp eweidesl)y. (even as a coregressorr evealed a significant predictive effect oTf hne ur-e maiwninthgin p aMnTe)l swi nh Feirgeuarsea 6m eoxnekmeyp’slif py styhceh orapnhgysei ocaf lv saerni-s itivity is to 120 impulrsoensa/lt rtiharle, stheo ldp r(opp

NeurometricNeurometric functionfunction PsychometricPsychometric f ufunctionnction

Neuronal Behavioral response judgement Choice probability?

Predicting the monkey’s decisions

Neurometric & psychometric Response distributions for functions: accuracy vs pref and non-pref decisions motion coherence at a fixed motion coherence

1 50 0% motion coherence 0.9 45 )

c 40 e t s / c

0.8 s e 35 r e r s l o c u 30

p n o 0.7 m i i t ( 25 r

o e s p

n 20 o r 0.6 o P p

s 15 e R 10 0.5 Neuron C‘pref’orrec tdecisions trials 5 Behavior E‘non-pref’rror trials decisions 0.4 0 0.1 1 10 100 10 20 30 40 50 60 70 80 Motion strength (% coherence) Trial number

Shadlen, Britten, Newsome & Movshon, 1996

Predicting the monkey’s decisions

Choice probability: accuracy with which one could predict monkey’s decision from the response of the neuron given that you know the distributions.

fN (r) fP (r) Probability

Neuronal response

fP (r) : response PDF when monkey reports pref direction

fN (r) : response PDF when monkey reports non-pref direction Choice probability

Example neuron Across all neurons recorded

Britten, Newsome, Shadlen, Celebrini & Movshon, 1996

Computational model

Pooled "up" signal

+ Pooling noise MT neuron pool Σ− Decision

+ Pooling noise

Pooled "down" signal

•Noise 0is.7 partially correlated across neurons. •Responses are pooled non-optimally over large populations of neurons including those that are not the most selective.

•Additional0.65 noise is added after pooling. A C y t

i Shadlen, Britten, Newsome & Movshon, 1996 l i b a b o r 0.6 p B D e c i o h DamageC to MT affects performance 0.55

0.5 1 10 Threshold (% coherence)

Newsome and Paré, 1988 Microstimulation in MT changes motion perception

Salzman, Britten, Newsome, 1990