A Brief Introduction of Signal Detection Theory
Total Page:16
File Type:pdf, Size:1020Kb
A brief introduction of Signal Detection Theory Dr. Zhuanghua Shi Changes of Schedule • Due to NCP Day next Wednesday, Course C schedule changes as follows: • Date Content Dec. 12 Signal Detection Theory Dec. 19 --- (NCP Day) Jan. 09 Practical – Signal Detection Analysis Jan. 16 Exam: Theoretical part 2 A demo: Detection task • There will be 4 x-ray photos of travel suitcases presented briefly. Your task is to identify if there is a gun inside. Please write down the number of the photos. 3 + 4 + 5 + 6 + 7 8 Results Photo No. Gun No Gun Gun No Gun Yes 11 2 1 2 11 No 15 24 2 11 2 3 0 13 4 0 13 9 Prior knowledge Expectation Bias Perception & Action Sensitivity Sensation External environment Brain 10 Sensitivity and Criterion • Performance is a function of TWO properties of the observer: • How well the observer perceives stimuli – or sensitivity • And, how does the observer choose to respond – or criterion • We need methods to assess the two independently • SIGNAL DETECTION THEORY 11 History of signal detection theory • WWII, researchers concerned with maximization of correct detection responses and minimization of false detection responses • No way to optimize both • First introduced by radar researchers (Marcum, 1960) • How to discriminate radar signal and noise Signal and noise Signal Noise S + N = stimulus 2 2 2 1.5 1.5 1.5 1 1 1 0.5 0.5 0.5 0 0 0 -0.5 -0.5 -0.5 -1 -1 -1 0 50 100 150 200 250 300 350 400 450 500 0 50 100 150 200 250 300 350 400 450 500 0 50 100 150 200 250 300 350 400 450 500 2 2 1.5 1.5 1 1 0.5 0.5 0 0 -0.5 -0.5 -1 -1 0 50 100 150 200 250 300 350 400 450 500 0 50 100 150 200 250 300 350 400 450 500 Signal Noise Sensory signal and decision making • Signals are always accompanied with background noise • luminance detection (background luminance) • target detection (against distractors) • For any sensory inputs the brain need to interpret its meaning (e.g., signal or noise) • Decision making (Criterion) • Simple two-alternative force choice • Above that criterion: Yes • Below that criterion: No Confusion matrix – decoding the observer • Different combinations of stimuli and responses Stimuli Signal Noise Yes Hit False Alarm Response No Miss Correct Rejection What is noise? External noise Internal noise • interferes the signal • corrupts signal representation Signal and Noise: Statistical view • Noisy signal representation, Rs: !" = $ + & where S is signal, N is noise • How can we decode the signal from !"? Distribution of signal representations • If noise is white, then the received noisy signal will distribute as a normal distribution • yielding a normal distribution of signal representations • The mean of this distribution would reflect the signal Mean signal, S Probability Signal representation 18 Distribution of noise • Gaussian distribution of noise representation near 0 • Usually we assume signal is positive, that is, !" > !$ Mean noise, N Probability Noise representation 19 Decoding the signal • Decision can be made based on the information from two distribution (e.g., likelihood) N S 20 Sometime decision is hard! N S 21 When you have to make a hard decision, flip a coin. Why? Because when that coin is in the air… You suddenly know what you’re hoping for. Bias emerges ! 22 Decision requires a criterion • By comparing signal representation with a criterion RS < C: Noise RS > C: Signal N C S 23 Easy and hard decisions • If the signal representation is far from the criterion, the decision is easy • The closer it gets, the harder the decision Potential S Potential N N C S 24 Describing the S, N, and C Misses Hits Correct rejections False alarms p(Rs < C|S) p(Rs > C|S) p(Rs < C|N) p(Rs > C|N) N C S N C S S N Y H FA N M CR 100% 100% 25 Describing the S, N and C • p(H) + p(M) = 1 , so knowing p(H) is sufficient • p(FA) + p(CR) = 1 , so knowing p(FA) is sufficient • We can describe S, N, and C fully by considering only hits and false alarms S H Y H FA N M CR 100% 100% 26 Describing the S, N and C • Overall decision difficulty is determined by the signal/noise relationship – sensitivity • Actual performance also depends on decision criterion Sensitivity N C S 27 Sensitivity and decision criterion • Sensitivity (Discriminability) • Measure of how close signal and noise are • Supposed to be a property only of the sensory process • Decision Criterion (Response Bias) • Measure of what is considered noise and what signal • Susceptible to motivation, strategy, etc. • Sensitivity and criterion are independent 28 Sensitivity and decision criterion Unbiased criterion Biased High Low sensitivity Low 29 Computing sensitivity and criterion • What are the measuring units of S and N? • Independently of the measuring scale (e.g., sound frequency, line length, etc.) we can standardize the distribution by computing deviance scores Real measure [x] Standard measure [z] 30 Normal distribution • Assumption – N and S are normally distributed 31 Normal distribution • For each probability, we can compute a z score Z score – a horizontal distance measure • p = 97.7%; z = 2 • p = 2.3%; z = -2 32 Computing sensitivity • d’ = difference between means of S and N d0 = µ µ S − N d’ d0 = z(H)+z(CR) = Z(H) Z(FA) − 33 Effects of N and S variability • If N and s variability differ, individual variability has to be computed • Sensitivity is then d a d’ (µs - µn ) d a = 2 2 s s +s n 2 34 Computing decision criterion • Criterion influences overall frequency of YES responses: 1. p(H) = 1, p(FA) = .84; d’ = 2 2. p(H) = .84, p(FA) = .16; d’ = 2 3. p(H) = .16, p(FA) = 0; d’ = 2 1 2 3 35 Computing decision criterion • Likelihood-ratio, β = p(C|S)/p(C|N) 1. p(C1|S) = .004, p(C1|N) = .242, β = .018 – liberal criterion 2. P(C2|S) = .242, p(C2|N) = .242, β = 1 – neutral criterion 3. P(C3|S) = .242, p(C3|N) = .004, β = 55 – conservative 1 2 3 Probability 36 Computing decision criterion • Response bias, C = -(zH + zFA)/2 1. zH = 3, zFA = 1, C = -2 – liberal criterion 2. zH = 1, zFA = -1, C = 0 – neutral criterion 3. zH = -1, zFA = -3, C = 2 – conservative 1 2 3 Z score 37 Relating criterion and sensitivity – ROC curve • Receiver operating characteristics – ROC curve • Relation between p(H) and p(FA) for varying response criteria • One curve shows one sensitivity, d’ 1,00 0,84; 1 1 2 3 0,16; 0,75 0,84 0,50 p(H) 0,25 0; 0,16 0,00 0,00 0,25 0,50 0,75 1,00 p(FA) 38 ROC and different sensitivities • Increasing sensitivity increases the slope of ROC curve = switching between ROC curves 39 ROC and different criteria • Changing criteria shifts the location at a single ROC curve 1 2 3 40 Sensitivity from ROC curve • The area under ROC reflects sensitivity • Usually, we have only ONE p(H)/p(FA) data point • Problem – how to estimate sensitivity from a single point? 41 Sensitivity from ROC curve • Solutions: • Assume normal distribution • Approximate the area 42 Criterion from ROC curve • The slope of the ROC curve at the observed point • b = 1 – neutral criterion • b < 1 – liberal criterion • b > 1 – conservative criterion 43 Two-alternative Discrimination task 7.1. THE TWO-ALTERNATIVE DISCRIMINATION TASK 115 • Two signals A and B • Yes responses are PA and PB • Assume equal variance, ! Figure 7.1: The signal-detection model for two-alternative discrimination, The "′ = % & + % & shaded areas correspond to correct responses. ' ) From Wickens (2002) more attractive to treat the discrimination stimuli symmetrically. Instead * = −(% & − % & of )speakin/2 g of hits and false alarms, a better representation is to use the ' )proportion of correct responses made to each stimulus. Denote the two signals by A and B and the responses by A and B. The probabilities of correct responses of each type are given by the conditional probabilities PA = P(A|A) and PB = P(B|B). Denote the corresponding proportions by their lowercase equivalents, PA and PB- Figure 7.1 shows the distributional model of the task. The decision axis is represented by a singl44e dimension, and observing a stimulus gives a value along this axis. The distribution of the observations depends on the which stimulus was presented. A decision criterion divides the continuum into response regions associated with the two responses. The shaded areas in the figure correspond to the probabili- ties PA and PB of correctly identifying the two types of stimulus. The decision model of Figure 7.1 also implies the existence of a discrim- ination operating characteristic. By varying the position of the decision criterion, performance can be varied from an emphasis on A responses, with PA 1 and PB 0, to an emphasis on B responses, with PA 0 and PB 1. The relationship of the two probabilities is shown in Figure 7.2. This discrimination operating characteristic is reversed left to right from the detection operating characteristics because the horizontal axis plots correct responses instead of errors. Its interpretation is the same. In par- ticular, the area under the curve has the same meaning as a measure of the distinguishability of the signals. Suppose further that the distributions are Gaussian, with means uA and uB and common variance o2. Because the decision continuum cannot be directly observed, exact values for uA, uB, o2 , and the decision criterion 2-Alternative Force-Choice (2AFC) task • 2AFC task can be treated as detection of the difference between the two • Suppose Noise (N), and Signal (N+S) both are normal distributed • The variance of the difference of the two has double! (See below) • !" = ! $% > $' = ! $% − $' > 0 ~ +(-%, 2) 6.3.