Introduction to Detection Theory

Introduction to Detection Theory

Introduction to Detection Theory Reading: • Ch. 3 in Kay-II. • Notes by Prof. Don Johnson on detection theory, see http://www.ece.rice.edu/~dhj/courses/elec531/notes5.pdf. • Ch. 10 in Wasserman. EE 527, Detection and Estimation Theory, # 5 1 Introduction to Detection Theory (cont.) We wish to make a decision on a signal of interest using noisy measurements. Statistical tools enable systematic solutions and optimal design. Application areas include: • Communications, • Radar and sonar, • Nondestructive evaluation (NDE) of materials, • Biomedicine, etc. EE 527, Detection and Estimation Theory, # 5 2 Example: Radar Detection. We wish to decide on the presence or absence of a target. EE 527, Detection and Estimation Theory, # 5 3 Introduction to Detection Theory We assume a parametric measurement model p(x | θ) [or p(x ; θ), which is the notation that we sometimes use in the classical setting]. In point estimation theory, we estimated the parameter θ ∈ Θ given the data x. Suppose now that we choose Θ0 and Θ1 that form a partition of the parameter space Θ: Θ0 ∪ Θ1 = Θ, Θ0 ∩ Θ1 = ∅. In detection theory, we wish to identify which hypothesis is true (i.e. make the appropriate decision): H0 : θ ∈ Θ0, null hypothesis H1 : θ ∈ Θ1, alternative hypothesis. Terminology: If θ can only take two values, Θ = {θ0, θ1}, Θ0 = {θ0}, Θ1 = {θ1} we say that the hypotheses are simple. Otherwise, we say that they are composite. Composite Hypothesis Example: H0 : θ = 0 versus H1 : θ ∈ (0, ∞). EE 527, Detection and Estimation Theory, # 5 4 The Decision Rule We wish to design a decision rule (function) φ(x): X → (0, 1): 1, decide H , φ(x) = 1 0, decide H0. which partitions the data space X [i.e. the support of p(x | θ)] into two regions: Rule φ(x): X0 = {x : φ(x) = 0}, X1 = {x : φ(x) = 1}. Let us define probabilities of false alarm and miss: Z PFA = E x | θ[φ(X) | θ] = p(x | θ) dx for θ in Θ0 X1 Z PM = E x | θ[1 − φ(X) | θ] = 1 − p(x | θ) dx X1 Z = p(x | θ) dx for θ in Θ1. X0 Then, the probability of detection (correctly deciding H1) is Z PD = 1−PM = E x | θ[φ(X) | θ] = p(x | θ) dx for θ in Θ1. X1 EE 527, Detection and Estimation Theory, # 5 5 Note: PFA and PD/PM are generally functions of the parameter θ (where θ ∈ Θ0 when computing PFA and θ ∈ Θ1 when computing PD/PM). More Terminology. Statisticians use the following terminology: • False alarm ≡ “Type I error” • Miss ≡ “Type II error” • Probability of detection ≡ “Power” • Probability of false alarm ≡ “Significance level.” EE 527, Detection and Estimation Theory, # 5 6 Bayesian Decision-theoretic Detection Theory Recall (a slightly generalized version of) the posterior expected loss: Z ρ(action | x) = L(θ, action) p(θ | x) dθ Θ that we introduced in handout # 4 when we discussed Bayesian decision theory. Let us now apply this theory to our easy example discussed here: hypothesis testing, where our action space consists of only two choices. We first assign a loss table: decision rule φ ↓ true state → Θ1 Θ0 x ∈ X1 L(1|1) = 0 L(1|0) x ∈ X0 L(0|1) L(0|0) = 0 with the loss function described by the quantities L(declared | true): • L(1 | 0) quantifies loss due to a false alarm, • L(0 | 1) quantifies loss due to a miss, • L(1 | 1) and L(0 | 0) (losses due to correct decisions) — typically set to zero in real life. Here, we adopt zero losses for correct decisions. EE 527, Detection and Estimation Theory, # 5 7 Now, our posterior expected loss takes two values: Z ρ0(x) = L(0 | 1) p(θ | x) dθ Θ1 Z + L(0 | 0) p(θ | x) dθ Θ 0 | {z0 } Z = L(0 | 1) p(θ | x) dθ Θ1 L(0 | 1) is constant Z = L(0 | 1) p(θ | x) dθ Θ1 | {z } P [θ∈Θ1 | x] and, similarly, Z ρ1(x) = L(1 | 0) p(θ | x) dθ Θ0 L(1 | 0) is constant Z = L(1 | 0) p(θ | x) dθ . Θ0 | {z } P [θ∈Θ0 | x] We define the Bayes’ decision rule as the rule that minimizes the posterior expected loss; this rule corresponds to choosing the data-space partitioning as follows: X1 = {x : ρ1(x) ≤ ρ0(x)} EE 527, Detection and Estimation Theory, # 5 8 or P [θ∈Θ1 | x] Zz }| { ( p(θ | x) dθ ) L(1 | 0) X = x : Θ1 ≥ (1) 1 Z L(0 | 1) p(θ | x) dθ Θ0 | {z } P [θ∈Θ0 | x] or, equivalently, upon applying the Bayes’ rule: ( R ) p(x | θ) π(θ) dθ L(1 | 0) X = x : Θ1 ≥ . (2) 1 R p(x | θ) π(θ) dθ L(0 | 1) Θ0 0-1 loss: For L(1|0) = L(0|1) = 1, we have decision rule φ ↓ true state → Θ1 Θ0 x ∈ X1 L(1|1) = 0 L(1|0) = 1 x ∈ X0 L(0|1) = 1 L(0|0) = 0 yielding the following Bayes’ decision rule, called the maximum a posteriori (MAP) rule: n P [θ ∈ Θ1 | x] o X1 = x : ≥ 1 (3) P [θ ∈ Θ0 | x] or, equivalently, upon applying the Bayes’ rule: ( R p(x | θ) π(θ) dθ ) X = x : Θ1 ≥ 1 . (4) 1 R p(x | θ) π(θ) dθ Θ0 EE 527, Detection and Estimation Theory, # 5 9 Simple hypotheses. Let us specialize (1) to the case of simple hypotheses (Θ0 = {θ0}, Θ1 = {θ1}): ( ) p(θ1 | x) L(1 | 0) X1 = x : ≥ . (5) p(θ0 | x) L(0 | 1) | {z } posterior-odds ratio We can rewrite (5) using the Bayes’ rule: ( ) p(x | θ1) π0 L(1 | 0) X1 = x : ≥ (6) p(x | θ0) π1 L(0 | 1) | {z } likelihood ratio where π0 = π(θ0), π1 = π(θ1) = 1 − π0 describe the prior probability mass function (pmf) of the binary random variable θ (recall that θ ∈ {θ0, θ1}). Hence, for binary simple hypotheses, the prior pmf of θ is the Bernoulli pmf. EE 527, Detection and Estimation Theory, # 5 10 Preposterior (Bayes) Risk The preposterior (Bayes) risk for rule φ(x) is Z Z E x,θ[loss] = L(1 | 0) p(x | θ)π(θ) dθ dx X1 Θ0 Z Z + L(0 | 1) p(x | θ)π(θ) dθ dx. X0 Θ1 How do we choose the rule φ(x) that minimizes the preposterior EE 527, Detection and Estimation Theory, # 5 11 risk? Z Z L(1 | 0) p(x | θ)π(θ) dθ dx X1 Θ0 Z Z + L(0 | 1) p(x | θ)π(θ) dθ dx X0 Θ1 Z Z = L(1 | 0) p(x | θ)π(θ) dθ dx X1 Θ0 Z Z − L(0 | 1) p(x | θ)π(θ) dθ dx X1 Θ1 Z Z + L(0 | 1) p(x | θ)π(θ) dθ dx X0 Θ1 Z Z + L(0 | 1) p(x | θ)π(θ) dθ dx X1 Θ1 = const | {z } not dependent on φ(x) Z n Z + L(1 | 0) · p(x | θ)π(θ) dθ X1 Θ0 Z o −L(0 | 1) · p(x | θ)π(θ) dθ dx Θ1 implying that X1 should be chosen as n Z Z o X1 : L(1 | 0)· p(x | θ)π(θ) dθ−L(0 | 1)· p(x | θ)π(θ) < 0 Θ0 Θ1 EE 527, Detection and Estimation Theory, # 5 12 which, as expected, is the same as (2), since minimizing the posterior expected loss ⇐⇒ minimizing the preposterior risk for every x as showed earlier in handout # 4. 0-1 loss: For the 0-1 loss, i.e. L(1|0) = L(0|1) = 1, the preposterior (Bayes) risk for rule φ(x) is Z Z E x,θ[loss] = p(x | θ)π(θ) dθ dx X1 Θ0 Z Z + p(x | θ)π(θ) dθ dx (7) X0 Θ1 which is simply the average error probability, with averaging performed over the joint probability density or mass function (pdf/pmf) or the data x and parameters θ. EE 527, Detection and Estimation Theory, # 5 13 Bayesian Decision-theoretic Detection for Simple Hypotheses The Bayes’ decision rule for simple hypotheses is (6): H p(x | θ1) 1 π0 L(1|0) Λ(x) = ≷ ≡ τ (8) | {z } p(x | θ0) π1 L(0|1) likelihood ratio see also Ch. 3.7 in Kay-II. (Recall that Λ(x) is the sufficient statistic for the detection problem, see p. 37 in handout # 1.) Equivalently, H 1 0 log Λ(x) = log[p(x | θ1)] − log[p(x | θ0)] ≷ log τ ≡ τ . Minimum Average Error Probability Detection: In the familiar 0-1 loss case where L(1|0) = L(0|1) = 1, we know that the preposterior (Bayes) risk is equal to the average error probability, see (7). This average error probability greatly EE 527, Detection and Estimation Theory, # 5 14 simplifies in the simple hypothesis testing case: Z av. error probability = L(1 | 0) p(x | θ0)π0 dx X 1 | {z1 } Z + L(0 | 1) p(x | θ1)π1 dx X 0 | {z1 } Z Z = π0 · p(x | θ0) dx +π1 · p(x | θ1) dx X1 X0 | {z } | {z } PFA PM where, as before, the averaging is performed over the joint pdf/pmf of the data x and parameters θ, and π0 = π(θ0), π1 = π(θ1) = 1 − π0 (the Bernoulli pmf). In this case, our Bayes’ decision rule simplifies to the MAP rule (as expected, see (5) and Ch. 3.6 in Kay-II): H p(θ1 | x) 1 ≷ 1 (9) p(θ0 | x) | {z } posterior-odds ratio or, equivalently, upon applying the Bayes’ rule: H p(x | θ1) 1 π0 ≷ .

View Full Text

Details

  • File Type
    pdf
  • Upload Time
    -
  • Content Languages
    English
  • Upload User
    Anonymous/Not logged-in
  • File Pages
    67 Page
  • File Size
    -

Download

Channel Download Status
Express Download Enable

Copyright

We respect the copyrights and intellectual property rights of all users. All uploaded documents are either original works of the uploader or authorized works of the rightful owners.

  • Not to be reproduced or distributed without explicit permission.
  • Not used for commercial purposes outside of approved use cases.
  • Not used to infringe on the rights of the original creators.
  • If you believe any content infringes your copyright, please contact us immediately.

Support

For help with questions, suggestions, or problems, please contact us