Basic Definitions and The Spectral Estimation Problem
Lecture 1
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–1 by P. Stoica and R. Moses, Prentice Hall, 1997 Informal Definition of Spectral Estimation
Given: A finite record of a signal.
Determine: The distribution of signal power over frequency. signal spectral density
t ω ω ω+∆ω π
t=1, 2, ... =
! (angular) frequency in radians/(sampling interval)
= !=2 f =frequencyincycles/(samplinginterval)
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–2 by P. Stoica and R. Moses, Prentice Hall, 1997 Applications
Temporal Spectral Analysis
Vibration monitoring and fault detection
Hidden periodicity finding
Speech processing and audio devices
Medical diagnosis
Seismology and ground movement study
Control systems design
Radar, Sonar
Spatial Spectral Analysis
Source location using sensor arrays
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–3 by P. Stoica and R. Moses, Prentice Hall, 1997
Deterministic Signals
1
y (t)g =
f discrete-time deterministic data
t= 1
sequence
1
X
2
y (t)j < 1
If: j
t= 1
1
X
i! t
Y (! ) (t)e
Then: = y
t= 1 exists and is called the Discrete-Time Fourier Transform (DTFT)
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–4 by P. Stoica and R. Moses, Prentice Hall, 1997 Energy Spectral Density
Parseval's Equality:
Z
1
X
1
2
jy (t)j (! )d!
= S
2
t= 1
where
4
2
S (! ) = jY (! )j
= Energy Spectral Density
We can write
1
X
i! k
(k )e S (! ) =
k = 1
where
1
X
(k ) = y (t)y (t k )
t= 1
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–5 by P. Stoica and R. Moses, Prentice Hall, 1997 Random Signals
Random Signal random signal probabilistic statements about future variations
t
current observation time
1
X
2
y (t)j = 1
Here: j
t= 1
n o
2
jy (t)j < 1
But: E fg
E = Expectation over the ensemble of realizations
n o
2
jy (t)j y (t) E =Averagepowerin
PSD = (Average) power spectral density
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–6 by P. Stoica and R. Moses, Prentice Hall, 1997
First Definition of PSD
1
X
i! k
(! ) (k )e
= r
k = 1
(k )
where r is the autocovariance sequence (ACS)
r (k ) = E fy (t)y (t k )g
r (k ) = r ( k ); r (0) jr (k )j
Note that
Z
1
i! k
(! )e d! (k ) =
r (Inverse DTFT)
2
Interpretation:
Z
n o
1
2
r (0) = E jy (t)j = (! )d!
2
so
(! )d! =
infinitesimal signal power in the band
d!
! 2
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–7 by P. Stoica and R. Moses, Prentice Hall, 1997
Second Definition of PSD
8 9
2
>
>
N
< =
X
1
i! t
(! ) lim E (t)e
= y
> >
N !1
N
:
;
t=1
Note that
1
2
jY (! )j (! ) = lim E
N
N !1 N
where
N
X
i! t
Y (! ) = y (t)e
N
t=1
y (t)g is the finite DTFT of f .
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–8 by P. Stoica and R. Moses, Prentice Hall, 1997
Properties of the PSD
(! ) = (! + 2 ) ! P1: for all .
Thus, we can restrict attention to
! 2 [ ; ] () f 2 [ 1=2; 1=2]
(! ) 0
P2:
(t)
P3: If y is real,
(! ) ( ! )
Then: =
(! ) 6= ( ! ) Otherwise:
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–9 by P. Stoica and R. Moses, Prentice Hall, 1997
Transfer of PSD Through Linear Systems
1
X
k
h q (q ) =
System Function: H
k
k =0
1 1
= q y (t) = y (t 1)
where q unit delay operator:
e(t) y (t)
- -
H (q )
2
(! ) (! ) = jH (! )j (! )
e y e
Then
1
X
y (t) e(t k )
= h
k
k =0
1
X
i! k
h e (! )
H =
k
k =0
2
(! ) jH (! )j (! ) e y =
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–10 by P. Stoica and R. Moses, Prentice Hall, 1997 The Spectral Estimation Problem
The Problem:
y (1); :::;y(N )g
From a sample f
^
(! ) f (! ); ! 2 [ ; ]g Find an estimate of :
Two Main Approaches :
Nonparametric:
– Derived from the PSD definitions.
Parametric:
– Assumes a parameterized functional form of the PSD
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–11 by P. Stoica and R. Moses, Prentice Hall, 1997 Periodogram and Correlogram Methods
Lecture 2
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–1 by P. Stoica and R. Moses, Prentice Hall, 1997
Periodogram
(! )
Recall 2nd definition of :
8 9
2
>
>
N
< =
X
1
i! t
(! ) = lim E y (t)e
> >
N !1
N
:
;
t=1
N
y (t)g
Given : f
t=1
E fg
Drop “ lim ” and “ ” to get
N !1
2
N
X
1
i! t
^
(! ) (t)e
= y
p
N
t=1
Natural estimator Used by Schuster ( 1900) to determine “hidden periodicities” (hence the name).
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–2 by P. Stoica and R. Moses, Prentice Hall, 1997
Correlogram
(! )
Recall 1st definition of :
1
X
i! k
(! ) = r (k )e
k = 1
P
(k ) ^r (k )
Truncate the “ ” and replace “r ” by “ ”:
N 1
X
i! k
^
^r (k )e (! )
=
c
k = (N 1)
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–3 by P. Stoica and R. Moses, Prentice Hall, 1997 Covariance Estimators (or Sample Covariances)
Standard unbiased estimate:
N
X
1
y (t)y (t k ); k 0 ^r (k ) =
N k
t=k +1
Standard biased estimate:
N
X
1
r^(k ) = y (t)y (t k ); k 0
N
t=k +1
For both estimators:
^r (k ) = ^r ( k ); k < 0
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–4 by P. Stoica and R. Moses, Prentice Hall, 1997
^ ^
(! ) (! )
Relationship Between and
p c
^
^r (k ) (! )
If: the biased ACS estimator is used in c ,
Then:
2
N
X
1
i! t
^
y (t)e (! ) =
p
N
t=1
N 1
X
i! k
^r (k )e =
k = (N 1)
^
= (! )
c
^ ^
(! ) = (! )
p c
Consequence:
^ ^
(! ) (! )
Both and can be analyzed simultaneously.
p c
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–5 by P. Stoica and R. Moses, Prentice Hall, 1997
^ ^
(! ) (! )
Statistical Performance of and
p c
Summary: N
Both are asymptotically (for large ) unbiased:
n o
^
E N ! 1 (! ) ! (! )
as
p N
Both have “large” variance, even for large .
^ ^
(! ) (! )
Thus, and have poor performance.
p c
Intuitive explanation:
r^(k ) r (k ) jk j
may be large for large
N 1
^r (k ) r (k )g
Even if the errors f are small,
jk j=0
there are “so many” that when summed in
^
(! ) (! )]
[ , the PSD error is large. p
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–6 by P. Stoica and R. Moses, Prentice Hall, 1997
Bias Analysis of the Periodogram
N 1
n o n o
X
i! k
^ ^
E (! ) = E (! ) = E f^r (k )g e
p c
k = (N 1)
!
N 1
X
jk j
i! k
r (k )e 1 =
N
k = (N 1)
1
X
i! k
= w (k )r (k )e
B
k = 1
8
< jk j
1 ; jk j N 1
N
w (k ) =
B
:
0; jk j N
= Bartlett, or triangular, window
Thus,
Z
o n
1
^
( )W (! ) d (! ) = E
B
p
2
(! ) = (! )
Ideally: W Dirac impulse . B
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–7 by P. Stoica and R. Moses, Prentice Hall, 1997
(! )
Bartlett Window W
B
" #
2
1 sin(!N=2)
W (! ) =
B
N sin(!=2)
W (! )=W (0) = 25
,forN
B B 0
−10
−20
−30 dB
−40
−50
−60 −3 −2 −1 0 1 2 3
ANGULAR FREQUENCY
1=N
Main lobe 3dB width .
W (! ) (! )
For “small” N , may differ quite a bit from . B
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–8 by P. Stoica and R. Moses, Prentice Hall, 1997 Smearing and Leakage
Main Lobe Width: smearing or smoothing
(! ) f 1=N Details in separated in by less than are not resolvable. φ(ω) ^φ(ω) smearing
<1/Ν ω ω
=N Thus: Periodogram resolution limit = 1 .
Sidelobe Level: leakage φ(ω)≅δ(ω) ^φ(ω)≅ (ω) WB leakage
ω ω
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–9 by P. Stoica and R. Moses, Prentice Hall, 1997 Periodogram Bias Properties
Summary of Periodogram Bias Properties: N
For “small” ,severebias
N ! 1 W (! ) ! (! )
As , ,
B
^
(! ) so is asymptotically unbiased.
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–10 by P. Stoica and R. Moses, Prentice Hall, 1997
Periodogram Variance
! 1
As N
nh ih io
^ ^
E (! ) (! ) (! ) (! )
1 1 2 2
p p
(
2
(! ); ! = !
1 1 2
=
0; ! 6= !
1 2
Inconsistent estimate
Erratic behavior
^φ(ω)
^φ(ω) asymptotic mean = φ(ω)
-+ 1 st. dev = φ(ω) too ω
Resolvability properties depend on both bias and variance.
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–11 by P. Stoica and R. Moses, Prentice Hall, 1997
Discrete Fourier Transform (DFT)
N
X
i! t
y (t)e (! ) =
Finite DTFT: Y
N
t=1
2
i
2
N
k = W = e
Let ! and .
N
2
k ) (
Then Y is the Discrete Fourier Transform (DFT):
N
N
N
X
tk
y (t)W ; k = 0;:::;N 1 Y (k ) =
t=1
N 1
N
Y (k )g fy (t)g
Direct computation of f from :
t=1
k =0
2
(N ) O flops
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–12 by P. Stoica and R. Moses, Prentice Hall, 1997
Radix–2 Fast Fourier Transform (FFT)
m
= 2
Assume: N
N=2
N
X X
tk tk
Y (k ) = y (t)W y (t)W +
t=1
t=N=2+1
N=2
X
Nk
tk
2
= [y (t) + y (t + N=2)W ]W
t=1
Nk
k ;
+1 for even
2 W
with =
1; k
for odd
~
2 i2=N
~ ~
= N=2 W = W = e
Let N and .
4
= 0; 2; 4;:::;N 2 = 2p
For k :
~
N
X
tp
~ ~
Y (2p) = [y (t) + y (t + N )]W
t=1
= 1; 3; 5;:::;N 1 = 2p + 1
For k :
~
N
X
t tp
~ ~
f[y (t) y (t + N )]W gW Y (2p + 1) =
t=1
~
= N=2 Each is a N -point DFT computation.
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–13 by P. Stoica and R. Moses, Prentice Hall, 1997
FFT Computation Count
k
= N = 2
Let c number of flops for point FFT. k
Then
k
2
c = + 2c
k k 1
2
k
k 2
) c =
k 2
Thus,
1
c = N log N
2
k 2
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–14 by P. Stoica and R. Moses, Prentice Hall, 1997 Zero Padding
Append the given data by zeros prior to computing DFT
(or FFT):
fy (1); ::: ;y(N ); 0; ::: 0 g
| {z } N
Goals:
N =
Apply a radix-2 FFT (so powerof2)
^
(! )
Finer sampling of :
1 N 1 N
2 2
k ! k
N N
k =0 k =0
^φ(ω) continuous curve
sampled, N=8
ω
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–15 by P. Stoica and R. Moses, Prentice Hall, 1997 Improved Periodogram-Based Methods
Lecture 3
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–1 by P. Stoica and R. Moses, Prentice Hall, 1997 Blackman-Tukey Method
Basic Idea: Weighted correlogram, with small weight
r (k ) jk j
applied to covariances ^ with “large” .
M 1
X
i! k
^
(! ) = w (k )^r (k )e
BT
k = (M 1)
w (k )g = f Lag Window w(k) 1
-M M k
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–2 by P. Stoica and R. Moses, Prentice Hall, 1997
Blackman-Tukey Method, con't
Z
1
^ ^
(! ) = ( )W (! )d
p
BT
2
(! ) = fw (k )g W DTFT
= Spectral Window
^
(! ) =
Conclusion: “locally” smoothed periodogram BT
Effect:
Variance decreases substantially
Bias increases slightly
By proper choice of M :
2
+ ! 0 N ! 1 MSE = var bias as
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–3 by P. Stoica and R. Moses, Prentice Hall, 1997 Window Design Considerations
Nonnegativeness:
Z
1
^ ^
( ) W (! )d (! ) =
p
BT
| {z }
2
0
(! ) 0 (, w (k )
If W is a psd sequence)
^
(! ) 0
Then: (which is desirable) BT
Time-Bandwidth Product
M 1
X
w (k )
k = (M 1)
N = =
e equiv time width
w (0)
Z
1
W (! )d!
2
= =
e equiv bandwidth
W (0)
N = 1
e e
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–4 by P. Stoica and R. Moses, Prentice Hall, 1997
Window Design, con't
= 1=N = 0(1=M )
e e
is the BT resolution threshold. M As increases, bias decreases and variance
increases. M ) Choose as a tradeoff between variance and
bias.
M N e Once is given, e (and hence ) is essentially fixed.
) Choose window shape to compromise between smearing (main lobe width) and leakage (sidelobe level).
Theenergyinthemainlobeandinthesidelobes
cannot be reduced simultaneously,onceM is given.
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–5 by P. Stoica and R. Moses, Prentice Hall, 1997
Window Examples
= 25 Triangular Window, M 0
−10
−20
−30 dB
−40
−50
−60 −3 −2 −1 0 1 2 3
ANGULAR FREQUENCY
= 25 Rectangular Window, M 0
−10
−20
−30 dB
−40
−50
−60 −3 −2 −1 0 1 2 3 ANGULAR FREQUENCY
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–6 by P. Stoica and R. Moses, Prentice Hall, 1997 Bartlett Method
Basic Idea: 1 Μ 2Μ . . . Ν
^φ (ω) ^φ (ω) ^φ (ω) 1 2 . . . L average
^φ (ω) B
Mathematically:
y (t) = y (j 1)M + t t = 1; :::;M
( )
j j
= the th subsequence
4
(j = 1; :::;L = [N=M ])
2
M
X
1
i! t
^
(! ) = y (t)e
j j
M
t=1
L
X
1
^ ^
(! ) = (! )
j
B
L
j =1
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–7 by P. Stoica and R. Moses, Prentice Hall, 1997 Comparison of Bartlett
and Blackman-Tukey Estimates
8 9
> >
M 1
L
< =
X X
1
i! k
^
^r (k )e (! ) =
j
B
> >
L
: ;
j =1
k = (M 1)
8 9
M 1
L
< =
X X
1
i! k
= ^r (k ) e
j
: ;
L
j =1
k = (M 1)
M 1
X
i! k
^r (k )e '
k = (M 1)
Thus:
^ ^
(! ) ' (! )
with a rectangular
B BT
(k )
lag window w
R
^
fw (k )g (! )
Since implicitly uses , the Bartlett
R B method has
High resolution (little smearing)
Large leakage and relatively large variance
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–8 by P. Stoica and R. Moses, Prentice Hall, 1997 Welch Method
Similar to Bartlett method, but
allow overlap of subsequences (gives more subsequences, and thus “better” averaging)
use data window for each periodogram; gives mainlobe-sidelobe tradeoff capability 12 N
. . . subseq subseq subseq
#1 #2 #S
= M
Let S # of subsequences of length .
> [N=M ] ) (Overlapping means S “better averaging”.)
Additional flexibility:
The data in each subsequence are weighted by a temporal
window
^
(! )
Welch is approximately equal to with a BT non-rectangular lag window.
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–9 by P. Stoica and R. Moses, Prentice Hall, 1997
Daniell Method
1
By a previous result, for N ,
^
f (! )g p
j are (nearly) uncorrelated random variables for
N 1
2
! = j
j
N
j =0
J + 1) Idea: “Local averaging” of (2 samples in the
frequency domain should reduce the variance by about
J + 1)
(2 .
k +J
X
1
^ ^
(! ) = (! )
p
j
D
k
2J + 1
j =k J
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–10 by P. Stoica and R. Moses, Prentice Hall, 1997 Daniell Method, con't
As J increases:
Bias increases (more smoothing)
Variance decreases (more averaging)
= 2J=N N 1
Let . Then, for ,
Z
1
^ ^
(! ) ' (! )d!
p
D
2
^ ^
(! ) ' (! )
Hence: with a rectangular spectral
D BT window.
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–11 by P. Stoica and R. Moses, Prentice Hall, 1997 Summary of Periodogram Methods
Unwindowed periodogram – reasonable bias – unacceptable variance
Modified periodograms – Attempt to reduce the variance at the expense of (slightly) increasing the bias.
BT periodogram
^
(! )
– Local smoothing/averaging of p by a suitably selected
spectral window.
r (k )
– Implemented by truncating and weighting ^ using a lag
^
(! )
window in c
Bartlett, Welch periodograms
^
(! )
– Approximate interpretation: with a suitable lag BT window (rectangular for Bartlett; more general for Welch). – Implemented by averaging subsample periodograms.
Daniell Periodogram
^
(! )
– Approximate interpretation: with a rectangular BT spectral window. – Implemented by local averaging of periodogram values.
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–12 by P. Stoica and R. Moses, Prentice Hall, 1997 Parametric Methods for Rational Spectra
Lecture 4
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–1 by P. Stoica and R. Moses, Prentice Hall, 1997 Basic Idea of Parametric Spectral Estimation
Observed Data ^ Estimate ^θ ^φ(ω) =φ(ω,θ) parameters Estimate
Assumed in φ(ω,θ) PSD functional form of φ(ω,θ)
possibly revise assumption on φ(ω)
Rational Spectra
P
i! k
e
k
jk jm
(! ) =
P
i! k
e
k
jk jn
i!
(! )
is a rational function in e .
(! )
By Weierstrass theorem, can approximate arbitrarily n well any continuous PSD,providedm and are chosen sufficiently large.
Note, however:
m n choice of and is not simple
some PSDs are not continuous
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–2 by P. Stoica and R. Moses, Prentice Hall, 1997
AR, MA, and ARMA Models
(! ) By Spectral Factorization theorem, a rational can be
factored as
2
B (! )
2
(! ) =
A(! )
1 n
A(z ) = 1 + a z + + a z
n
1
1 m
B (z ) = 1 + b z + + b z
m
1
A(! ) = A(z )j
and, e.g., i!
z =e
Signal Modeling Interpretation:
y (t) e(t)
B (q )
- -
2
B (! )
2 2
A(q )
(! ) = (! ) =
y e
A(! )
ltered white noise white noise
(q )y (t) = B (q )e(t)
ARMA: A
(q )y (t) = e(t)
AR: A
(t) = B (q )e(t) MA: y
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–3 by P. Stoica and R. Moses, Prentice Hall, 1997 ARMA Covariance Structure
ARMA signal model:
m n
X X
b e(t j ); (b = 1) a y (t i) = y (t)+
j 0 i
j =0 i=1
(t k ) E fg
Multiply by y and take to give:
n m
X X
r (k ) + a r (k i) = b E fe(t j )y (t k )g
i j
i=1 j =0
m
X
2
= b h
j
j k
j =0
= 0 > m
for k
1
X
B (q )
k
= h q ; (h = 1) H (q ) =
where 0
k
A(q )
k =0
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–4 by P. Stoica and R. Moses, Prentice Hall, 1997
AR Signals: Yule-Walker Equations
= 0 AR: m .
Writing covariance equation in matrix form for
= 1 ::: n
k :
2 3 3 2 2 3
2
1 r (0) r ( 1) ::: r ( n)
6 7 7 6 7
6 .
a (1) r (0)
r .
0
6 7 7 6 6 7
. 1
=
6 7 7 6 7
6 . . . .
4 5 5 4 4 5
( 1)
. .. r . .
r (n) ::: r (0) a
0
n
" # " #
2
1
R =
0 These are the Yule–Walker (YW) Equations.
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–5 by P. Stoica and R. Moses, Prentice Hall, 1997 AR Spectral Estimation: YW Method
Yule-Walker Method:
2
r (k ) r^(k ) f^a g ^
Replace by and solve for i and :
2 3 2 2 3 3
2
^r (0) ^r ( 1) ::: ^r ( n) 1
^
6 7 6 6 7
. 7
r (1) r^(0) ^a
^ .
0
6 7 6 6 7 7
. 1
=
6 7 6 6 7
. . . . 7
4 5 4 4 5 5
r ( 1)
. .. ^ . .
^r (n) ::: r^(0) ^a
0 n
Then the PSD estimate is
2
^
^
(! ) =
2
^
jA(! )j
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–6 by P. Stoica and R. Moses, Prentice Hall, 1997 AR Spectral Estimation: LS Method
Least Squares Method:
n
X
T
e(t) = y (t) + a y (t i) = y (t) + ' (t)
i
i=1
4
= y (t) + y^(t)
T
(t) = [y (t 1); :::;y(t n)]
where ' .
T
= [a ::: a ] n
Find 1 to minimize
N
X
2
f ( ) = je(t)j
t=n+1
1
^
= (Y Y ) (Y y )
This gives where
2 2 3 3
y (n +1) y (n) y (n 1) y (1)
y (n +2) y (n +1) y (n) y (2)
6 6 7 7
y = ; Y =
4 4 5 . 5 . .
. . .
y (N ) y (N 1) y (N 2) y (N n)
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–7 by P. Stoica and R. Moses, Prentice Hall, 1997 Levinson–Durbin Algorithm
Fast, order-recursive solution to YW equations
2 3 2 3 2 3
2
1
n
0 1
n
6 7 6 7 6
. . 7
. .
0
6 7 6 7 6
. 7 0
1 .
=
6 7 6 7 6
. . . 7 .
4 5 4 5 4 5 . . . .
. . n 1
. .
0
n
1 0
{z } |
R
n+1
= (k ) r^(k )
either r or . k
Direct Solution:
3
n O (n )
For one given value of : flops
4
k = 1;:: :;n O (n ) For : flops
Levinson–Durbin Algorithm:
Exploits the Toeplitz form of R to obtain the solutions
n+1
2
= 1; :::;n O (n ) for k in flops!
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–8 by P. Stoica and R. Moses, Prentice Hall, 1997 Levinson-Durbin Alg, con't
Relevant Properties of R :
T
] Rx = y $ R x~ = y~ x~ = [x :::x
,where
n 1
Nested structure
3 2
2 3
n
n+1
R
7 6
n+1 4
5 .
~r
R = ~r =
; .
5 4 n
n .
n+2
~r
0
n+1
n
1
Thus,
2 2 3 3 2 3 3 2
2
1 1
n
n+1
R
n+1
4 4 5 5 4 5 5 4
= = R
0
~r
n n
n
n+2
0 0
~r
n
0
n+1
n
= + ~r n
where n
n+1 n
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–9 by P. Stoica and R. Moses, Prentice Hall, 1997
Levinson-Durbin Alg, con't
2 3 2 3 3 2 2 3
2
0
1
n
n
4 5 4 5 5 4 4 5
~
0
R = ; R =
0
n
n
n+2 n+2
2
0
1
n n
Combining these gives:
8 9
2 3 2 3 2 3 3 2
2
2
< = 0
+ k 1
n
n n
n+1
4 5 4 5 4 5 5 4
~
0
R = = + k
n
0
n
n
n+2
: ;
2
0
1
+ k
0
n n
n
2
k = = ) n
Thus, n
n
# " # "
~
n
n
+ k =
n
n+1
0
1
2 2 2 2
= + k = (1 jk j )
n n
n n n
n+1
Computation count:
2k k ! k + 1
flops for the step
2 2 n
n f ; g
) flops to determine
k
k k =1
2
(n ) This is O times faster than the direct solution.
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–10 by P. Stoica and R. Moses, Prentice Hall, 1997
MA Signals
= 0
MA: n
y (t) = B (q )e(t)
= e(t) + b e(t 1) + + b e(t m)
m 1
Thus,
(k ) = 0 jk j > m r for
and
m
X
i! k 2 2
r (k )e (! ) = jB (! )j =
k = m
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–11 by P. Stoica and R. Moses, Prentice Hall, 1997
MA Spectrum Estimation
(! )
Two main ways to Estimate :
2
b g
1. Estimate f and and insert them in
k
2 2
(! ) = jB (! )j
nonlinear estimation problem
^
(! ) 0
is guaranteed to be
^r (k )g
2. Insert sample covariances f in:
m
X
i! k
(! ) = r (k )e
k = m
^
(! )
This is with a rectangular lag window of
BT
m + 1
length 2 .
^
(! ) 0 is not guaranteed to be
Both methods are special cases of ARMA methods
= 0 described below, with AR model order n .
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–12 by P. Stoica and R. Moses, Prentice Hall, 1997 ARMA Signals
ARMA models can represent spectra with both peaks
(AR part) and valleys (MA part).
A(q )y (t) = B (q )e(t)
P
2
m
i! k
e
B (! )
k
k = m
2
= (! ) =
2
A(! ) jA(! )j
where
= E f[B (q )e(t)][B (q )e(t k )] g
k
= E f[A(q )y (t)][A(q )y (t k )] g
n n
X X
a a r (k + p j ) =
j
p
j =0 p=0
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–13 by P. Stoica and R. Moses, Prentice Hall, 1997 ARMA Spectrum Estimation
Two Methods:
2
B (! )
2 2
fa ;b ; g (! ) =
j
1. Estimate i in
A(! )
nonlinear estimation problem; can use an approximate
linear two-stage least squares method
^
(! ) 0
is guaranteed to be
P
m
i! k
e
k
k = m
fa ; r (k )g (! ) =
2. Estimate i in
2
jA(! )j
linear estimation problem (the Modified Yule-Walker
method).
^
(! ) 0 is not guaranteed to be
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–14 by P. Stoica and R. Moses, Prentice Hall, 1997 Two-Stage Least-Squares Method
Assumption: The ARMA model is invertible:
A(q )
e(t) = y (t)
B (q )
= y (t) + y (t 1) + y (t 2) +
1 2
(1) j j ! 0 k ! 1
= AR with as k
Step 1: Approximate, for some large K
e(t) ' y (t) + y (t 1) + + y (t K )
1
K
K
g
1a) Estimate the coefficients f by using AR
k
k =1 modelling techniques.
1b) Estimate the noise sequence
^e (t) = y (t) + ^ y (t 1) + + ^ y (t K )
1 K
and its variance
N
X
1
2 2
^ = j^e(t)j
N K
t=K +1
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–15 by P. Stoica and R. Moses, Prentice Hall, 1997
Two-Stage Least-Squares Method, con't
e(t)g ^e(t)
Step 2: Replace f by in the ARMA equation,
A(q )y (t) ' B (q )^e (t)
fa ; b g j and obtain estimates of i by applying least squares
techniques.
a b j Note that the i and coefficients enter linearly in the
above equation:
y (t) ^e (t) ' [ y (t 1) ::: y (t n);
^e(t 1) :::^e(t m)]
T
= [a ::: a b :::b ]
n m
1 1
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–16 by P. Stoica and R. Moses, Prentice Hall, 1997 Modified Yule-Walker Method
ARMA Covariance Equation:
n
X
a r (k i) = 0; k > m r (k ) +
i
i=1
= m + 1;:::;m + M
In matrix form for k
2 3 3 2
" #
r (m) ::: r (m n +1) r (m +1)
a
1
r (m +1) r (m n +2) r (m +2)
6 7 7
. 6
. =
4 5 5 . . . . 4 .
. .. . .
a
n
r (m + M 1) ::: r (m n + M ) r (m + M )
fr (k )g f^r (k )g fa g
Replace by and solve for i .
= n
If M , fast Levinson-type algorithms exist for
f^a g
obtaining i .
> n Y W
If M overdetermined system of equations; least
f^a g
squares solution for i .
Note: For narrowband ARMA signals, the accuracy of
f^a g M > n
i is often better for
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–17 by P. Stoica and R. Moses, Prentice Hall, 1997 Summary of Parametric Methods for Rational Spectra
Computational Guarantee
0 ) ! ( Method Burden Accuracy ^ ? Use for AR: YW or LS low medium Yes Spectra with (narrow) peaks but no valley MA: BT low low-medium No Broadband spectra possibly with valleys but no peaks ARMA: MYW low-medium medium No Spectra with both peaks and (not too deep) valleys ARMA: 2-Stage LS medium-high medium-high Yes As above
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–18 by P. Stoica and R. Moses, Prentice Hall, 1997 Parametric Methods for Line Spectra — Part 1
Lecture 5
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–1 by P. Stoica and R. Moses, Prentice Hall, 1997 Line Spectra
Many applications have signals with (near) sinusoidal components. Examples:
communications
radar, sonar
geophysical seismology
ARMA model is a poor approximation
Better approximation by Discrete/Line Spectrum Models
(! )
6
2
6
(2 )
2
3
(2 )
6
1
2
(2 )
6
2
2
-
!
! ! !
1 2 3 An “Ideal” line spectrum
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–2 by P. Stoica and R. Moses, Prentice Hall, 1997 Line Spectral Signal Model
Signal Model: Sinusoidal components of frequencies
2
f! g f
and powers g, superimposed in white noise of
k
k 2
power .
y (t) = x(t) + e(t) t = 1; 2;: ::
n
X
i(! t+ )
k k
e x(t) =
k
| {z }
k =1
x (t) k
Assumptions:
> 0 ! 2 [ ; ]
A1:
k k
(prevents model ambiguities)
' g =
A2: f independent rv's, uniformly
k
; ] distributed on [
(realistic and mathematically convenient)
2
(t) =
A3: e circular white noise with variance
2
E fe(t)e (s)g = E fe(t)e(s)g = 0 t;s (can be achieved by “slow” sampling)
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–3 by P. Stoica and R. Moses, Prentice Hall, 1997 Covariance Function and PSD
Note that:
n o
i'
i'
p
j
e E e p = j 1
= ,for
n o n o n o
i' i'
i' i'
p p
j j
E e e = E e E e
Z
2
1
i'
= p 6= j d' = 0
e ,for
2
Hence,
n o
2 i! k
p
E x (t)x (t k ) = e
p
p;j
p
j
r (k ) = E fy (t)y (t k )g
P
n 2 i! k 2
p
= e +
k;0
p
p=1
and
n
X
2 2
(! ) = 2 (! ! ) +
p
p
p=1
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–4 by P. Stoica and R. Moses, Prentice Hall, 1997 Parameter Estimation
Estimate either:
2 n
; f! ; ;' g
(Signal Model)
k k k
k =1
2 n 2
; g f! ;
(PSD Model)
k
k =1 k
!^ g
Major Estimation Problem: f
k
!^ g
Once f are determined:
k
2
g f ^
can be obtained by a least squares method from
k
n
X
2 i!^ k
p
^r (k ) =
e + residuals
p
p=1
OR:
f ^ g f'^ g
Both and can be derived by a least
k k
squares method from
n
X
i!^ t
k
(t) = e
y + residuals
k
k =1
i'
k
= e
with .
k k
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–5 by P. Stoica and R. Moses, Prentice Hall, 1997
Nonlinear Least Squares (NLS) Method
2
N n
X X
i(! t+' )
k k
min e y (t)
k
f! ; ;' g
k k k
t=1
k =1
{z } |
F (! ; ;')
Let:
i'
k
= e
k k
T
= [ ::: ]
n
1
T
Y = [y (1) :::y(N )]
2 3
i! i!
n
1
e e 6
. . 7 =
B . . 4
. . 5
iN ! iN !
n
1
e e
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–6 by P. Stoica and R. Moses, Prentice Hall, 1997 Nonlinear Least Squares (NLS) Method, con't
Then:
2
F = (Y B ) (Y B ) = kY B k
1
= [ (B B ) B Y ] [B B ]
1
[ (B B ) B Y ]
1
+Y Y Y B (B B ) B Y
This gives:
1
^
= (B B ) B Y
! =^!
and
1
!^ = arg max Y B (B B ) B Y !
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–7 by P. Stoica and R. Moses, Prentice Hall, 1997 NLS Properties
Excellent Accuracy:
2
6
(^! ) = N 1)
var (for
k
2
3
N
k
= 300
Example: N
2 2
= = 30
SNR = dB
k
k
q
5
! ) 10
Then var (^ . k
Difficult Implementation:
The NLS cost function F is multimodal; it is difficult to avoid convergence to local minima.
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–8 by P. Stoica and R. Moses, Prentice Hall, 1997 Unwindowed Periodogram as an Approximate NLS Method
For a single (complex) sinusoid, the maximum of the
unwindowed periodogram is the NLS frequency estimate:
= 1
Assume: n
B = N
Then: B
N
X
i! t
y (t)e = Y (! ) Y =
B (finite DTFT)
t=1
1
1 2
Y B (B B ) B Y = jY (! )j
N
^
= (! ) p
= (Unwindowed Periodogram)
So, with no approximation,
^
!^ = arg max (! )
p !
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–9 by P. Stoica and R. Moses, Prentice Hall, 1997 Unwindowed Periodogram as an
Approximate NLS Method, con't
> 1 Assume: n
Then:
n
!^ g ' n
f the locations of the largest
k
k =1
^
(! )
peaks of p
provided that
inf j! ! j > 2=N
p k which is the periodogram resolution limit.
If better resolution desired then use a High/Super Resolution method.
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–10 by P. Stoica and R. Moses, Prentice Hall, 1997 High-Order Yule-Walker Method
Recall:
n
X
i(! t+' )
k k
y (t) = x(t) + e(t) = e +e(t)
k
| {z }
k =1
x (t)
k
(t)
“Degenerate” ARMA equation for y :
i! 1
k
(1 e q )x (t)
k
n o
i(! t+' ) i! i[! (t 1)+' ]
k k k k k
e e e = 0 = k
Let
L
X
4
k
b q = A(q )A(q ) B (q ) = 1 +
k
k =1
i! 1 i! 1
n
1
A(q ) = (1 e q ) (1 e q )
(q ) =
A arbitrary
(q )x(t) 0 )
Then B
B (q )y (t) = B (q )e(t)
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–11 by P. Stoica and R. Moses, Prentice Hall, 1997 High-Order Yule-Walker Method, con't
Estimation Procedure:
L
^
fb g
Estimate i using an ARMA MYW technique
i=1
n
^
B (q ) f!^ g L n
Roots of give , along with
k
k =1 “spurious” roots.
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–12 by P. Stoica and R. Moses, Prentice Hall, 1997 High-Order and Overdetermined YW Equations
ARMA covariance:
L
X
r (k ) + b r (k i) = 0; k > L
i
i=1
= L + 1;::: ;L + M
In matrix form for k
2 2 3 3
r (L + 1) r (L) ::: r (1)
6 6 7 7
r (L + 2) r (L + 1) ::: r (2)
6 6 7 7
b =
6 7 7
6 . . .
4 5 5
4 . . .
r (L + M 1) ::: r (M ) r (L + M )
| {z } | {z }
4 4
= =
> n
This is a high-order (if L ) and overdetermined
> L (if M ) system of YW equations.
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–13 by P. Stoica and R. Moses, Prentice Hall, 1997 High-Order and Overdetermined YW Equations,
con't
= n
Fact: rank( )
= U V
SVD of :
U U = I U = (M n)
with n
V = (n L) V V = I
with n
= (n n) , diagonal and nonsingular
Thus,
(U V )b =
The Minimum-Norm solution is
y 1
b = = V U
L n)
Important property: The additional ( spurious
(q ) zeros of B are located strictly inside the unit circle, if
the Minimum-Norm solution b is used.
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–14 by P. Stoica and R. Moses, Prentice Hall, 1997
HOYW Equations, Practical Solution
^
= f^r (k )g fr (k )g
Let but made from instead of .
^
^ ^
U V U V
Let , , be defined similarly to , , from the ^
SVD of .
1
^
^
^ ^
U ^ = V
Compute b
n
^
f!^ g n (q )
Then are found from the zeroes of B that
k
k =1 are closest to the unit circle.
When the SNR is low, this approach may give spurious
> n
frequency estimates when L ; this is the price paid for
> n increased accuracy when L .
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–15 by P. Stoica and R. Moses, Prentice Hall, 1997 Parametric Methods for Line Spectra — Part 2
Lecture 6
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–1 by P. Stoica and R. Moses, Prentice Hall, 1997 The Covariance Matrix Equation
Let:
i! i(m 1)! T
a(! ) = [1 e ::: e ]
A = [a(! ) ::: a(! )] (m n)
n
1
A) = n m n Note: rank( (for )
Define
2 3
y (t)
6 7
4
y (t 1)
6 7
y~(t) = = Ax~(t) + ~e (t) 6
. 7 4
. 5
y (t m + 1)
where
T
x~(t) = [x (t) :::x (t)]
n
1
T
~e (t) = [e(t) ::: e(t m + 1)]
Then
4
2
R = E fy~(t)~y (t)g = AP A + I
with
2 3
2
0
1
6 7
.
= E fx~(t)~x (t)g =
P . 4
. 5
2
0 n
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–2 by P. Stoica and R. Moses, Prentice Hall, 1997
Eigendecomposition of R and Its Properties
2
R = AP A + I (m > n)
Let:
::: R
m 2
1 : eigenvalues of
fs ;:::s g n
1 : orthonormal eigenvectors associated
f ;:: :; g n
with 1
fg ;:::;g g
m n
1 : orthonormal eigenvectors associated
f ;:::; g
with m
n+1
S = [s :::s ] (m n)
n
1
G = [g :::g ] (m (m n))
m n 1
Thus,
2 3
" #
1
S 6
. 7
= [S G]
R . 4
. 5
G
m
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–3 by P. Stoica and R. Moses, Prentice Hall, 1997 Eigendecomposition of R and Its Properties,
con't
AP A ) = n
As rank ( :
2
> k = 1;::: ;n
k
2
= k = n + 1;:::;m
k
3 2
2
0
1
7
6 .
= =
. 5
4 . nonsingular
2
0 n
Note:
2 3
0
1
6 7 2
.
= AP A S + S = S
RS . 4
. 5
0
n
4
1
S = A(PA S ) = AC
C j 6= 0 (S ) = (A) = n
with j (since rank rank ).
G = 0
Therefore, since S ,
A G = 0
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–4 by P. Stoica and R. Moses, Prentice Hall, 1997
MUSIC Method
2 3
a (! )
1
6 7
.
G = G = 0
A . 5
4 .
a (! )
n
n
) fa(! )g ? R(G)
k
k =1
Thus,
n
! g
f are the unique solutions of
k
k =1
(! )GG a(! ) = 0 a .
Let:
N
X
1
^
(t) y~(t)~y R =
N
t=m
^ ^
G = S;G
S; made from the ^ eigenvectors of R
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–5 by P. Stoica and R. Moses, Prentice Hall, 1997 Spectral and Root MUSIC Methods
Spectral MUSIC Method:
n
= !^ g n
f the locations of the highest peaks of the
k
k =1
“pseudo-spectrum” function:
1
; ! 2 [ ; ]
^ ^
a (! )G G a(! )
Root MUSIC Method:
n
!^ g = n
f the angular positions of the roots of:
k
k =1
T 1
^ ^
a (z )GG a(z ) = 0
that are closest to the unit circle. Here,
1 (m 1) T
a(z ) = [1;z ;:::;z ]
Note: Both variants of MUSIC may produce spurious frequency estimates.
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–6 by P. Stoica and R. Moses, Prentice Hall, 1997
Pisarenko Method
= n + 1 Pisarenko is a special case of MUSIC with m
(the minimum possible value).
= n + 1
If: m
^
G = ^g
Then: 1 ,
n
f!^ g
) can be found from the roots of
k
k =1
T 1
a (z )^g = 0 1
no problem with spurious frequency estimates
computationally simple
m n + 1 (much) less accurate than MUSIC with
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–7 by P. Stoica and R. Moses, Prentice Hall, 1997 Min-Norm Method
Goals: Reduce computational burden, and reduce risk of
false frequency estimates.
n
Uses m (as in MUSIC), but only one vector in
(G) R (as in Pisarenko).
Let
" #
1
^
(G)
the vector in R , with first element equal
= g ^ to one, that has minimum Euclidean norm.
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–8 by P. Stoica and R. Moses, Prentice Hall, 1997 Min-Norm Method, con't
Spectral Min-Norm
n
= !^ g n
f the locations of the highest peaks in the
k =1
“pseudo-spectrum”
# "
2
1
a (! ) 1 =
^g
Root Min-Norm
n
f!^ g n
= the angular positions of the roots of the
k =1
polynomial
" #
1
T 1
a (z )
^g
that are closest to the unit circle.
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–9 by P. Stoica and R. Moses, Prentice Hall, 1997 g
Min-Norm Method: Determining ^
" #
g 1
^ =
Let S
S g m 1
Then:
" # " #
1 1
^ ^
2 R(G) ) S = 0
^g ^g
) S ^g =
1
S ) g = S (S
Min-Norm solution: ^
1
^ ^
S S = + S S ) = S (S
As: I , exists iff
2
= k k 6= 1
1 (This holds, at least, for N .)
Multiplying the above equation by gives:
2
(1 k k ) = (S S )
1 2
) (S S ) = =(1 k k )
2
) ^g = S =(1 k k )
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–10 by P. Stoica and R. Moses, Prentice Hall, 1997
ESPRIT Method
A = [I 0]A
m 1
Let 1
A = [0 I ]A
2 m 1
A = A D 1
Then 2 ,where
3 2
i!
1
e 0 7
6 . =
D . 5
4 .
i!
n
0 e
S = [I 0]S
m 1
Also, let 1
S = [0 I ]S
2 m 1
= AC jC j 6= 0
Recall S with .Then
1
S = A C = A DC = S C DC
2 2 1 1
{z } |
D So has the same eigenvalues as . is uniquely
determined as
1
= (S S ) S S
1 1 1 2
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–11 by P. Stoica and R. Moses, Prentice Hall, 1997
ESPRIT Implementation
^ ^ ^
R S S
From the eigendecomposition of , find ,then 1 and
^ S
2 .
The frequency estimates are found by:
n
f!^ g = arg( ^ )
k k
k =1
n
^ g
where f are the eigenvalues of
k
k =1
1
^
^ ^ ^ ^
= (S S ) S S
1 1 1 2
ESPRIT Advantages:
computationally simple
no extraneous frequency estimates (unlike in MUSIC or Min–Norm)
accurate frequency estimates
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–12 by P. Stoica and R. Moses, Prentice Hall, 1997 Summary of Frequency Estimation Methods
Computational Accuracy / Risk for False Method Burden Resolution Freq Estimates Periodogram small medium-high medium Nonlinear LS very high very high very high Yule-Walker medium high medium Pisarenko small low none MUSIC high high medium Min-Norm medium high small ESPRIT medium very high none
Recommendation:
Use Periodogram for medium-resolution applications
Use ESPRIT for high-resolution applications
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–13 by P. Stoica and R. Moses, Prentice Hall, 1997 Filter Bank Methods
Lecture 7
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–1 by P. Stoica and R. Moses, Prentice Hall, 1997 Basic Ideas
Two main PSD estimation approaches:
(! ) 1. Parametric Approach: Parameterize by a finite-dimensional model.
2. Nonparametric Approach: Implicitly smooth
(! )g (! )
f by assuming that is nearly
! =
constant over the bands
[! ; ! + ]; 1
2 is more general than 1, but 2 requires
N > 1
to ensure that the number of estimated values
2=2 = 1= < N
(= )is .
> 1 N leads to the variability / resolution compromise associated with all nonparametric methods.
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–2 by P. Stoica and R. Moses, Prentice Hall, 1997 Filter Bank Approach to Spectral Estimation
c
!
!
6
1
) ! ( H
-
bandwidth xed
le bandwidth lter Calculation
c
and ! arying v with
er w o P
h band the Signal
y b Division c
) ! ( ) t ( y
- - - -
^
Filter Bandpass
rin er w o P Filtered
6 6
- -
c c
! !
! !
Q
Q
@ @ @ @
' ' ) ! ( '
1 1
) a ) b ) c
FB ( ( (
) ! (
^
y ~ y
) ! ( ) ! (
!
2 2
= d ) ( j ) ( H j = d ) (
2
+ ! (a) consistentZ power calculation Z
(b) Ideal passband filter with bandwidth
] (c)) ( +2 ! ; constant2 ! [ on2
Note that assumptions (a) and (b), as well as (b) and (c), are conflicting.
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–3 by P. Stoica and R. Moses, Prentice Hall, 1997
Filter Bank Interpretation of the Periodogram
2
N
X
1
4
i!t~
^
y (t)e (~! ) =
p
N
t=1
2
N
X
1
i!~ (N t)
= y (t)e
N
t=1
2
1
X
h y (N k ) = N
k
k =0
where
(
1
i!k~
e ; k = 0;::: ;N 1
N
h =
k ;
0 otherwise
1
iN (~! ! )
X
1 e 1
i! k
H (! ) = h e =
k
i(~! ! )
N
1 e
k =0
(! ) = !~
center frequency of H
H (! ) ' 1=N 3dB bandwidth of
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–4 by P. Stoica and R. Moses, Prentice Hall, 1997 Filter Bank Interpretation of the Periodogram,
con't
H (! )j (~! ! ) N = 50 j as a function of ,for . 0
−5
−10
−15
−20 dB
−25
−30
−35
−40 −3 −2 −1 0 1 2 3
ANGULAR FREQUENCY
^
(! )
Conclusion: The periodogram p is a filter bank PSD estimator with bandpass filter as given above, and:
narrow filter passband,
power calculation from only 1 sample of filter output.
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–5 by P. Stoica and R. Moses, Prentice Hall, 1997 Possible Improvements to the Filter Bank Approach
1. Split the available sample, and bandpass filter each subsample.
more data points for the power calculation stage.
This approach leads to Bartlett and Welch methods.
2. Use several bandpass filters on the whole sample. ! Each filter covers a small band centered on ~ .
provides several samples for power calculation.
This “multiwindow approach” is similar to the Daniell method.
Both approaches compromise bias for variance, and in fact are quite related to each other: splitting the data sample can be interpreted as a special form of windowing or filtering.
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–6 by P. Stoica and R. Moses, Prentice Hall, 1997 Capon Method
Idea: Data-dependent bandpass filter design.
m
X
h y (t k ) y (t) =
F
k
k =0
2 3
y (t) 6
. 7
[h h ::: h ]
= .
4 5
m 1
0 .
| {z }
y (t m)
h
{z } |
y~(t)
n o
2
E jy (t)j = h Rh; R = E fy~(t)~y (t)g
F
m
X
i! k
H (! ) = h e = h a(! )
k
k =0
i! im! T
(! ) = [1;e :: : e ] where a
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–7 by P. Stoica and R. Moses, Prentice Hall, 1997 Capon Method, con't
Capon Filter Design Problem:
(h Rh) h a(! ) = 1
min subject to
h
1 1
= R a=a R a
Solution: h 0
The power at the filter output is:
n o
2 1
E jy (t)j = h Rh = 1=a (! )R a(! )
0
F
0
(t) which should be the power of y in a passband centered
on ! .
1 1 =
The Bandwidth ' +1 m (filter length)
Conclusion Estimate PSD as:
m + 1
^
(! ) =
1
^
a (! )R a(! )
with
N
X
1
^
y~(t)~y (t) R =
N m
t=m+1
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–8 by P. Stoica and R. Moses, Prentice Hall, 1997
Capon Properties m is the user parameter that controls the compromise between bias and variance:
– as m increases, bias decreases and variance increases.
Capon uses one bandpass filter only, but it splits the
N m N -data point sample into ( ) subsequences of
length m with maximum overlap.
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–9 by P. Stoica and R. Moses, Prentice Hall, 1997 Relation between Capon and Blackman-Tukey
Methods
^
(! )
Consider with Bartlett window:
BT
m
X
m + 1 jk j
i! k
^
r^(k )e (! ) =
BT
m + 1
k = m
m m
X X
1
i! (t s)
= ^r (t s)e
m + 1
t=0 s=0
^
a (! )Ra(! )
^
; R = [^r (i j )] =
m + 1
Then we have
^
a (! )Ra(! )
^
(! ) =
BT
m + 1
m + 1
^
(! ) =
C
1
^
a (! )R a(! )
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–10 by P. Stoica and R. Moses, Prentice Hall, 1997 Relation between Capon and AR Methods
Let
2
^
AR k
^
(! ) =
k
2
^
jA (! )j
k
y (t) be the k th order AR PSD estimate of .
Then
1
^
(! ) =
C
m X
1 AR
^
1= (! )
k
m + 1
k =0
Consequences:
^
k (! )
Due to the average over , generally has less C statistical variability than the AR PSD estimator.
Due to the low-order AR terms in the average,
^
(! )
generally has worse resolution and bias C properties than the AR method.
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–11 by P. Stoica and R. Moses, Prentice Hall, 1997 Spatial Methods — Part 1
Lecture 8
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–1 by P. Stoica and R. Moses, Prentice Hall, 1997
The Spatial Spectral Estimation Problem
v
Source 2
Source 1
B
v
B
Source n
v
@
B
,
,
,
@
B
,
,
,
c
c
,
c
@
BN
,
,
c
@R
c
c ,
c
,
c
c
,
c
c
c
@
@
@
@
@
@
Sensor m
Sensor 1
Sensor 2
Problem: Detect and locate n radiating sources by using
an array of m passive sensors. Emitted energy: Acoustic, electromagnetic, mechanical Receiving sensors: Hydrophones, antennas, seismometers Applications: Radar, sonar, communications, seismology, underwater surveillance Basic Approach: Determine energy distribution over space (thus the name “spatial spectral analysis”)
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–2 by P. Stoica and R. Moses, Prentice Hall, 1997 Simplifying Assumptions
Far-field sources in the same plane as the array of sensors
Non-dispersive wave propagation
Hence: The waves are planar and the only location parameter is direction of arrival (DOA)
(or angle of arrival, AOA). n The number of sources is known. (We do not treat the detection problem)
The sensors are linear dynamic elements with known transfer characteristics and known locations (That is, the array is calibrated.)
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–3 by P. Stoica and R. Moses, Prentice Hall, 1997
Array Model — Single Emitter Case
(t) = x the signal waveform as measured at a reference
point (e.g.,atthe“first” sensor) =
the delay between the reference point and the k
k th sensor
(t) =
h the impulse response (weighting function) of k
sensor k
e (t) = k
“noise” at the th sensor (e.g.,thermalnoisein k
sensor electronics; background noise, etc.)
2 R Note: t (continuous-time signals).
Then the output of sensor k is
y (t) = h (t) x(t ) + e (t)
k k k k =
( convolution operator).
g h (t)
Basic Problem: Estimate the time delays f with
k k
(t) known but x unknown.
This is a time-delay estimation problem in the unknown input case.
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–4 by P. Stoica and R. Moses, Prentice Hall, 1997 Narrowband Assumption
Assume: The emitted signals are narrowband with known !
carrier frequency c .
x(t) = (t) cos[! t + '(t)]
Then: c
(t); '(t)
where vary “slowly enough” so that
(t ) ' (t); '(t ) ' '(t)
k k
' !
Time delay is now to a phase shift c :
k
x(t ) ' (t)cos[! t + '(t) ! ]
c c
k k
h (t) x(t )
k k
' jH (! )j (t) cos[! t + '(t) ! +arg fH (! )g]
c c c c
k k k
(! ) = Ffh (t)g k
where H is the th sensor's transfer
k k function
Hence, the k th sensor output is
y (t) = jH (! )j (t)
c
k k
cos[! t + '(t) ! + arg H (! )] + e (t)
c c c
k k k
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–5 by P. Stoica and R. Moses, Prentice Hall, 1997 Complex Signal Representation
The noise-free output has the form:
z (t) = (t)cos [! t + (t)] =
c
n o
(t)
i[! t+ (t)] i[! t+ (t)]
c c
e + e =
2
(t)
Demodulate z (translate to baseband):
! t i (t) i[2! t+ (t)]
c c
2z (t)e = (t)f e + e g
| | {z } {z }
lowpass highpass
i! t i (t)
c
z (t)e (t)e Lowpass filter 2 to obtain
Hence,bylow-passfiltering and sampling the signal
i! t
c
y~ (t)=2 = y (t)e
k k
= y (t)cos(! t) iy (t) sin(! t)
c c
k k
2 Z
we get the complex representation: (for t )
i'(t) i arg[H (! )] i!
c c
k k
y (t) = (t) e jH (! )j e e + e (t)
c
k k k
| {z }
| {z }
s(t)
H (! )
c k
or
i!
c
k
y (t) = s(t)H (! ) e + e (t)
c
k k k
(t) x(t) where s is the complex envelope of .
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–6 by P. Stoica and R. Moses, Prentice Hall, 1997 Vector Representation for a Narrowband Source
Let =
the emitter DOA =
m the number of sensors
2 3
i!
c
1
H (! ) e
c
1 6
. 7
( ) =
a . 4
. 5
i!
c m
H (! ) e
m c
3 2 3 2
e (t) y (t)
1 1
7 6 7
6 . .
e(t) = (t) =
y . .
5 4 5
4 . .
y (t) e (t)
m m
Then
y (t) = a( )s(t) + e(t)
a( ) f g fH (! )g
NOTE: enters via both and c .
k k
fH (! )g
For omnidirectional sensors the c do not depend k
on .
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–7 by P. Stoica and R. Moses, Prentice Hall, 1997 Analog Processing Block Diagram
k
) t ( x
Analog processing for each receiving array element !
!
Signal
Incoming
ransducer T
,
:
,
,
Filter
l
l
wpass Lo
X z
! - - - -
l
X
!
X
X
Filtering
X
X
X
Noise
?
6
and
External
k
Cabling
) t ( y
Noise
- -
Oscillator
ternal In
Filter
A/D
?
SENSOR
c
wpass Lo
) t ! ( -sin
- - - -
k k
)] t ( y Im[ )] t ( ~ y Im[
TION DEMODULA
c
) t ! ( cos
A/D
k k
)] t ( y Re[ )] t ( ~ y Re[
CONVERSION
Lecture notes to accompany Introduction to Spectral Analysis A/D Slide L8–8 by P. Stoica and R. Moses, Prentice Hall, 1997 Multiple Emitter Case
Given n emitters with
n
fs (t)g
received signals:
k
k =1
DOAs: k
Linear sensors )
y (t) = a( )s (t) + + a( )s (t) + e(t)
n n
1 1
Let
A = [a( ) :::a( )]; (m n)
n
1
T
s(t) = [s (t) :::s (t)] ; (n 1)
n 1
Then, the array equation is:
y (t) = As(t) + e(t)
Use the planar wave assumption to find the dependence of
on . k
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–9 by P. Stoica and R. Moses, Prentice Hall, 1997
Uniform Linear Arrays
Source
h
J
J
J
J
J
J
J
J
J^
J
J
J
-
J
J
+
J
J]
J
J
d sin
J
J
J^
J
J
J
J
Sensor
]
J
v v v v v
?
J
p p p
1 2 3 4 m
J
J
J
- d ULA Geometry
Sensor #1 = time delay reference
Time Delay for sensor k :
d sin
= (k 1)
k c
where c = wave propagation speed
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–10 by P. Stoica and R. Moses, Prentice Hall, 1997 Spatial Frequency
Let:
d sin d sin d sin
4
= ! ! = 2 = 2
c s
c c=f
c
= c=f =
c signal wavelength
i! i(m 1)! T
s s
a( ) = [1;e ::: e ]
(! ) By direct analogy with the vector a made from
uniform samples of a sinusoidal time series,
! =
s spatial frequency
! 7! a( )
The function s is one-to-one for
dj sin j
j! j $ 1 d =2
s
=2
As =
d spatial sampling period
=2 d is a spatial Shannon sampling theorem.
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–11 by P. Stoica and R. Moses, Prentice Hall, 1997 Spatial Methods — Part 2
Lecture 9
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–1 by P. Stoica and R. Moses, Prentice Hall, 1997 Spatial Filtering
Spatial filtering useful for
DOA discrimination (similar to frequency discrimination of time-series filtering)
Nonparametric DOA estimation
There is a strong analogy between temporal filtering and spatial filtering.
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–2 by P. Stoica and R. Moses, Prentice Hall, 1997 Analogy between Temporal and Spatial Filtering
Temporal FIR Filter:
m 1
X
h u(t k ) = h y (t) y (t) =
F
k
k =0
h = [h :::h ]
o
m 1
T
y (t) = [u(t) :::u(t m + 1)]
i! t
(t) = e
If u then
y (t) = [h a(! )] u(t)
F
| {z }
filter transfer function
i! i(m 1)! T
a(! ) = [1;e ::: e ]
We can select h to enhance or attenuate signals with
different frequencies ! .
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–3 by P. Stoica and R. Moses, Prentice Hall, 1997 Analogy between Temporal and Spatial Filtering
Spatial Filter:
m
= y (t)g
f the “spatial samples” obtained with a
k
k =1 sensor array.
Spatial FIR Filter output:
m
X
h y (t) = h y (t) y (t) =
F
k k
k =1 Narrowband Wavefront: The array's (noise-free)
response to a narrowband ( sinusoidal) wavefront with
(t)
complex envelope s is:
y (t) = a( )s(t)
i! T i!
c m c
2
::: e ] a( ) = [1;e
The corresponding filter output is
y (t) = [h a( )] s(t)
F
| {z } filter transfer function
We can select h to enhance or attenuate signals coming from different DOAs.
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–4 by P. Stoica and R. Moses, Prentice Hall, 1997
Analogy between Temporal and Spatial Filtering
h
0
i! t
u(t)=e
?
-
z s -
1 0
@
1
@
?
C B
h
1
1
@
C B
q
1
C B
?
@
i!
[h a(! )]u(t) @R
C B
P
e
s - - -
-
C B
C B
?
C B
C B
q
u(t)
.
C B
q .
C B
.
q
q
C B
C B
q
C B
q
C B
?
A @
1
q
m 1
i(m 1)!
h
m 1
e
{z } |
?
-
-
a(! )
(Temp oral sampling)
(a) Temporal filter
narrowband source with DOA=
z
Z
Z
Z
Z
h
Z~
0
?
1
a
-
a
-
reference p oint
!
@
!
1 0
@
1
h
1
@
C B
C B
?
@
@R
[h a( )]s(t) 2
a P
C B
a -
i! ( )
- -
2
-
!
C B
e
!
C B
C B
s(t) C B
C B
.
C B
.
q
.
C B
q
q
C B
q
q
A @
q
i! ( )
m
e
h
m 1
{z } |
?
a( )
m
a
-
a
-
!
! (Spatial sampling)
(b) Spatial filter
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–5 by P. Stoica and R. Moses, Prentice Hall, 1997
Spatial Filtering, con't
h a( )j Example: The response magnitude j of a spatial
filter (or beamformer) for a 10-element ULA. Here,
h = a( ) = 25 0 0 ,where 0
30 −30 Theta (deg)
60 −60
90 −90 0 2 4 6 8 10 Magnitude
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–6 by P. Stoica and R. Moses, Prentice Hall, 1997 Spatial Filtering Uses
Spatial Filters can be used
To pass the signal of interest only, hence filtering out interferences located outside the filter's beam (but possibly having the same temporal characteristics as the signal).
To locate an emitter in the field of view, by sweeping the filter through the DOA range of interest (“goniometer”).
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–7 by P. Stoica and R. Moses, Prentice Hall, 1997 Nonparametric Spatial Methods
A Filter Bank Approach to DOA estimation.
Basic Ideas
h( ) Design a filter such that for each
– It passes undistorted the signal with DOA =
= – It attenuates all DOAs 6
Sweep the filter through the DOA range of interest,
and evaluate the powers of the filtered signals:
o o n n
2 2
= E jh ( )y (t)j E jy (t)j
F
= h ( )Rh( )
= E fy (t)y (t)g
with R .
h ( )Rh( ) The (dominant) peaks of give the DOAs of the sources.
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–8 by P. Stoica and R. Moses, Prentice Hall, 1997 Beamforming Method
Assume the array output is spatially white:
R = E fy (t)y (t)g = I
o n
2
= h h jy (t)j
Then: E F
Hence: In direct analogy with the temporally white
(t) assumption for filter bank methods, y can be considered as impinging on the array from all DOAs.
Filter Design:
(h h) h a( ) = 1
min subject to h
Solution:
h = a( )=a ( )a( ) = a( )=m
n o
2 2
E jy (t)j = a ( )Ra( )=m F
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–9 by P. Stoica and R. Moses, Prentice Hall, 1997
Implementation of Beamforming
N
X
1
^
R = y (t)y (t)
N
t=1
The beamforming DOA estimates are:
^
g = n
f the locations of the largest peaks of
k
^
( )Ra( ) a . This is the direct spatial analog of the Blackman-Tukey periodogram.
Resolution Threshold:
wavelength
inf j j > p k array length
= array beamwidth
Inconsistency problem:
= 1
Beamforming DOA estimates are consistent if n ,but
> 1 inconsistent if n .
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–10 by P. Stoica and R. Moses, Prentice Hall, 1997 Capon Method
Filter design:
(h Rh) h a( ) = 1
min subject to h
Solution:
1 1
h = R a( )=a ( )R a( )
n o
2 1
E jy (t)j = 1=a ( )R a( ) F
Implementation:
^
g = n
f the locations of the largest peaks of
k
1
^
1=a ( )R a( ):
Performance: Slightly superior to Beamforming.
Both Beamforming and Capon are nonparametric approaches. They do not make assumptions on the covariance properties of the data (and hence do not depend on them).
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–11 by P. Stoica and R. Moses, Prentice Hall, 1997 Parametric Methods
Assumptions:
The array is described by the equation:
y (t) = As(t) + e(t)
The noise is spatially white and has the same power in
all sensors:
2
E fe(t)e (t)g = I
The signal covariance matrix
P = E fs(t)s (t)g
is nonsingular.
Then:
2
R = E fy (t)y (t)g = AP A + I
Thus: The NLS, YW, MUSIC, MIN-NORM and ESPRIT methods of frequency estimation can be used, almost without modification, for DOA estimation.
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–12 by P. Stoica and R. Moses, Prentice Hall, 1997
Nonlinear Least Squares Method
N
X
1
2
min ky (t) As(t)k
N
f g; fs(t)g
k
t=1
{z } |
f (;s) s
Minimizing f over gives
1
^s(t) = (A A) A y (t); t = 1; :::;N
Then
N
X
1
1 2
f (; ^s ) = k [I A(A A) A ]y (t)k
N
t=1
N
X
1
1
= y (t)[I A(A A) A ]y (t)
N
t=1
1
^
f[I A(A A) A ]R g
= tr
1
^
^
g = arg max f[A(A A) A ]R g
Thus, f tr
k
f g
k
= 1 For N , this is precisely the form of the NLS method of frequency estimation.
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–13 by P. Stoica and R. Moses, Prentice Hall, 1997 Nonlinear Least Squares Method
Properties of NLS:
Performance: high
Computational complexity: high
Main drawback: need for multidimensional search.
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–14 by P. Stoica and R. Moses, Prentice Hall, 1997
Yule-Walker Method
" # " # " #
y(t) A e(t)
y (t) = = s(t) +
~
y~(t) ~e(t)
A
fe (t)~e (t)g = 0 Assume: E
Then:
4
~
= E fy(t)~y (t)g = AP A (M L)
Also assume:
M > n; L > n ( ) m = M + L > 2n)
~
(A) = (A) = n
rank rank
= n
Then: rank( ) ,andtheSVDof is
" # " #
0 V g n
nn
1
= [ U U ]
1 2
|{z} |{z}
0 0 V g L n
2
n
M n
~ ~
A V = 0 V 2 R(A)
2 Properties: 1
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–15 by P. Stoica and R. Moses, Prentice Hall, 1997
YW-MUSIC DOA Estimator
^
g = n
f the largest peaks of
k
^ ^
~a ( ) 1=~a ( )V V
2 2
where
~a ( ) (L 1) y~(t) , ,isthe“array transfer vector” for
at DOA
^
V V 2
2 is defined similarly to ,using
N
X
1
^
= y(t)~y (t)
N
t=1
Properties:
Computational complexity: medium
m 2n Performance: satisfactory if
Main advantages:
e(t)g
– weak assumption on f – the subarray A need not be calibrated
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–16 by P. Stoica and R. Moses, Prentice Hall, 1997 MUSIC and Min-Norm Methods
Both MUSIC and Min-Norm methods for frequency estimation apply with only minor modifications to the DOA estimation problem.
Spectral forms of MUSIC and Min-Norm can be used for arbitrary arrays
Root forms can be used only with ULAs
MUSIC and Min-Norm break down if the source
signals are coherent; that is, if
P ) = (E fs(t)s (t)g ) < n rank( rank Modifications that apply in the coherent case exist.
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–17 by P. Stoica and R. Moses, Prentice Hall, 1997 ESPRIT Method
Assumption: The array is made from two identical subarrays separated by a known displacement vector.
Let
m =
# sensors in each subarray
A = [I 0]A
m
1 (transfer matrix of subarray 1)
A = [0 I ]A
m
2 (transfer matrix of subarray 2)
A = A D 1
Then 2 ,where
3 2
i! ( )
c
1
e 0 7
6 . =
D . 5
4 .
i! ( )
c n
0 e
( ) = the time delay from subarray 1 to
subarray 2 for a signal with DOA = :
( ) = d sin ( )=c where d is the subarray separation and is measured from the perpendicular to the subarray displacement vector.
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–18 by P. Stoica and R. Moses, Prentice Hall, 1997 ESPRIT Method, con't
ESPRIT Scenario source
θ
known displ acement vector subarray 1
subarray 2
Properties:
Requires special array geometry
Computationally efficient
No risk of spurious DOA estimates
Does not require array calibration
Note: For a ULA, the two subarrays are often the first
1 m 1 m = m 1
m and last array elements, so and
A = [I 0]A; A = [0 I ]A
1 m 1 2 m 1
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–19 by P. Stoica and R. Moses, Prentice Hall, 1997