Digital Signal Processing Lecture # 4 Convolution, Autocorrelation, and Cross-Correlation 1 Problem Set #1 Is Due Wednesday

Announcements Digital Signal Processing Lecture # 4 Convolution, Autocorrelation, and Cross-Correlation 1 Problem Set #1 is due Wednesday.

2 You may want to read the handout entitled Monson H. Hayes Discrete-Time Signals and Systems [email protected] from the book Schaum’s Outline on Digital Signal Processing that may be found by clicking on the References tab on the course web Chung-Ang University page. Seoul, Korea

3 You may also want to download the Discrete Convolution Demo that may be found in the Demos section of the course web page. This will give you the opportunity to see graphically how convolution is performed. Spending some time with this demo will be worth the This material is the property of the author and is for the sole and exclusive use of his students. It is not for publication, nor is it to be sold, reproduced, or generally distributed. eﬀort.

M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 1 / 52 M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 2 / 52

Review Performing Convolutions

When the input to a linear system is x(n), the output is The convolution sum is given by ∞ X y(n) = x(k)h (n) ∞ k X k=−∞ y(n) = x(k)h(n − k) k=−∞ This is called the superposition sum. The signal hk (n) is the response of the linear system to the input x(n) = δ(n − k). The steps involved in performing a convolution are: For a system that is both linear and shift-invariant (LSI) 1 Consider x(k) as a function of k. h (n) = h (n − k) ≡ h(n − k) k 0 2 Consider h(n − k) as a function of k for some ﬁxed value of n.

and the superposition sum becomes the convolution sum 3 Form the product sequence x(k)h(n − k) which is a sequence in k, ∞ parameterized by n. X y(n) = x(k)h(n − k) ≡ x(n) ∗ h(n) 4 Sum all the samples of this product to generate the nth sample of y(n). k=−∞ where h(n) is the unit sample response, i.e, the response of the LSI – Repeat for all n – system to a unit sample δ(n).

M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 3 / 52 M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 4 / 52 Some Useful Formulas Performing Convolutions Using the “Slide Rule Approach”

1 Write x(n) along the top of a piece of paper as a sequence of N−1 X 1 numbers. 1 n = N(N − 1) 2 2 Write h(−n) along the top of another piece of paper as a sequence of n=0 numbers. N−1 N X n 1 − r 2 r = 3 Line up the two pieces of paper, multiply each pair of numbers and 1 − r n=0 add the products to form the value of y(0). ∞ 4 X n 1 Slide the paper to the left or right to calculate y(n) for other values 3 r = provided |r| < 1. 1 − r of n. n=0 ∞ X n r 4 nr = provided |r| < 1. ··· x(−2) x(−1) x(0) x(1) x(2) x(3) ··· 1 − r 2 n=0 ··· h(2) h(1) h(0) h(−1) h(−2) h(−3) ···

M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 5 / 52 M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 6 / 52

Examples Convolution of Two Finite-Length Sequences

1 Perform the convolution of x(n) with h(n) where Suppose that x(n) is a signal that is ﬁnite in length and 1 The ﬁrst non-zero value occurs at index n = N1, 2 The last non-zero value occurs at index n = N2. x(n) = [1, 3, 2, −1, 0, 2] h(n) = [0, 1, 1, −1, −1, 2, 0] It then follows that the length of x(n) is N = N1 + N2 + 1 samples.

Suppose that h(n) is a signal that is ﬁnite in length and

1 The ﬁrst non-zero value occurs at index n = M1, 2 The last non-zero value occurs at index n = M2. It then follows that the length of h(n) is M = M1 + M2 + 1 samples.

What is the location of the ﬁrst and last non-zero value of y(n) = x(n) ∗ h(n) and what is the length of y(n)?

M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 7 / 52 M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 8 / 52 Convolution of Inﬁnite-Length Sequences Another Example

n The results above hold most of the time when the signals are inﬁnite in Find the convolution of x(n) = z0 with h(n) where z0 is an arbitrary length. But, there are some exceptions. (complex) constant.

Example: Consider the convolution of Solution: ∞ ∞ x(n) = anu(n) h(n) = δ(n) − aδ(n − 1) n X n−k n X −k n z0 ∗ h(n) = h(k)z0 = z0 h(k)z0 = H(z0)z0 k=−∞ k=−∞ The convolution is where ∞ y(n) = x(n) ∗ h(n) = x(n) ∗ δ(n) − ax(n) ∗ δ(n − 1) X −k H(z0) = h(k)z0 What is this convolution equal to? k=−∞ is a complex constant whose value depends on z0.

M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 9 / 52 M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 10 / 52

Eigenfunctions of LSI systems – Eigensignals System Function

Signals of the form

Recall the notion of an eigenvector from linear algebra: x(n) = zn − ∞ < n < ∞

Ax = λx are the eigenfunctions (or eigensignals) and

where λ is the eigenvalue and x is the eigenvector. ∞ X H(z) = h(k)z−k Eigenfunctions of LSI systems are those signals which, when used as k=−∞ inputs, pass through the system with only a change in (complex) amplitude are the eigenvalues. Deﬁnition x(n) y(n) = λx(n) - - ∞ h(n) X H(z) = h(k)z−k k=−∞ is the system function of the LSI system with unit sample response h(n).

M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 11 / 52 M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 12 / 52 Examples Examples

Find the system function of the systems having the following unit sample 2 h(n) = δ(n + 1) + 2δ(n) + δ(n − 1) responses: 1 h(n) = δ(n − 1)

x(n) y(n) = x(n − 1) - z−1 -

z−1 x(n) - x(n − 1)

M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 13 / 52 M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 14 / 52

Examples Frequency Response

jω 3 h(n) = (0.9)nu(n) A special case occurs when z is a complex exponential, z = e , x(n) = ejnω =⇒ y(n) = H(ejω)ejnω

x(n) = ejnω y(n) = H(ejω)ejnω - h(n) -

∞ X H(ejω) = h(k)e−jkω k=−∞ Deﬁnition The frequency response of an LSI system with unit sample response h(n) is

∞ X H(ejω) = h(n)e−jnω n=−∞

M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 15 / 52 M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 16 / 52 Relationship Between H(z) and H(ejω) Examples

∞ 1 Example 1: Ideal delay. X H(z) = h(n)z−n n=−∞ y(n) = x(n − n0) =⇒ h(n) = δ(n − n0) ∞ ∞ ∞ jω X −jnω X −n H(e ) = h(n)e = h(n)z jω X −jnω −jn0ω z=ejω H(e ) = δ(n − n0)e = e n=−∞ n=−∞ n=−∞ So, H(ejω) is equal to H(z) evaluated around the unit circle, z = ejω

M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 17 / 52 M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 18 / 52

Examples Examples

2 Example 2: h(n) = anu(n), |a| < 1. 3 Example 3: h(n) = δ(n) − aδ(n − 1).

M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 19 / 52 M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 20 / 52 Examples Autocorrelation

4 Example 4: h(n) = u(n) − u(n − N). The autocorrelation of a sequence x(n) is deﬁned by:

This is an N-point Moving Average Filter. ∞ X rx (k) = x(k) ∗ x(−k) = x(n)x(n + k) n=−∞

The index k is called the lab. Note that in performing the convolution of x(k) with x(−k), to ﬁnd the value of rx (0), we ﬁrst time-reverse x(−n), which gives us x(n), multiply this by x(n) and sum over all n. note that ∞ X 2 rx (0) = x (n) n=−∞ For values of k 6= 0, x(n) is shifted by n, the product x(n)x(n + k) is formed, and the sum of all terms is found.

M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 21 / 52 M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 22 / 52

Example Properties of the Autocorrelation

Find the autocorrelation of The autocorrelation appears in applications such as communications (matched ﬁltering), signal modeling (normal equations), and pattern x(n) = [1, −1, 1, −1, 1, −1, ] recognition (template matching). The autocorrelation is a symmetric sequence:

rx (k) = rx (−k)

This may be seen by noting that

∞ ∞ X X rx (k) = x(n)x(n + k) = x(l − k)x(l) = rx (−k) n=−∞ l=−∞

where in the second equality we made the substitution l = n + k.

M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 23 / 52 M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 24 / 52 Symmmetry of the Autocorrelation Properties of the Autocorrelation We may also see graphically that the autocorrelation is symmetric. Another property is that rx (0) ≥ |rx (k)| for all k.

∞ X {x(n) ± x(n − k)}2 ≥ 0 n=−∞

∞ X x2(n) ± 2x(n)x(n − k) + x2(n − k) ≥ 0 n=−∞ ∞ ∞ X X x2(n) ≥ ± x(n)x(n − k) n=−∞ n=−∞

rx (0) ≥ |rx (k)|

M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 25 / 52 M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 26 / 52

Example Cross-correlation

The cross-correlation between two sequences x(n) and y(n) is deﬁned x(n) = anu(n) by: ∞ X ∞ x(n) ∗ y(−n) = x(k)y(n + k) X n n−k k=−∞ rx (k) = a u(n)a u(n − k) n=−∞ The cross-correlation also appears in many applications such as

For k ≥ 0, ∞ ∞ communications, signal modeling, and pattern recognition. X n n−k X n+k n rx (k) = a a = a a Find the cross-correlation of n=k n=0 ∞ x(n) = [1, 0, 2, 1] ; y(n) = [1, 1, 2, 1] X ak = ak a2n = 1 − a2 n=0 Since the autocorrelation is symmetric, we may then write a|k| r (k) = x 1 − a2

M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 27 / 52 M.Hayes (CAU-GT) Lecture # 4 September 17, 2012 28 / 52