sign in sign up
<<
Home , Convolution theorem, Fourier analysis, Fourier transform

Transform and its applications Correlation Applications of Notes

So far, only considered Fourier transform as a way to obtain the spectrum of a /. However, there are other important applications: Convolution: Real physical systems can smear out input due to the finite response time of the apparatus. e.g. if you send an impulse (δ-function) in, the response will be different. Fourier can deconvolve the response of the apparatus to recover the true input signal. Correlation: Correlations are used to compare two signals and test if they are correlated or not (crosscorrelation). Useful for velocimetry, /radar ranging. Can also use to correlate a signal with itself (). Especially useful for signals corrupted by noise.

Convolution Fourier Transform and its applications Correlation Convolution Notes

Convolution arises when we try to predict the response of a linear physical system to a given input. Physical systems have a non-ideal response. e.g. capacitance of a detector will cause the input signal to become smeared out in the output If the response of the system is known (or measured), one can deconvolve the output signal using to obtain the true signal. Convolution is also useful for image processing purposes. e.g. finding certain features in the image or filtering.

Convolution Fourier Transform and its applications Correlation Notes

The current I through the resistor is V − V I = in out R while the current I through the is dV I = C out dt

Convolution Fourier Transform and its applications Correlation Notes Both currents are the same, therefore V − V dV in out = C out R dt Therefore we get a differential equation dV V V out + out = in dt RC RC

Multiplying both sides with et/RC we get dV V V out et/RC + out et/RC = in et/RC dt RC RC d V (V et/RC ) = in et/RC dt out RC Convolution Fourier Transform and its applications Correlation Notes Integrating we obtain the analytic solution:

−t/RC Z t  e τ/RC Vout (t) = e Vin(τ)dτ + C1 (1) RC −∞ Z t 1 −(t−τ)/RC = e Vin(τ)dτ (2) RC −∞

where we put the of integration C1 = 0.

Now consider a δ-function input: Vin(t) = δ(t). Performing the integration we get

 0, t < 0 Vout (t) = 1 −t/RC RC e , t = 0

Convolution Fourier Transform and its applications Correlation Succession of δ pulses Notes

Consider a train of δ function inputs. Since the system is linear, Vout is just the sum of the individual pulses. If they are close together they will overlap! The signal gets convolved with the exponential response.

Convolution Fourier Transform and its applications Correlation Notes As the pulse separation becomes smaller and smaller we pass to the continuous case. We can write Z ∞ Vin(t) = Vin(τ)δ(t − τ)dτ −∞ In our example, the response r(τ) to a δ pulse is just an exponential: r(τ) ∝ e−t/RC . The output can be written as Z ∞ Vout (t) = Vin(τ)r(t − τ)dτ (3) −∞ = Vin ⊗ r (4)

which is the convolution of the input signal with the response function of the system. (Compare with analytic solution eqn. (2)).

Convolution Fourier Transform and its applications Correlation Convolution and Fourier transform Notes

Convolution 1 Z ∞ p ⊗ q = √ p(τ)q(t − τ)dτ (5) 2π −∞

What has this to do with Fourier transforms ? Let’s apply Fourier transform to convolution: 1 Z ∞ F[p ⊗ q] = √ [p ⊗ q]e−iωt dt 2π −∞ 1 Z ∞  1 Z ∞  = √ √ p(τ)q(t − τ)dτ e−iωt dt 2π −∞ 2π −∞ 1 Z ∞  1 Z ∞  = √ p(τ) √ q(t − τ)e−iωt dt dτ 2π −∞ 2π −∞ Convolution Fourier Transform and its applications Correlation Fourier Notes

Use shifting property of Fourier transform for the term in square brackets: 1 Z ∞ √ q(t − τ)e−iωt dt = e−iωτ Q(ω), 2π −∞ where Q(ω) = F[q]. Hence,

1 Z ∞ F[p ⊗ q] = √ p(τ)e−iωτ Q(ω)dτ 2π −∞ = P(ω)Q(ω)

Therefore, F[p ⊗ q] = F[p] · F[q] (6) This is the Fourier convolution theorem: Convolution in the is just a in the .

Convolution Fourier Transform and its applications Correlation Fourier convolution Theorem Notes

Typically, this is used to deconvolve a signal. If the system is linear and the response function r to a δ-pulse is known or measured we can use the theorem to deconvolve the output signal Vout :

Vout = Vin ⊗ r

Therefore, F[Vout ] = F[Vin ⊗ r] = F[Vin]F[r] And finally, F[V ] F[V ] = out in F[r] or F[V ] V = F−1 out in F[r]

Convolution Fourier Transform and its applications Correlation Convolution - final remarks Notes

Deconvolution only works for linear systems where superposition holds. Convolution is commutative: p ⊗ q = q ⊗ p Many applications: “cleaning up” a smeared signal by ; finding certain features in an image. In real world applications, signal not only gets smeared out by response function, but also has noise on top of it. Can be addressed using (next week).

Convolution Fourier Transform and its applications Correlation Correlation Notes

Correlation provides a of similarity between two signals. Mathematically it is defined as Correlation 1 Z ∞ p q = √ p∗(τ)q(t + τ)dτ (7) 2π −∞

Note the difference between correlation and convolution: 1 Z ∞ p ⊗ q = √ p(τ)q(t − τ)dτ 2π −∞ Convolution Fourier Transform and its applications Correlation Notes

The correlation is a function of the lag time t. A function correlated with itself is called autocorrelation: 1 Z ∞ p p = √ p∗(τ)p(t + τ)dτ (8) 2π −∞ Unlike convolution, correlation is not commutative: p q 6= q p.

Convolution Fourier Transform and its applications Correlation Correlation of two functions - example Notes

Example: Consider the two functions p(t) and q(t):    0, t < 0  0, t < 0 p(t) = 1, 0 < t < 1 ,and q(t) = 1 − t, 0 < t < 1  0, t > 1  0, t > 1

Convolution Fourier Transform and its applications Correlation Graphical illustration of correlation integral Notes

Convolution Fourier Transform and its applications Correlation Correlation integral Notes Convolution Fourier Transform and its applications Correlation Average correlation function Notes

If functions being correlated are not of finite duration and don’t vanish as t → ±∞, the correlation integral may not exist. In this case can define an average correlation function:

Z T /2 1 ∗ [p q]avg = lim p (τ)q(t + τ)dτ (9) T →∞ T −T /2

If functions p and q are periodic with period T0, set T = T0 in above definition.

Convolution Fourier Transform and its applications Correlation What does it if two functions are uncorrelated? Let’s write Notes p(t) = hpi + ∆p(t) , and q(t) = hqi + ∆q(t),

where we decomposed the functions into their mean and their time-dependent deviations from the mean ∆p(t). Also assume that the functions are real. 1 Z T /2 [p q]avg = lim [hpi + ∆p(τ)][hqi + ∆q(t + τ)]dτ T →∞ T −T /2 1 Z T /2 = hpihqi + hpi lim ∆q(t + τ)dτ T →∞ T −T /2 1 Z T /2 +hqi lim ∆p(τ)dτ T →∞ T −T /2 1 Z T /2 + lim ∆p(τ)∆q(t + τ)dτ T →∞ T −T /2

Convolution Fourier Transform and its applications Correlation Uncorrelated functions Notes

By definition,

1 Z T /2 1 Z T /2 lim ∆q(t + τ)dτ = lim ∆p(τ)dτ = 0, T →∞ T −T /2 T →∞ T −T /2

since the deviations from the mean have to average to zero:

1 Z T /2 1 Z T /2 hpi ≡ lim p(τ)dτ = lim [hpi + ∆p(τ)]dτ T →∞ T −T /2 T →∞ T −T /2 1 Z T /2 = hpi + lim ∆p(τ)dτ T →∞ T −T /2 | {z } =0

Convolution Fourier Transform and its applications Correlation Uncorrelated functions Notes

Since two terms are zero, the correlation function reduces to

1 Z T /2 [p q]avg = hpihqi + lim ∆p(τ)∆q(t + τ)dτ T →∞ T −T /2

If the variations in p are unrelated to the variations in q (e.g. if one of them is noise), the integral on the right hand side will be zero and the functions are considered uncorrelated:

[p q]avg = hpihqi

The correlation integral is constant and reduces to the product of the two mean values. In particular if the mean value of either p or q is zero, the correlation will be also zero. e.g. if one of them is white noise. Convolution Fourier Transform and its applications Correlation Example - Sonar/Radar ranging Notes

By measuring the time delay between the transmission of a signal and the reception of its echo that bounces off an object, one can infer the distance by knowing the speed of the .

Problem: Echo is weak, since intensity falls off as 1/r 4. Moreover, echo corrupted by noise.

Solution: Rather than looking for the echo directly, cross-correlate echo with the original reference signal. Correlation will be large at the lag time that corresponds to the travel time of the signal. Will show that correlation works well even if signal/noise is low.

Convolution Fourier Transform and its applications Correlation Example - Sonar/Radar ranging Notes

Let the signal pulse s(t) be several cycles of a wave:

The echo e(t) has two components: The attenuated original signal (α < 1) delayed in time by ∆ and some noise n(t) e(t) = αs(t − ∆) + n(t)

Convolution Fourier Transform and its applications Correlation Example - Sonar/Radar ranging Notes

In this example, chose α = 0.1 and signal/noise≈ 0.03. Clearly, attenuated echo drowned in noise.

Convolution Fourier Transform and its applications Correlation Example - Sonar/Radar ranging Notes

Cross-correlation given by

s e = s(t) (αs(t − ∆) + n(t)) = αs(t) s(t − ∆) + s(t) n(t)

Since signal and noise are uncorrelated,

s n = hsi hn(t)i = 0 | {z } =0 So, s e = αs(t) s(t − ∆) Therefore, crosscorrelation of signal with echo is just the autocorrelation of the signal multiplied with α and shifted by ∆. We eliminated the noise. Convolution Fourier Transform and its applications Correlation Example - Sonar/Radar ranging Notes

Computed crosscorrelation:

Maximum corresponds to lag=∆, the travel time of the pulse.

Convolution Fourier Transform and its applications Correlation Example - Sonar/Radar ranging Notes

Autocorrelation of the signal:

Convolution Fourier Transform and its applications Correlation Autocorrelation and Power spectrum Notes

1 Z ∞ F[p q] = √ [p q]e−iωt dt 2π −∞ 1 Z ∞ Z ∞  = √ p∗(τ)q(t + τ)dτ e−iωt dt 2π −∞ −∞  

Z ∞ Z ∞  1 ∗  −iωt  = √ p (τ)  q(t + τ)e dt dτ 2π −∞  −∞  | {z } eiωτ F[q] - shift property  1 Z ∞  = F[q] √ p(τ)∗eiωτ dτ 2π −∞

Convolution Fourier Transform and its applications Correlation Autocorrelation and Power spectrum Notes

Finally, we have  ∗  1 Z ∞   −iωτ  F[p q] = F[q] √ p(τ)e dτ (10)  2π −∞  | {z } F[p] = F∗[p]F[q] (11)

In case of the autocorrelation, q = p,

F[p p] = F∗[p]F[p] = |F[p]|2 (12)

This is the Wiener-Khinchin theorem.

The Fourier transform of the autocorrelation is the power spectrum. Convolution Fourier Transform and its applications Correlation Example Notes

White noise is defined as

p p = δ(t) (13)

Only non-zero correlation at lag t = 0. Use Wiener-Khinchin: 1 F[p p] = F[δ(t)] = √ 2π Power spectrum is a constant. All occur equally. The name “White noise” comes from white light, in which all frequencies are present.

Convolution Fourier Transform and its applications Correlation Summary Notes

Both the convolution and correlation integral reduce to simple products of the respective Fourier transform: Convolution theorem:

F[p ⊗ q] = F[p] · F[q]

Wiener Khinchin theorem:

F[p q] = F∗[p]F[q]

In higher dimensions it is computationally more efficient to perform correlations and in Fourier space. This is due to the existence of the (FFT) , which we discuss next time.

Notes

Notes