Convolution Fourier Transform and its applications Correlation Applications of Fourier transform Notes So far, only considered Fourier transform as a way to obtain the frequency spectrum of a function/signal. However, there are other important applications: Convolution: Real physical systems can smear out input signals due to the finite response time of the apparatus. e.g. if you send an impulse (δ-function) in, the response will be different. Fourier analysis can deconvolve the response of the apparatus to recover the true input signal. Correlation: Correlations are used to compare two signals and test if they are correlated or not (crosscorrelation). Useful for velocimetry, sonar/radar ranging. Can also use to correlate a signal with itself (autocorrelation). Especially useful for signals corrupted by noise. Convolution Fourier Transform and its applications Correlation Convolution Notes Convolution arises when we try to predict the response of a linear physical system to a given input. Physical systems have a non-ideal response. e.g. capacitance of a detector will cause the input signal to become smeared out in the output If the response of the system is known (or measured), one can deconvolve the output signal using Fourier analysis to obtain the true signal. Convolution is also useful for image processing purposes. e.g. finding certain features in the image or filtering. Convolution Fourier Transform and its applications Correlation Notes The current I through the resistor is V − V I = in out R while the current I through the capacitor is dV I = C out dt Convolution Fourier Transform and its applications Correlation Notes Both currents are the same, therefore V − V dV in out = C out R dt Therefore we get a differential equation dV V V out + out = in dt RC RC Multiplying both sides with et=RC we get dV V V out et=RC + out et=RC = in et=RC dt RC RC d V (V et=RC ) = in et=RC dt out RC Convolution Fourier Transform and its applications Correlation Notes Integrating we obtain the analytic solution: −t=RC Z t e τ=RC Vout (t) = e Vin(τ)dτ + C1 (1) RC −∞ Z t 1 −(t−τ)=RC = e Vin(τ)dτ (2) RC −∞ where we put the constant of integration C1 = 0. Now consider a δ-function input: Vin(t) = δ(t). Performing the integration we get 0; t < 0 Vout (t) = 1 −t=RC RC e ; t = 0 Convolution Fourier Transform and its applications Correlation Succession of δ pulses Notes Consider a train of δ function inputs. Since the system is linear, Vout is just the sum of the individual pulses. If they are close together they will overlap! The signal gets convolved with the exponential response. Convolution Fourier Transform and its applications Correlation Notes As the pulse separation becomes smaller and smaller we pass to the continuous case. We can write Z 1 Vin(t) = Vin(τ)δ(t − τ)dτ −∞ In our example, the response r(τ) to a δ pulse is just an exponential: r(τ) / e−t=RC . The output can be written as Z 1 Vout (t) = Vin(τ)r(t − τ)dτ (3) −∞ = Vin ⊗ r (4) which is the convolution of the input signal with the response function of the system. (Compare with analytic solution eqn. (2)). Convolution Fourier Transform and its applications Correlation Convolution and Fourier transform Notes Convolution 1 Z 1 p ⊗ q = p p(τ)q(t − τ)dτ (5) 2π −∞ What has this to do with Fourier transforms ? Let's apply Fourier transform to convolution: 1 Z 1 F[p ⊗ q] = p [p ⊗ q]e−i!t dt 2π −∞ 1 Z 1 1 Z 1 = p p p(τ)q(t − τ)dτ e−i!t dt 2π −∞ 2π −∞ 1 Z 1 1 Z 1 = p p(τ) p q(t − τ)e−i!t dt dτ 2π −∞ 2π −∞ Convolution Fourier Transform and its applications Correlation Fourier convolution Theorem Notes Use shifting property of Fourier transform for the term in square brackets: 1 Z 1 p q(t − τ)e−i!t dt = e−i!τ Q(!); 2π −∞ where Q(!) = F[q]. Hence, 1 Z 1 F[p ⊗ q] = p p(τ)e−i!τ Q(!)dτ 2π −∞ = P(!)Q(!) Therefore, F[p ⊗ q] = F[p] · F[q] (6) This is the Fourier convolution theorem: Convolution integral in the time domain is just a product in the frequency domain. Convolution Fourier Transform and its applications Correlation Fourier convolution Theorem Notes Typically, this is used to deconvolve a signal. If the system is linear and the response function r to a δ-pulse is known or measured we can use the theorem to deconvolve the output signal Vout : Vout = Vin ⊗ r Therefore, F[Vout ] = F[Vin ⊗ r] = F[Vin]F[r] And finally, F[V ] F[V ] = out in F[r] or F[V ] V = F−1 out in F[r] Convolution Fourier Transform and its applications Correlation Convolution - final remarks Notes Deconvolution only works for linear systems where superposition holds. Convolution is commutative: p ⊗ q = q ⊗ p Many applications: \cleaning up" a smeared signal by deconvolution; finding certain features in an image. In real world applications, signal not only gets smeared out by response function, but also has noise on top of it. Can be addressed using Wiener deconvolution (next week). Convolution Fourier Transform and its applications Correlation Correlation Notes Correlation provides a measure of similarity between two signals. Mathematically it is defined as Correlation 1 Z 1 p q = p p∗(τ)q(t + τ)dτ (7) 2π −∞ Note the difference between correlation and convolution: 1 Z 1 p ⊗ q = p p(τ)q(t − τ)dτ 2π −∞ Convolution Fourier Transform and its applications Correlation Notes The correlation is a function of the lag time t. A function correlated with itself is called autocorrelation: 1 Z 1 p p = p p∗(τ)p(t + τ)dτ (8) 2π −∞ Unlike convolution, correlation is not commutative: p q 6= q p. Convolution Fourier Transform and its applications Correlation Correlation of two functions - example Notes Example: Consider the two functions p(t) and q(t): 8 8 < 0; t < 0 < 0; t < 0 p(t) = 1; 0 < t < 1 ,and q(t) = 1 − t; 0 < t < 1 : 0; t > 1 : 0; t > 1 Convolution Fourier Transform and its applications Correlation Graphical illustration of correlation integral Notes Convolution Fourier Transform and its applications Correlation Correlation integral Notes Convolution Fourier Transform and its applications Correlation Average correlation function Notes If functions being correlated are not of finite duration and don't vanish as t ! ±∞, the correlation integral may not exist. In this case can define an average correlation function: Z T =2 1 ∗ [p q]avg = lim p (τ)q(t + τ)dτ (9) T !1 T −T =2 If functions p and q are periodic with period T0, set T = T0 in above definition. Convolution Fourier Transform and its applications Correlation What does it mean if two functions are uncorrelated? Let's write Notes p(t) = hpi + ∆p(t) ; and q(t) = hqi + ∆q(t); where we decomposed the functions into their mean and their time-dependent deviations from the mean ∆p(t). Also assume that the functions are real. 1 Z T =2 [p q]avg = lim [hpi + ∆p(τ)][hqi + ∆q(t + τ)]dτ T !1 T −T =2 1 Z T =2 = hpihqi + hpi lim ∆q(t + τ)dτ T !1 T −T =2 1 Z T =2 +hqi lim ∆p(τ)dτ T !1 T −T =2 1 Z T =2 + lim ∆p(τ)∆q(t + τ)dτ T !1 T −T =2 Convolution Fourier Transform and its applications Correlation Uncorrelated functions Notes By definition, 1 Z T =2 1 Z T =2 lim ∆q(t + τ)dτ = lim ∆p(τ)dτ = 0; T !1 T −T =2 T !1 T −T =2 since the deviations from the mean have to average to zero: 1 Z T =2 1 Z T =2 hpi ≡ lim p(τ)dτ = lim [hpi + ∆p(τ)]dτ T !1 T −T =2 T !1 T −T =2 1 Z T =2 = hpi + lim ∆p(τ)dτ T !1 T −T =2 | {z } =0 Convolution Fourier Transform and its applications Correlation Uncorrelated functions Notes Since two terms are zero, the correlation function reduces to 1 Z T =2 [p q]avg = hpihqi + lim ∆p(τ)∆q(t + τ)dτ T !1 T −T =2 If the variations in p are unrelated to the variations in q (e.g. if one of them is noise), the integral on the right hand side will be zero and the functions are considered uncorrelated: [p q]avg = hpihqi The correlation integral is constant and reduces to the product of the two mean values. In particular if the mean value of either p or q is zero, the correlation will be also zero. e.g. if one of them is white noise. Convolution Fourier Transform and its applications Correlation Example - Sonar/Radar ranging Notes By measuring the time delay between the transmission of a signal and the reception of its echo that bounces off an object, one can infer the distance by knowing the speed of the wave. Problem: Echo is weak, since intensity falls off as 1=r 4. Moreover, echo corrupted by noise. Solution: Rather than looking for the echo directly, cross-correlate echo with the original reference signal. Correlation will be large at the lag time that corresponds to the travel time of the signal. Will show that correlation works well even if signal/noise is low. Convolution Fourier Transform and its applications Correlation Example - Sonar/Radar ranging Notes Let the signal pulse s(t) be several cycles of a sine wave: The echo e(t) has two components: The attenuated original signal (α < 1) delayed in time by ∆ and some noise n(t) e(t) = αs(t − ∆) + n(t) Convolution Fourier Transform and its applications Correlation Example - Sonar/Radar ranging Notes In this example, chose α = 0:1 and signal/noise≈ 0:03.