Lab 7: Filters and Convolution

Lab 7: Filters and convolution

One of the most remarkable properties of a linear dynamical system is its response to sinusoidal inputs. As we learned in lecture, a sinusoidal input to a linear dynamical system produces a sinusoidal output having the same frequency as the input. However, the output sinusoid can differ in two ways from the input sinusoid: 1) its amplitude, and 2) its phase. Measurements of the phase and amplitude of the output to a sinusoidal input can be used to identify the differential equation which best captures the input-output re- lationships of the dynamical system. The graphical technique for estimating the transfer function from these experimental observations is the Bode plot. Once the transfer function has been determined the response of the dynamical system to any realizable input can be predicted. The purpose of this laboratory is to use computer experiments to verify these observations and to introduce the concepts of a filter and convolution.

1 Background

The lecture brieﬂy reviews the concept of an impulse function, ﬁltering and convolution. A practical, but important, detail concerns the representation of a sine wave wave in a computer program. We can write a simple program that plots a 10 second interval of a sine wave for a given frequency, f, and a given time step. import numpy as np import matplotlib.pyplot as plt f=0.1 N=11 NN=101

1 start=0.0 stop=100.0 t=np.linspace(start,stop,num=N,endpoint=True) tt=np.linspace(start,stop,num=NN,endpoint=True) plt.plot(t,np.sin(2*np.pi*f*t),’k-o’) plt.plot(tt,np.sin(2*np.pi*f*tt),’r-’) plt.axis([0,stop,-1.1,1.1]) plt.show()

1 The period, T , is f , so when f = 0.1, we have T = 10. If we choose start and stop to be, respectively, 0 and 100, then this would draw ten cycles of a 0.1Hz provided that N is chosen appropriately (red line in ﬁgure produced by this program). The positive integer, N, is the number of equally spaced time samples between, in this case, 0 and 100. It is important to remember that by choosing the endpoint=True option for np.linspace() both the starting and the end point are included, N points altogether1. The time step, dt is stop-start dt = times per period N What is a good choice of the number of time steps, N? If we choose N = 10, then dt = 1 per period and we would obtain 10 identical xalues of x since we are sampled once per period and the rtime series is 10 periods long (compare black and red lines). This is not a good choice of N since we cannot uniquely represent a function using a single value. Next we could try N = 20 which would give us two point per period. In principle we can uniquely represent a sine wave if we sample at twice the frequency (this is the concept of the Nyquist frequency that we will discuss in the next lab). However, x would not look like a nice sine wave. So we must pick N larger: the larger N, the smaller dt, and hence the more frequently the sine wave is sampled. However, it is also true that the larger N the more computer memory it takes to store in this case x and t, and the greater the number of calculations that the computer must make. Thus there is a practical trade– oﬀ between the accuracy by which functions are to be represented and the time it takes the computer to complete its calculations. The bottom line is that as we change f we need to adjust dt. A reasonable choice might be to sample the sine wave 10 − 100 times per period. Ideally we would like to keep the number of sample points per period the same as we change f.

1Note that to get say 10 evenly spaced samples you need to set num= N + 1.

2 Browser use: Have available the document intro_python.pdf on your browser for review, in particular, the sections that deal with how to de- clare a function. As always, information concerning the use of Python and matplotlib functions can be obtained from the Internet using a browser. Housekeeping: Labs 8 and 9 use some of the programs and results that we develop in this lab. Thus it is useful to put today’s material in a sub- directory of ~\pyprogs called, for example, convol, namely ~\pyprogs\convol. This will make it easier for you to ﬁnd these results when we need them. Exercise 1: Bode plots: low–pass ﬁlter

1. Write a computer program to integrate dx = k x + sin 2πft , (1) dt 1

where k1 is a constant.

2. Take k1 = −1. 3. Vary frequency over a large range (say f = 0.01 to f = 100) and measure the amplitude. Plot the amplitude versus frequency.

4. Vary the frequency and measure the shift between the sine wave and the solution of the equation. For example, we can plot the input and output sine wave n the same ﬁgure. Plot the phase shift versus the frequency.

5. Now change a to, let’s say, k1 = 10 and then k1 = 0.1. What happens? 6. Construct the Bode plot by choosing a suitable range of f. For each choice of f enter the amplitude and phase shift of the steady–state solution and write these values in a *.tsv ﬁle. Use this *.tsv ﬁle to construct the phase part of the Bode plot.

7. You have just constructed your ﬁrst low pass ﬁlter. Answer the following:

• The frequency on the input sine wave and the output sine wave is the same. True of false?

3 • The output amplitude changes as a function of frequency. True or false? • The output sine wave is phase shifted compared to the input sine wave. True or false? • Why is this type of ﬁlter called a low pass ﬁlter? 8. What is the transfer function for (1)? 9. Show that the impulse function, I(t, t0), is

0 I(t, t ) = k1 exp(k1t) . (Remember that the impulse function corresponds to the case when the input is a delta function.) Is this the same as the answer we derived in class (Section 7.3)? 10. The transfer function, C(s), for the thermometer is of the form 1 C(s) = . 1 + αs Sketch the Bode plot. What is the corner frequency? 11. Consider the transfer function 10 C(s) = . s(1 + 0.5s)(1 + 0.1s) Sketch the Bode plot for the amplitude showing the contribution of each term.

Exercise 2: The convolution integral: The concept of convolution is one of the most important concepts in science: every time one uses a device to make a measurement, one essentially performs a convolution. All physical observations are limited by the resolving power of instruments, and for this reason alone convolution is ubiquitous. This principle does not only apply to laboratory instruments: our senses, e.g. vision, touch, smell, audition, proprioception, are also convoluted by the sensory receptors which translate physical stimuli into neural spikes trains. This convolution is mathematically as the operation Z ∞ y(t) = x(t)h(t − u)du , (2) −∞

4 where x(t) describes the input to the dynamical system and h(t − u) describes the impulse function. In practice it is very easy to perform a convolution numerically. In Python this is accomplished using the single command np.convolve() (similar commands exist, for example, in MATLAB). In order to use the convolution integral to solve (1) we need to know that the impulse function. Convolution: Graphical interpretation Although the mechanics of evaluating the convolution integral seem straight- forward, it is quite diﬃcult to understand in a simple way the process of evaluating the convolution integral. The purpose of this exercise is to demonstrate a graphical interpretation of convolution [1]. This graphical method is very useful and, for example, we can use it to understand issues related to zero- padding and to appreciate the diﬀerence between convolution and correlation (Lab 14).

Figure 1: Graphical method for convolving a delta–function input (vertical dashed line in ’Multiplication panel’) with an alpha–type impulse response (solid line). See text for discussion.

The graphical convolution method divides the process of convolution into

5 four steps [1]:

1. Folding: Reﬂect the impulse response, I(t), about the ordinate axis to obtain I(−t).

2. Displacement Shift I(−t) by an amount ∆t to obtain I(−t + ∆t).

3. Multiplication: Multiply I(−t + ∆t) by the input, b(t).

4. Integration: Determine the area under the curve produced by I(−t + ∆t) and b(t).

Figure 1 illustrates this procedure in the evaluation of the alpha function response, I(t), of a neuron to a single delta-function input, b(t). The step ‘Folding’ means that we reflect the impulse response about the y-axis. The step ‘Displacement’ means that we slowly shift the reflected impulse in the positive direction along the x-axis. The step ‘Multiplication’ means that we multiple the reflected impulse with the input. It is easiest to understand this step iteratively: for each time step do the multiplication (and then Step 4), then displace the reflected impulse response another time step to the right and repeat the process. In general this corresponds to the shaded region under the two curves for which both functions are non-zero. Finally the step ‘Integration’ calculates the area under the the two curves. In this example the area is easy to determine since by the shifting property of the delta- function (Section 6.3), the area is just the value of the alpha function at that point. The result of convolution is the area computed in Step 4 as a function of time, namely as the reflected impulse response is shifted iteratively to the right. Questions to be answered:

1. Use the graphical technique discussed in Section 1 to compute the convolution for the following choices of b(t) and I(t)

• b(t) = constant and I(t) = e−t • b(t) is a single square wave pulse and I(t) = e−t (This is animated on the Wikipedia site for Convolution.) • Both b(t) and I(t) are square wave pulses (This is animated on the Wikipedia site for Convolution.)

6 Exercise 3: Computing the convolution Numpy has a function called np.convolve(). Although it is tempting to think that it would be easy to calculate the convolution of any function; there are some subtleties. The function np.convolve(r,s,mode=’full’) calculates the convolution integral, denoted (r ∗ s), as

N/2 X (r ∗ s)j = sj−krk k=−N/2+1 where s is the discretely sampled signal and r is the discretely sampled impulse function [2]. Unfortunately this approximation assumes that the length of r is the same as that of s and that s is periodic. Thus we need to figure out how to get around these constraints. The solution involves padding the data and impulse response with zeros. This procedure is called zero padding. The first constraint is easily dealt with since it is typically the case that the length of impuse response, M, is much shorter than the length of the signal, N. Thus we need to pad the impulse function with zero is make M = N. The second problem is more subtle. Not only must the signal be considered to be periodic, but also the impulse function. Therefore it happens that a portion at each end of the original signal is erroneously wrapped around by the convolution with the response function. In other words, the first output channel (s∗r)0 is contaminated wuth some wrapped-around data from the far end of the data stream sN−1, sn−2, ···. This we need to set up a buffer zone of zero-padded values at the end of s in order to make the effects of this contamination equal to zero. How many zeros do we need top add? We need to pad the signal function on one end with the number of zeros equal to the length of the response function. The numpy function, np.pad(), is very useful for handling issues related to zero padding np.pad(array,pad_width,mode=None).

A useful feature of np,pad() is that we can pad to the left and/or to the right of the array. For example, if x=[1,2,3] then the command

7 X=np.pad(x,(0,3),’constant’) yields

X=[1,2,3,0,0,0] and the command

X=np.pad(x,(3,0),’constant’) yields the array

X=[0,0,0,1,2,3] However, we are still not done. In terms of our fold-displacement-multiplication- integration analogy for convolution, convolve() does not include the integration step. Thus we need to multiple the output of np.convolve() by the time step. Questions to be answered: 1. Write a computer program that uses np.convolve() to construct the Bode plot for (1). Again for each choice of F , measure the amplitude and phase shift using the steady state solution.

Exercise 4: Neural inputs: The program convolve_neural_spikes_alpha.py uses the convolution integral to estimate the sub-threshold membrane potential when the impulse function is given by the alpha function

( 0 ±k2 (t − t0)e−km(t−t ) if t ≥ t0 , I (t, t0) = m (3) α 0 t < t0 .

The input spike train is modeled as a sequence of delta functions. Figure 2 shows how the membrane potential changes for a given periodic excitatory input. Hints:

1. A common use of def is to deﬁne and introduce functions that are used for calculations in the program. The command r=impulse_response(t) evaluates the function for all of the values of t. Note that the commands in the def are indented by usually 4 spaces

8 Figure 2: The membrane potential of a neuron (bottom panel) in response to a pulsatile, excitatory input whose timing corresponds to the timing of spikes in the spike train shown in the top panel. The changes in the neuron’s membrane potential following the arrival of each EPSP is described by (3) with km = 0.5.

2. Useful array manipulations:

[1:5] means from 1 to 5 [1:] means 1 to end [len(a):1] means from length of a to end [::n] means each n-th item in entire sequence

3. The function plt.subplot() has the form

\verb+plt.subplot(row,column,panel number)

Which panel would be drawn using the command plt.subplot(223)?

4. The package matplotlib.ticker includes a number of useful functions that make it possible to, for example, change the spacing of the number on the axis, etc. Labeling the subplots ax, bx, .. is necessary in order

9 to use the functions within matplotlib.ticker. In the case of a ﬁgure that has only one panel, we can use the command plt.subplot(111).

Question to be answered: An impulse function can be used to model the changes in blood concentra- tion of a drug ingested orally. For example, if we take (3) as a simple impulse function for drug ingestion, then the sharp rising portion indicates the time course of the drug going from the stomach to the blood (more typically min- utes), and the falling phase describes the clearance of drug from the body by the actions of the liver and kidneys (typically hours to days). Consider that a doctor prescribes a drug for which the e−1 time is ∼ 6 hours and asks that the patient take the medication 4 times a day. Most patients do not take the medication every 6 hours, but more often at 8am–noon–5pm–midnight. Write a computer program to look at the effects of different dosing schedules on blood levels to answer the following questions. • Would it be better for the patient to take the medication three times a day, i.e. every 8 hours? • If the e−1 time is ∼ 4 hours, what would be the difference between the two dosing schedules? • If the e−1 time is ∼ 12 hours, what would be the difference between the two dosing schedules?

Low-pass filters with two-poles Here we discuss the transfer function for a second-order linear ordinary differential equation. As we discussed in lecture it is traditional and conve- nient to write the differential equation in the form 1 d2x 2 dx 2 2 + + x = b(t) , (4) ωn dt ωn dt where ωn = 2πfn is the natural frequency of the associated harmonic oscil- lator (Note that ωn = 2πfn). The damping ratio is : > 1 : Overdamped , = 1 : Critically damped , 1 > > 0 : Underdamped , = 0 : Undamped .

10 For the homogeneous ordinary differential equation d2x dx + 2ω + ω2x = 0 , (5) dt2 n dt n the two eigenvalues are q 2 2 2ωn ± (2ωn) − 4ωn λ1,2 = , √2 2 = −ω2 ± ωn − 1 . When we determine the impulse function we need to remember that the λ1,2 are either be real negative numbers (the over-damped case) or a pair of complex numbers with negative real parts (the under-damped case). This is because we can only define the impulse function for a dynamical system which in the absence of an input has a stable fixed–point. For the over–damped case we have ωn h i I(t, t0) = √ eλ1t − eλ2t . (6) 2 2 − 1 For the under-damped case we have σ2 + ω2 I(t, t0) = e−σt sin(ωt) , (7) ω where σ is the absolute value of the real part of the complex eigenvalue and ω is the imaginary part. Questions to be answered: Consider the second–order linear differential equation d2x dx + k + k x = sin 2πft , (8) dt2 1 dt 2 where k1, k2 are constants to be determined.

1. Choose the values of k1, k2 such that both eigenvalues are real and negative. In particular, pick the parameters so that one eigenvalues, λ1, is equal to −1. Construct the Bode plot when λ2 is smaller than λ1 and another when λ2 is larger. Compare these Bode plots to the one we obtained for (1). You should be able to sketch these Bode plots without using computer simulations (Of course you are welcome to use a computer program to conﬁrm the validity of your sketches.) What are the diﬀerences?

11 2. Pick values of k1, k2 such that the ﬁxed–point of the homogeneous equation (i.e., the right–hand side is 0) is a stable spiral point.

• For these choice of parameters, what is the natural frequency?

• Write a computer program to show that as k1 approaches zero, (8) exhibits resonance as f is closed to the natural frequency, fn. Your ﬁgure should show the ampliﬁed solution (x versus t). • Sketch the Bode plot for this case.

Deliverables: Assignment should be submitted as a *.pdf prepared using LaTeX. An- swer the questions. In those cases in which the ﬁgures are sketched, attach the sketches to the LaTeX document.

References

[1] E. O. Brigham. The fast Fourier transform and its applications. Prentice Hall, Englewood, N. J., 1988.

[2] W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery. Nu- merical recipes: The art of scientiﬁc computing, third edition. Cambridge University Press, New York, 2007.