AN-237 Convolution: Digital Signal Processing

Literature Number: SNAA080 OVLTO:DgtlSga rcsigAN-237 Processing Signal Digital CONVOLUTION:

National Semiconductor CONVOLUTION: Digital Application Note 237 Signal Processing January 1980

Introduction As digital signal processing continues to emerge as a major Decreasing the pulse width while increasing the pulse discipline in the field of electrical engineering, an even height to allow the area under the pulse to remain constant, greater demand has evolved to understand the basic theo- Figure 1c, shows from eq(1) and eq(2) the bandwidth or retical concepts involved in the development of varied and spectral-frequency content of the pulse to have increased, diverse signal processing systems. The most fundamental Figure 1d. concepts employed are (not necessarily listed in the order Further altering the pulse to that of Figure 1e provides for an [ ] of importance) the sampling theorem 1 , Fourier transforms even broader bandwidth, Figure 1f. If the pulse is finally al- [ ][] 2 3, convolution, covariance, etc. tered to the limit, i.e., the pulsewidth being infinitely narrow The intent of this article will be to address the concept of and its amplitude adjusted to still maintain an area of unity convolution and to present it in an introductory manner under the pulse, it is found in 1g and 1h the unit impulse hopefully easily understood by those entering the field of produces a constant, or ‘‘flat’’ spectrum equal to 1 at all digital signal processing. frequencies. Note that if ATe1 (unit area), we get, by defini- It may be appropriate to note that this article is Part II (Part I tion, the unit impulse function in time. is titled ‘‘An Introduction to the Sampling Theorem’’) of a Since this time function contains equal frequency compo- series of articles to be written that deal with the fundamental nents at all frequencies, applying it or a good approximation concepts of digital signal processing. of it to the input of a linear network would be the equivalent Let us proceed.... of simultaneously impressing upon the system an array of oscillators inclusive of all possible frequencies, all of equal Part II Convolution amplitude and phase. The frequencies could thus be deter- Perhaps the easiest way to understand the concept of con- mined from this one input time function. Again, variations in volution would be an approach that initially clarifies a sub- amplitude and phase at the system output would be due to ject relating to the frequency spectrum of linear networks. the system itself. Determining the frequency spectrum or frequency transfer Empirically speaking the frequency spectrum or the network function of a linear network provides one with the knowl- frequency transfer function can thus be determined by ap- edge of how a network will respond to or alter an input plying an impulse at the input and using, for example, a signal. Conventional methods used to determine this entail spectrum analyzer at the network output. At this point, it is the use of spectrum analyzers which use either sweep gen- important to emphasize that the above discussion holds erators or variable-frequency oscillators to impress upon a true for only linear networks or systems since the superposi- network all possible frequencies of equal amplitude and tion principle (The response to a sum of excitations is equal equal phase. to the sum of the responses to the excitations acting sepa- The response of a network to all frequencies can thus be rately), and its analytical techniques break down in non-lin- determined. Any amplitude and phase variations at the out- ear networks. put of a network are due to the network itself and as a result Since an impulse response provides information of a net- define the frequency transfer function. work frequency spectrum or transfer function, it additionally Another means of obtaining this same information would be provides a means of determining the network response to to apply an impulse function to the input of a network and any other time function input. This will become evident in then analyze the network impulse-response for its spectual- the following development. frequency content. Comparison of the network-frequency If the input to a network, Figure 2, having a transfer function transfer function obtained by the two techniques would yield H(0) is an impulse function e(t) at te0, its Fourier transform the same information. using eq(1) can be found to be F(0)e1. This is found to be easily understood (without elaborate ex- The output of the network G(0) is therefore perimentation) if the implications of the impulse function are initially clarified. G(0) e H(0) # F(0) If the pulse of Figure 1a is examined, using the Fourier inte- G(0) e H(0) gral, its frequency spectrum is found to be The inverse transform is % F(0) e # f(t) f bj0t dt (1) g(t) e h(t) b% and h(T) is defined as the impulse response of the network T/2 as a result of being excited by a unit impulse time function at e A f bj0t dt (2) te0. #bT/2 Extending this train of thought further, the response of a 0T network to any input excitation can be determined using the sin same technique. # 2 J F(0) e AT 0T % # 2 J –

as shown in Figure 1b.

C1995 National Semiconductor Corporation TL/H/5621 RRD-B30M115/Printed in U. S. A. TL/H/5621–1 FIGURE 1. Development of a unit impulse; (a) (c) (e) (g) its time function (b) (d) (f) (h) its frequency spectrum

Hence, finding the Fourier transform of the input excitation, As a proof using eq(1) let F(0), multiplying it by the transfer function transform H(0) (or the transform of time domain network impulse response) % t F f(t) * h(t) e f bj0t f(u) h(t b u)du dt(5) and inverse transforming to find the output g(t) as a function Ð ( #0 Ð #0 ( of time. Defined by the shifted step function By definition the convolution integral1 u(t b u) e 1 for u s t (6a) t f(t) * h(t) e f(u) h(t b u)du (3) and #0 u(t b u) e 0 for u l t (6b) (where * denotes the convolution operation, h(t) denotes the impulse response function described above and both f(t) Footnote: and h(t) are zero for tk0. Note that the meaning of the 1. It is important to note that the convolution integral is com- variables t and u will be clarified, later in the article) makes mutative. This implies the reversability of the f(t) and h(t) the same claim but in the realm of the time domain alone. terms in the definition. If this is true then the Fourier transform of the convolution % integral eq(3) should have the following equivalence b g(t) e f(u) h(t b u)dueFb1[F(0) # H(0)] #b% t % F Ð # f(u) h(t b u)du (eF(0) # H(0) (4) f(t b u)h(u)dueFb1[H(0) # F(0)] 0 #b%

2 put excitation function, multiplying the two transforms and finally computing the inverse FFT of the product. Moving averages and smoothing operations can further be characterized as lowpass filtering functions and can addi- TL/H/5621–2 tionally be implemented using convolution. The above are H(0)eH (0) # H (0) a b just a few of the many operations convolution performs and G(0) e H(0) the remainder of this discussion will focus on how convolu- F(0) tion is realized. FIGURE 2. Block diagram of a network transfer function To start with, an illustrative analysis will be performed as- suming continuous functions followed by one performed in the following identity can be made discrete form similar to that realized in computer aided sam- t % pled-data systems techniques. f(u) h(t b u)due f(u) h(t b u) u(t b u)du (7) #0 #0 As an example, if it were desired to determine the response of a network to the excitation pulse f(t) shown in Figure 3a, Rewriting eq(5) as knowing the network impulse he(t), Figure 3b, the impulse % % response of an RC network, would allow one to determine F[f(t) * h(t)] e fbj0t f(u) h(t b u) u(t b u)dudt (8) the output g(t) using the convolution integral, eq(3). #0 #0 The convolution of f(t) and he(t) and letting x e t b u so that f(t)e10[u(t)b(u(tbTo)] (13) fbj0t fbj0(x a u) (9) e he(t)efbat eq(8) finally becomes could be obtained by first substituting the dummy variable t u fortinh (t) so that % % b e F[f(t) * h(t)] e # # f(u) h(x) u(x) fbj0tfbj0x dudx 0 0 he(tbu)efba(tbu) (15) % % j0x j0u e h(x) u(x) *fb dx f(u) fb du By definition g(t)ef(t) * he(t) thus becomes #0 #0 t t u F[f(t)*h(t)]eH(0) # F(0) (10) f(u)he(tbu)due 10[u(t)bu(tbTo)]fba(tb ) du (16) #0 #0 which is the equivalent of eq(4). In essence the above proof describes one of the most important and powerful tools used in signal processing .... the convolution theorm. In words, Convolution Theorem: The convolution theorem allows one to mathemati- cally convolve in the time domain by simply multiplying in the frequency domain. That is, if f(t) has the Fourier transform F(0) and x(t) has the Fourier TL/H/5621–3 transform X(0), then the convolution f(t) * x(t) has the Fourier transform F(0) # X(0). f(t)e10[u(t)bu(t bTo)] (a) For the time convolution f(t) * x(t) wx F(0) # X(0) (11) and the dual frequency convolution is f(t) # x(t) wx F(0) * X(0) (12) Convolutions are fundamental to time series sampled data analysis. First of all, as described earlier all linear networks can be completely characterized by their impulse response functions and furthermore the response to any input is given by its (the input function) convolution with the network impulse response function. Digit filters being linear systems TL/H/5621–4 accomplish the filtering task using convolutions. A network or filter transfer function for example can be represented by he(t)efbat its inpulse response in the form of a Fourier series. A filtered (b) input excitation response can then be found by convolving FIGURE 3. (a) rectangular pulse excitation the input time function with the network Fourier series or (b) impulse response of a single RC network impulse response. With the aid of a high speed computer the same result could be obtained by storing the FFT (Fast Fourier Transform) of the network impulse response into memory, performing an FFT on the sampled continuous in-

3 TL/H/5621–5 (c)

FIGURE 3. (c) output or convolution of the network (b) excited by (a)

Since the piecewise nature of the excitation makes it conve- defines the graphical procedure. Using the same example nient to calculate the response in corresponding pieces the depicted in Figure 3 the excitation and impulse response output is found to be functions replaced with the dummy variable is defined as past data or historical information to be used in a convolution process. Thus 0 k t s To t f(u)e10[u(u)bu(ubTo)] (19) g(t) e f(t) he(t) e 10fba(t b u) du #0 and e 10(1 b fbat) (17) he(u)efbau (20) t t To are shown in Figure 4a and b. Figure 4c, he(bu), represents To the impulse response folded over [mirror image of he(u)] g(t) e f(t)*he(t) e 10fba(t b u) du #0 about the ordinate and Figure 4d,he(tbu), is simply the function he(bu) time shifted by the quantity t. 10fbat (fau 1) (18) e o b Evaluation of the convolution integral is performed by multiplying f(u) by each incremental shift in he(tbu). It is under- The output response g(t) is plotted in Figure 3c and is clear- stood in Figure 4e that a negative value of bt produces no ly what might be expected from a simple RC network excit- output. For t l O however as the present time t varies, the ed by a rectangular pulse. impulse response he(tbu) scans the excitation function f(u), Though simplistic in its nature, the analysis of the above always producing a weighted sum of past inputs and weigh- u example quickly becomes unrealistically cumbersome when ing most heavily those values of f( ) closet to the present. complex excitation and impulse response functions are As seen in Figures 4e through 4n, the response or output of used. Turning to a numerical evaluation of the convolution the network at anytime t is the integral of the functions or integral may perhaps be the most desirable method of real- calculated shaded area under the curves. In terms of the ization. Prior to a numerical development however, an intui- superposition principle the filter response g(t) may be inter- u tive graphical illustration of convolution will be presented preted as being the weighted superposition of past input f( ) u which should make discrete numeric convolution easily un- values weighted or multiplied by he(tb ). derstood. An extension of the continuous convolution to its numerical The convolution integral Ð discrete form is made and shown in Figure 5. Again the excitation and impulse response of Figure 3 are used and t are further represented as two finite duration sequences f(n) f(u)he(t b u)du #0 IMPULSE RESPONSE EXCITATION

TL/H/5621–6 FIGURE 4. a) he(u): network impulse response b) f(u): excitation function

4 TL/H/5621–7 FIGURE 4. cont’d c) he(bu): he(u) folded about the ordinate d) he(tbu): he(r) folded and shifted e) through n) the output response g(t) of the network whose impulse response he(u) is excited by a function f(u). Or the convolution, f(u)*he(t), of f(t) with he(t).

5 IMPULSE RESPONSE EXCITATION

TL/H/5621–8 FIGURE 5. Illustrative description of discrete convolution and he(n) respectively, Figures 5a and b. If f(n) and he(n) were next considered to be periodic sequences and a convolution was desired using either shifting It is observed additionally that the duration of f(n) is Nae7 techniques or performing an FFT on the excitation and im- samples [f(n) is nonzero for the interval 0 s n s Na b1 and pulse response sequences and finally inverse FFT trans- the duration of he(n) is Nbe8 samples [he(n) is nonzero for forming to achieve the output response, some care must be the interval 0 s n s Nb b1]. The sequence g(n), a discrete convolution, can thus be defined as taken when preparing the convolving sequences. From Fig- n ure 5h it is observed that the convolution is completed in a N N 1 point sequence. To acquire the nonoverlapping g(n) f(x) h (n x) (21) aa bb e & e b or nondistorted periodic sequence of Figure 6c the convolu- x e 0 tion thus requires f(n) and he(n) to be NaaNbb1 point se- having a finite duration sequence of NaaNbb1 samples, quences. This is achieved by appending the appropriate Figure 5h. The convolution using numerical integration (area number of zero valued samples, also known as zero filling, under the curve) can be defined as to f(n) and he(n) to make them both NaaNbb1 point se- n quences. The undistorted and correct convolution can now g(n)T e T f(x) he(n b x) (22) be performed using the zero filled sequences Figure 6a and & 6b to achieve 6c. x e 0 where T is the sampling interval used to obtain the sampled data sequences.

6 TL/H/5621–9 FIGURE 6. Linear periodic discrete convolution of f(n) and he(n), f(n)*he(n). A Final Note This article attempted to simplify the not-so-obvious con- Finally, two examples of discrete convolution were present- cept of convolution by first developing the readers knowl- ed. The first example dealt with finite duration sequences edge and feel for the implications of the impulse function and the second dealt with periodic sequences. Additionally, and its effect upon linear networks. This was followed by a precautions in the selection on n-point sequences was dis- short discussion of network transfer functions and their rela- cussed in the second example to alleviate distorting or tive spectrum. Having set the stage, the convolution integral spectually overlapping the excitation and impulse response and therorem were introduced and supported with an ana- functions during the convolution process. lytical and illustrative example. This example showed how the response of a simple RC network excited by a rectangular pulse could be determined using the convolution integral.

7 AN-237 CONVOLUTION: Digital Signal Processing .R .Hamming, W. R. 7. .L ncsn .K Otnes, K. R. Enochson, L. 4. .M Schwartz, M. 9. .B od .R Rabiner, R. L. Gold, B. 8. .F .Kuo, F. F. Rader, 6. M. C. Gold, B. 5. Brigham, O. E. 3. Papoulis, A. 2. 1 .Kreyszig, E. 11. 0 .D Stearns, D. S. 10. .Cro Chen, Carson 1. A Appendix ainlde o sueayrsosblt o s faycrutydsrbd ocrutptn iessaeipidadNtoa eevstergta n iewtotntc ocag adcrutyadspecifications. and circuitry said change to notice without time any at right the reserves National and implied are licenses patent circuit no described, circuitry any of use for responsibility any assume not does National .Lf upr eie rssesaedvcso .Aciia opnn saycmoeto life a of component any is component critical A 2. or devices are systems NATIONAL or OF SUPPORT PRESIDENT devices LIFE THE support IN OF COMPONENTS APPROVAL Life WRITTEN CRITICAL 1. EXPRESS AS herein: used THE USE As WITHOUT FOR CORPORATION. SEMICONDUCTOR SYSTEMS AUTHORIZED OR NOT DEVICES ARE PRODUCTS NATIONAL’S POLICY SUPPORT LIFE 1966. 1979. crwHl,1969. McGraw-Hill, 1978. Wiley, John 1962. McGraw-Hill, Corporation. Semiconductor National inlProcessing, Signal Noise, ie,1979. Wiley, alr oprom hnpoel sdi codnespotdvc rsse,o oafc t aeyor safety its affect to life or the of system, effectiveness. failure or can the perform cause device to to failure support expected whose reasonably injury system be significant or can a device labeling, in support user. result the the to to in accordance expected provided in reasonably use whose be used for and properly life, instructions implant when sustain with or surgical perform, support to for (b) failure or intended body, are the into (a) which, systems e:180 7-99DushTl ( Tel: Deutsch 272-9959 1(800) Tel: a:180 3-08EgihTl ( Tel: English 737-7018 1(800) Fax: oprto uoeHn ogLd aa Ltd. Japan Ltd. Kong Hong Semiconductor National Semiconductor National cnjwge Email: ( Fax: Europe Semiconductor National 76017 TX Arlington, Road Bardin West 1111 Corporation Semiconductor National crwHl,1970. McGraw-Hill, ewr nlssadSynthesis, and Analysis Network h ore nerladIsApplications, Its and Integral Fourier The dacdEgneigMathematics, Engineering Advanced nomto rnmsin ouain and Modulation, Transmission, Information nItouto oteSmln Theorem, Sampling the to Introduction An h atFuirTransform, Fourier Fast The iia inlAnalysis, Signal Digital iia Filters, Digital rnieHl,1975. Prentice-Hall, hoyadApiaino Digital of Application and Theory iia rcsigo Signals, of Processing Digital ple ieSre Analysis, Series Time Applied rnieHl,1977. Prentice-Hall, tlaoTl ( Tel: Italiano Fran 3 i e:( Tel: ais asn 1975. Haysen, Prentice-Hall, a a a a a onWiley, John 9 -8-3 68 a:(5)2736-9960 (852) Fax: Kong Hong 80 16 0-180-534 49) 32 78 0-180-532 49) 9 -8-3 58 3hFor tagtBok e:81-043-299-2309 Tel: 2737-1600 (852) Tel: Kowloon Tsimshatsui, Block, Straight Floor, 13th 58 93 0-180-532 49) 85 85 0-180-530 49) 86 85 0-180-530 49) @ em.s.o ca ete atnR.Fx 81-043-299-2408 Fax: Rd. Canton 5 Centre, Ocean tevm2.nsc.com John- 5 .D Stanley, D. W. 15. 8 .W oly .A .Lws .D Welch, D. P. Lewis, W. A. P. Cooley, W. J. 18. 7 .C gwl .S Burrus, S. C. Agawal, C. R. 17. h uhrwse otakJmsMyradBrySiegel criticisms. Barry constructive and and Moyer guidance James their thank for to wishes author The Acknowledgements 2 .A Tretter, A. S. 12. 6 .F ute Jr., Tuttle, F. D. 16. 3 .Papoulis, A. 13. 4 .Shat,L Shaw, L. Schwartz, M. 14. il 1975. Hill, EE o.6,p.5050 pi 1975. April 550–560, pp. 63, Vol. IEEE, EETas ui lcracutc,V.A-5 pp. Au-15, Vo. 1967. June Electroaccoustics, 79–84, Audio Trans, IEEE om oIpeetFs iia Convolution, Digital Fast Implement to forms h atFuirTasomt optto fFourier of Computation to Transform Fourier Fast the pcrlAayi,Dtcin n Estimation, and Detection, Analysis, Spectral cessing, nerl,FuirSre,adCnouinIntegrals, Convolution and Series, Fourier Integrals, onWly 1976. John-Wiley, inlAnalysis, Signal nrdcint iceeTm inlPro- Signal Discrete-Time to Introduction iia inlProcessing, Signal Digital Circuits, crwHl,1977. McGraw-Hill, inlPoesn:Discrete, Processing: Signal ubrTertclTrans- Theoretical Number crwHl,1977. McGraw-Hill, 1975. plcto of Application McGraw- Proc. IMPORTANT NOTICE Texas Instruments Incorporated and its subsidiaries (TI) reserve the right to make corrections, modifications, enhancements, improvements, and other changes to its products and services at any time and to discontinue any product or service without notice. Customers should obtain the latest relevant information before placing orders and should verify that such information is current and complete. All products are sold subject to TI’s terms and conditions of sale supplied at the time of order acknowledgment. TI warrants performance of its hardware products to the specifications applicable at the time of sale in accordance with TI’s standard warranty. Testing and other quality control techniques are used to the extent TI deems necessary to support this warranty. Except where mandated by government requirements, testing of all parameters of each product is not necessarily performed. TI assumes no liability for applications assistance or customer product design. Customers are responsible for their products and applications using TI components. To minimize the risks associated with customer products and applications, customers should provide adequate design and operating safeguards. TI does not warrant or represent that any license, either express or implied, is granted under any TI patent right, copyright, mask work right, or other TI intellectual property right relating to any combination, machine, or process in which TI products or services are used. Information published by TI regarding third-party products or services does not constitute a license from TI to use such products or services or a warranty or endorsement thereof. Use of such information may require a license from a third party under the patents or other intellectual property of the third party, or a license from TI under the patents or other intellectual property of TI. Reproduction of TI information in TI data books or data sheets is permissible only if reproduction is without alteration and is accompanied by all associated warranties, conditions, limitations, and notices. Reproduction of this information with alteration is an unfair and deceptive business practice. TI is not responsible or liable for such altered documentation. Information of third parties may be subject to additional restrictions. Resale of TI products or services with statements different from or beyond the parameters stated by TI for that product or service voids all express and any implied warranties for the associated TI product or service and is an unfair and deceptive business practice. TI is not responsible or liable for any such statements. TI products are not authorized for use in safety-critical applications (such as life support) where a failure of the TI product would reasonably be expected to cause severe personal injury or death, unless officers of the parties have executed an agreement specifically governing such use. Buyers represent that they have all necessary expertise in the safety and regulatory ramifications of their applications, and acknowledge and agree that they are solely responsible for all legal, regulatory and safety-related requirements concerning their products and any use of TI products in such safety-critical applications, notwithstanding any applications-related information or support that may be provided by TI. Further, Buyers must fully indemnify TI and its representatives against any damages arising out of the use of TI products in such safety-critical applications. TI products are neither designed nor intended for use in military/aerospace applications or environments unless the TI products are specifically designated by TI as military-grade or "enhanced plastic." Only products designated by TI as military-grade meet military specifications. Buyers acknowledge and agree that any such use of TI products which TI has not designated as military-grade is solely at the Buyer's risk, and that they are solely responsible for compliance with all legal and regulatory requirements in connection with such use. TI products are neither designed nor intended for use in automotive applications or environments unless the specific TI products are designated by TI as compliant with ISO/TS 16949 requirements. Buyers acknowledge and agree that, if they use any non-designated products in automotive applications, TI will not be responsible for any failure to meet such requirements. Following are URLs where you can obtain information on other Texas Instruments products and application solutions: Products Applications Audio www.ti.com/audio Communications and Telecom www.ti.com/communications Amplifiers amplifier.ti.com Computers and Peripherals www.ti.com/computers Data Converters dataconverter.ti.com Consumer Electronics www.ti.com/consumer-apps DLP® Products www.dlp.com Energy and Lighting www.ti.com/energy DSP dsp.ti.com Industrial www.ti.com/industrial Clocks and Timers www.ti.com/clocks Medical www.ti.com/medical Interface interface.ti.com Security www.ti.com/security Logic logic.ti.com Space, Avionics and Defense www.ti.com/space-avionics-defense Power Mgmt power.ti.com Transportation and Automotive www.ti.com/automotive Microcontrollers microcontroller.ti.com Video and Imaging www.ti.com/video RFID www.ti-rfid.com OMAP Mobile Processors www.ti.com/omap Wireless Connectivity www.ti.com/wirelessconnectivity TI E2E Community Home Page e2e.ti.com