FAST" FOURIER TRANSFORMS-FOR FUN AND PROFIT

W. M. Gentleman

BeU Telephone Laboratories Murray HiU, New Jersey and G. Sande·

Princeton University Princeton, New Jersey

IMPLEMENTING FAST FOURIER which become practical when used with the fast TRANSFORMS algorithm, make the technique important. Let us therefore first consider some properties of such Definition and Elementary Properties of Fourier transforms. Transj017l'l8 The usual infinite Fourier integral transform is The "Fast Fourier Transform" has now been well known and widely used-the physicist when widely known for about a year. During that time it solving a partial differential equation, the communi­ has had a ma~or effect on several areas of computing, cation engineer looking at noise and the statistician the most striking example being techniques of numer­ studying distributions may all resort to Fourier icaJ convolution, which have been completely ~vo­ transforms, not only be.cause the mathematics may lutionized. What exactly is the "Fast Fourier be simplified, but because nature itself is often easier Transform''? to understand in terms of frequency. What is less well known is that many of the properties of the In fact. the Fast Fourier Transform is nothing usual Fourier transform also hold, perhaps with more than an algorithm whereby, for appropriate slight modification, for the Fourier transform defined length sequences, the finite discrete Fourier transform on finite, equispaced, discrete sequences. of the sequence may be computed much more rapidly than by other available algorithms. The properties We see in Table I that the most significant change and uses of the finite discrete Fourier transform, required to modify the usual theorems so that they apply to the finite discrete case is that indexing must .,... work made use of computer facilities supported in be considered modulo N. A useful heuristic inter­ part by National ~CDce Foundation grant NSF-GPS79. pretation of this is to think of the sequence X(t) as Research was partially ~ported by the Office of Naval a function defined at equispaced points on a circle. ~ UDder CODtract Nonr 1858(05) and by the Na­ UUIIIU Reeearch CouncIl of Canada. This interpretation is to be contrasted with what is

.. 563 564 PROCEEDINGS-FALL J~INT COMPUTER. CONFERENCE, 1966 TABLE I TABLE l-(Continued) A Comparison Usual and Finite Discrete 0/ Uaual Finite Dilcrete Fourier Trans/orms that is, the inverse Fourior that is, tho inverse Fourier Usual Finite Discrete transform of the product of transform of the product of the Fourier transforms. the F 0 uri e r tranaforma. Definition NOTE: Tho convolutioD ....1' here must be conaidered as AA N-I ,X(n ... !(t)et-lI~dt X(t) ~ X(t)e lf cyclic, i.e.. the indicoa of X. f__ '=0 and Y must be intorpreled modulo N. Linearity Operational Calculru The Fourior transform is a The Fourier transform is a An operational calculus can An operational calculus can linear operator. linear operator. be defined, based on the be defined, based on the OnhogonaUty property that property that N-I ... ,1 o:..llc1'-~dt - a(i'-'h X(t)j"'d' ~ (~X(t»e'- f-co iG. '..0 · A where a(i'- f) is the Dirac where IN is the Kronecker - -2'11'01(i) - (e --;:' -1) X(I) delta function. That is, delta function with ita argu­ ment being considered mod­ i.e., the Fourior transform Le., tho Fourier transform ulo N. That is, b(kN) of the derivative of a func­ of the (forward) cWfereoce /(0) if 0 /.i(i>a(l)di "" .... I, for intesor k, othOl­ tion ia tho FoUriOl trans­ of a function is the Fourier wise b - 0. , form of the function, mul­ traDlform of the fgnc:tIoa. is fD the interval (a,b), tiplied by -2'11'li: otherwise I multiplied by (e ~t-1). NOTE: Tho difference here J.bf(i> a(i>Ji ... o. must be considered cycJl. cally, so that ~X(t) - X(t+ 1) - X(t) becomr:a Inverse Trans/orm ~X(N-1) .... K(N) - 00 AI' X(N-l) = X(O) - A A A If X(/) If X(t) - 1_ !(/)e... lldl X(N-l) for the case of I = N - 1. thon then Symmetries A If X is real then X is bermi­ If X is real then X is bermf. eo", A " i\ 1 N-l -..III X(t):- X(t)e-Iw"'dt , X(I) ... -~. X(;)e-·- til].D symmetric, i.e., t(t) - til].D aympletric, i.e., .I(t) .. -co f Nt:: {X(-1»·j if X is hepni­ {X(N-1)}·j if X is ~er­ l tian symmetric then jf is mitian symmetric, then Is which we observe can be . wbich we observe can be r considered real. , A ~. If Y is imasfnary, thea considered If Y is imasinarY, then Y r is hermitian antiaym­ 1 is hOJ..lDitian antispnmotric, m 0 tri c, i.e., yO) - as X(t) ... as X(t) -- X N i.e., yet) - -{f(-n)·j -("(N-1»·j if Y Is her­ if Y is hermitian antisym­ mitian antiaymmolric theD N-I "lit}. metric thon Y is imasinary. t is imagiOary. . )"e-:-r {f-:X(iW..... '.G} • ~ {X(i> • I { NOTE: The uao of tile tenni hermitian aymmeCric or the complex conjugate of or the complex conjugate of and hermitian antisymmetric the Fourier transform of the tho Fourior transform of the complex conjupte of X. complex conjugate of X, di­ fD the discre~ cue is COD­ vided by N. sistont with that in tho umaI caso if we interpret ~ Convolution Theorem modUlo, N. Shifting Theore11U --X N FAST FOURIER TRANSFORMS-FOR FUN AND PROFIT 565 often the natural interpretation-as sampled data where from ..a continuous infinite function.

Basic Algebra 01 Fast Fourier Trans/orms We recognize At this point we will define the notation e(X) tJlflU so $at we may wP~ expressions such as those B-1 (bb) A-I ~ e - Z:(b) and 1 e (A)~ W,,(a) of Table ~ in a ~pler fonia. The two most impor­ "=0 B a=o A tant properties of e(X) are'·that as Fourier transforms themselves, applied to shorter e(X+Y) = e(X)e(Y) sequences. Obtaining a subsequence by starting at and the bth element and taking every Bth element e(X) = 1 if X is an integer. thereafter, in the manner W,,(a) is obtained from X(b+aB), is called decimating by B. Observe that Using this notation, the finite discrete Fourier tlJi sequence Z~(b) are not quite the sequence of transform' of the sequence X(I) is frequency a values from the transforms of these decimated sequences, but these values multiplied 'by N-t ( It) . = X(t)e N XC?) ~ Ii "twiddle fac;tor" e ( ab ). , AB Suppose N has factors A and B so that N AB. = The Fourier transform of the complete AB point • '" A A Then writing I = a + bA and I = b + aB where sequence may thus be accomplished by doing the B a,a = 0,1, .. . ,A-l and b,8 = 0,1, .. . ,B-1; we different A point Fourier transforms, multiplying have through by the appropriate twiddle factors, then X(a+bA) doing the A different B point Fourier transforms. This is a recursive formula which defines the larger = ~ ~ X(b+aB)e( {a+bAl~b+aB} ) Fourier transform in terms of smaller ones. The total number of operations is now proportional to AB(A +B) rather than (AB)" as it would be for a B-1 A-I {fA aa bb .. ) = ~ ~ X(b+aB)e - + - + - + ab direct implementation of the definition, hence the hOa=o B A B name "Fast Fourier Transform". Associated observations: = ~ ~ ':~(b+aB)' (~) e (;) e( ~) 1. Attention is drawn to the special ad­ vantage of factors 2 or 4, in that since a , as as is in~gral implies e( as) = 1 Fourier transform of a 2 or 4 point se­ quence may be done using only additions = ~ e (~)l e ~ ) ~ X(b+aB)e (~a)~ and subtractions, one stage of complex arithmetic may be avoided. If we define the B different sequences 2. Although the recursive algorithm above may be implemented directly, it is Wt(a) X(b+aH) a 0,1, .. . ,A-l = = much better to simulate the recursion as and the A different sequences described in the following sections, since one may avoid the unnecessary calculation Zt(b) = e (!) ~ X(b+aB)e (;) of some complex exponentials. With our programs, for example, the simulating th~ b = 0,1, .. . ,B-l recursion takes only 2/5 as long for a 210 point transform .• we can write the above equation as • ' . B 1 A • All programs discuaed in this paper were fmplemeated X(a+GB) = i e (bb) Za(b) usins ALCOR Algol 60 on tho mM 7094 at Princeton 11110 B University. .. 566 PROCEEDING~FALL JOINT COMPUTER CONFERBN,CB, 1966

3. The multi-dimensional finite discrete Fourier transform

e(~(a+bA+CAB») X(a+bA+cAB) . ABC

In both cases we may factor ~e tWiddle factor out­ side the summation. if we do tliis we obtain in the first case ...... can be computed efficiently by factorizing X(c+bc+aBC) = ~ e(~) eC~[~~~) in each dimension separately, as above. 4. The algebra presented here is not 2 e(b"&) ~(b~)2 e(~)X(a+bA+cA.B) quite that appearing in "An Algorithm for &..0 B B CaD .C the Machine Calculations of Complex (Cooley version) Fourier Series," by I. W. Cooley' and I. W. In the second case we obtain Tukey.1I The difference can be seen if we consider N as having three factors, N = X(c+tC+DBC) ~ A-1~ e (""')~ e ( ABab )"'1l , ABC. . . _ A ' '-0 the COoley-Tukey algebra then is e(bb) e(' e.(a+bA»)~ e(' .c2) X(a+bA+cA.B) B ABC .c-O C (Sande'version) The difference between these two versions boo comes important when we come to the details of implementing the fast Fourier transform. We go on to this next.

Fast Fourier Tra'ns!orms Using Scratch Storage

Let us consider the Sande version applied to an X sequence stored in serial order. The summation over c represents a C point Fourier transform of points spaced AB apart. There is one such transform for each of the AB values of a + bA. The result of one such transform is a frequency sequence indexed by ~. U we look at the twiddle. factor e ( 2-(a+ bA») we ~ /( ABC see that it depends upon ~ and a + bA in a very convenient manner. To introduce some descriptive If, rather than collecting on the unhatted variables terminology we may call "C the frequency for this as above, we choose to collect on the hatted vari­ analysis and a + bA the displacement for tbis anal­ ables, we obtain ysis. At this stage there are no free indices corre­ sponding to replications. When we store the inter­ mediate results in the scratch storage area, placing eaph of the C point Fourier transforms in contiguous blocks indexed by the displacement leads to elements X(a+bA +cAB)e +CAB») (;i-(a+~ stored at ~ + C(a+bA). The intermediate summa­ tion over b represents a B point Foini.er transform . e (b'(a+bA +CAB).) ~ (~(a+bA +CA~») with points spaced AC apart; one s\lCh 1ransform for AB ABC each of the AC values of c+ aC. The result of any FABr FOURIER TRANSFORMS-FOR FUN AND PROFIT 567

one of,these transforms is a frequency sequence in­ an algorithmic viewpoint when we use storage in this manner. We have only'reversed the roles of the dexed by ,. The twiddle factor e ( a&) depends , AB hatted and unhatted variables in the fo~atio~ of:the only upon a and b, leaving e as a free index. Here twiddle factors. we would call &the frequency for the analysis, a the To obtain the general case of more, than' :threc displacement for the: analysis and e the replication factors we proceed to gI'9up our factors, in three index. We would store the intermediate results at groups, for example N = (Pt .. •Pj-1)P J(PJ+l' . . P,,). 8 + tc + aBC. In this case the contiguous blocks If we identify are ~ out by the replication factor C: The outer summation over a represents an .If point FoUrier A = Pl" 'PH B = PI transform with points spaced BC apart; one such " trimsform for each of the BC Values of 8 + te. The t = PI +t ••• P" result of any' one of these transforms is ~ frequency and perform the above reductions we find that after sequence indexed by a. There ,is no twiddle factor in two steps with this identification we arrive at exactly this case. Here we would call a the frequency and the same place as after only one step with the ~ + ~C the replication index. There fs no displace­ identificatioIJ, "" ment at thiS stage: We' would store the results at C + bC + aBC., At this point we see that the' results A = Pl" 'PH B = PH are stored in serial order of frequency. C = P ••• P" When we compute one of the transforms we may J wish to include the twiddle factor in the coefficients We can think of this as moving factors from the "A" for the values of the replication index before com­ part through the lOB" part to the "c" part. puting new coefficients for a different displacement. When we write a program to implement this, we If the Fourier transform is on two points there ap­ set up a triply nested iteration. The outer loop selects pears tc? be ,no advantage to either choice. For four the current factor being ,moved and sets up th~ ljmits and ind~xing parameters for the innc;r loops. TJle points' it is more economical to ~o the Fourier tr~­ inte~ediate loop, in which ~e would compute, the D;lultiply by the factor. In the f~ an4 ~~ twid~e twiddle factor, corresponds to the displaceJD.ent. The other c.ses'it is more efficient if the twiddle factor 'is inner loop provides 'indexing over the replication absorbed into the transform coeo\cients. & • • \ • variables. ' If we ,rere tq uslf the Cooley vers~on OQ a s~uence stored in serial order we would obtain an algorithm Fast Fourier Transforms in Place ,which differs only ,in minor details. The summation over c represents a C point Fourier ~orm of Lei us reConsider 'the storage requirements for our algorithm. In particular, let us consider one of points spaced AB apart. The twiddle' ~t e ( be ) the small Fourier transforms' on c for some value of , • BC depends upon the frequency e and displacement b, II + bA. It is only during this transform that we will leaving a free as a replication index: The e1ements need this set of intermediate results. We have just are stored at + aC + bAC. The intermediate s1.im­ vacated the' C cells a + bA + cAB (indexed by c) e and are looking about for C cells in which to store mation' over b represents a B point ;Fourier trahsform our' answers (ilidexed by ~). Why not use the cells of j)Omts spaced AC apart. twiddie factor The that have just been vacated? e (cr(e+bc») ,depends upon ' 'the combined fre- Having 'made this observation, let us reexamine " ABC ,'J, , ' " the algorithm corresponding to the Sande factoriza­ queo.cy e + 1c and the displaCement a. T4ere is rio tiop. The' summation on c represents a C point free. indel§~ ne intermediate res~ are sto~ed ~t Fourier transform of points spaced AB apart. The C+ ~C -t a!!C. The outer ~u~atio~ represellts an , . ( c-(a+bA) ) , twiddle factor e depends upon the fro- ..( poJnt lloQder transfOp;)1 of results spaced BC ABC ,apart. ~ is no tWIddle factor in this case. The quency c and the displac~ent a +. bA. There is no final resultB ' would be stoIed in serial order at replication index. The intermediate tesults are stored c + be:... aBc. , at a + bA +cAB. The sunmiatiori on b repres~ts a These twofred11ctions are essentially identical fl:om B point Fourier transform of pOints A apart. The .. 568 PROCEEDINGS-PALL JOINT COMPUTER CONFERENCE, 15166 , TABLE n twi~e, fa~tor e ( Aia1J ') depends upon the frequency TIME FOR A 1024 POINT 'fRI\lt{SFORM : and the displacement a. The replication index is ~. BY VARIOUS METHODS We treat each of the blocks of length AB indexed by ~ in an exactly equivalent manner. The summation Melhbd Time over a,represen~ contiguous A point Fourier trans­ forms. The frequency is aand the replication index is radix 2 " 'S + cJ. There is no displacement. The final answers radix 4 (also radix 4 + 2 and mixed flcliccI) are then stored at a + SA + CAB. The use of "dis­ radix 2 (recursively implemented) - placement" and "replication" is now much more Goertzel's method suggestive and natural than when we used these - , ~ whlle describing the use of scratch storage. incrementing by one) and one with revcraCd dJ&ba The more general case of more than three factors (easily obtained by nesting loops with tho bmermoat is again obtained by grouping to obtain "A", "B", stepping by the largest increments) ~~ recopyJua and "C" and-then moving factors from the "A" part from one array to another where one coUDtm' is used to the "C" part; the program being written as a triply for each array. If all of the factors are tho aam.e. the nested iteration loop. , req~ed interchanges become ~e and it is pol­ • When we reexamine the Cooley factorization for si.,le to do the unscrambling in place. Ono may un­ serially stored data, we will see that it has an un­ scramble the real and imaginary pans separately, 10 pleasant feature. The summation over c represents a scratch space could be generated by backing ODD or C point Fourier transformation of points spaced AB the other onto auxiliary store. A slower method of unscrambling is to follqw the permutatlon cydca • ~part. The twiddle factor e( :! ) depends only that no scratch s~go is needed. upon b and C, leaving a as 'a free index. The inter­ Two impleDlentations, bo~ for ~eriallJ stored so­ mediate t:esults are stored at a + bA + cAB. The quenc~, have been described. Al~tely, two b;a­ summation over b represents a B point Fourier trans­ plementations, both for data stored with digit 10- fbrm" of points spaced A apart. The twiddle factor versed 'subscripts,' may be developed: in this ~ a(C'+Sc») '. the Cooley factorizJltion has more cOnvenient twick11e e ( ABC depends upon the displacement a ~d factors. For the last two, the final results are c:or­ A .... I rectIy stored-the ''unscrambling" having been doDe the combined frequency c + bC. For data which first Digit reversed subscripts may arise because was originally stored serially, this is not convenient we have to compute. The intermediate results are stored at a :+ bA + cAB. The summation over a represents 1. scrambled serially stored data, contiguous A point Fourier transforms. The final 2. not unscrambled after a previous results are stored at a + SA + eAB. More than Fourier transform, three factors can be handled in the same manner as 3. generated data in scrambled order. before. We have written various Fourier transform pr0- The CODlJQon unpleasant feature of both of these grams in the manner described above. These include algorithms is that, the Fourier coefficient for fre­ one- and multi-d~ensional verSions of radix 2 (aD q~ency e + tc + -aBc is stored at a+ tA + ~AB. factors equal to 2), radix 4 (all factors equal to 4), Such a storage scheme may be described as storage radix 4 + 2 (all factors. equal to 4 except perhaps with "digit reversed subscripts." Before we may use the last, wJtich may be 2), and mbted radices (N is our Fourier coefficients we must unscramble them.· factored into as many 4's as possible, a 2 if nec:ea­ UD8C1'BDlbling has been found to require only a small sary, then any 3'8, 5'8 or other prim~). 'TImes re­ proportion of the total time for the algorithm. A very quired to transform a 1024 point iequence are giveD elegant solution corresponds to running two counters, in Table n, including the time required by a recur­ one with normal digitS (easily obtained by simply sive radix 2 transform and by the pn>fast F01J1ier 4o'IbiI problem has nothiq to do with the representation transform "eftfcient" Goertze1's method-(which sdB f:!, munbera in any particular machine, and unless the ma­ requires order N~ operations) for comparison. cbiDo has a "revel'lO digit order" instruction (e.,., "reverse bit ontu" for a binary machine), DO advantap can be taken Our standBrd tool is the radix 4 + 2 Fourier traDI­ of· JUCh representation. form which combines the speed of radix 4 with the FAST FOURIER TRANSFORMS-FOR FUN AND PROFIT ~69

1Iexibility of ra~ 2.. The ~ed radices Fourier The 64 files are now brought into core one at a transform is used when other factors, for example time. When a file comes in, it is Fourier tran,sformed 10, are desired. The mixed radix is used less as it is using a standard in-core transform, then the points more bulky (requiring scratch storage for unscram­ are multiplied by the twiddle factor e(~/218), where bling as well as being a larger program) and is less b is the number of the file (0 through 63) and a is thoroughly optimized than .th~ radix 4 + 2 Fourier the frequency index for the particular point. The file transform. is then written out again. By choosing to use ·trans­ forms of only 4096 points, we can fully buffer these Fast Fourier Transforms Using Hierarchical Store operations, one file being Fourier transformed while As we become interested in doing larger and ~e previous file is being written out and the succeed­ larger Fourier transforms, we reach a point where we ing file read in. may not be able to have all of the data in core simul­ We now start on the second set of transforms. taneously, and some must be kept in slower store Here we make use of the random access feature of such as drum, disk, or tape. At the other end of the disk to reacomplex number) transmission that could be avoided if the sequence initially on tape, returning the transform to tape. We was initially available, and the final answers accept­ will assume that we are willing to do transforms as able, in appropriate form or could at least be over­ large as 4096 points (2la) in core. lapped with other computing. The first step is to create 64 (2°) files on disk, each of length 4096. The tape is read into core and the Roundoff Considerations various decimated sequences read out onto- disk­ the zeroth point going into the zeroth file, the first So far we have discussed 'the fast Fouper trans­ into the first, etc. up to the sixty-third into the sixty­ form as though it could be done with complete a~­ third, then the ~-fourth point starting. back in the curacy. What happens when the algorithm is carried zeroth file again, ' etc. Appropriately buffered, this out in a real fixed word-length computer using 1Ioat­ can run almost at disk transfer speed. . ing point arithmetic? How does the error in doing 570 PROCEBDINGS-FALL JOINT COMPUTER CONFE~CE. 1966 this compare with the error from a direct application q is the extent of the summation in the matrix of the definition? multiplication. . ... We may approach this question two ways-by The definition of the Fourier transform of X(t) is . strict bounds or by empirical values determined by also the definition of the multiplication of the vector experimentation. We shall derive bounds for the X(t) by the matrix {e(tt/N) }1,. ratio "of the Euclidean norm * of the error to the Expressing this in real arithmetic, we multiply the Euclidean norm of the data sequence, showing that real sequence {Re(X(t», Im(X(t»} by the real whereas for direct application of the defining formula matrix (; we can only bound this ratio by 1.06VN (2N)"·2-b, -~). if we factor N into n1n ••.• 1IJ; and use the fast I Fourier transform we can bound the ratio by where C = {cos 21r1t/N}f, and S = {sin 2."ft/N}i,. The norm of this lNX IN real matrix is 1.06\fN {f. (2nJ)8/1} 2-b. In particular: if all 11-1 11-1 the nJ are the same, so that N = trr, then the ratio is ~ ~ {c0sa21rU/N + sinI2-rrIt/N + less than 1.06VN k(2n)8/a2-', which isk/n'I.(lt-1) coSl21r1t/N + sinB21rtt/N} = V1Ni times that for the direct calculation. (b here is the I number of bits used in the m~tissa of the floating­ Since the extent of the summation is IN, we have point representation). We shall then see from em­ by the lemma that IIf.l{X>-XII. < 1.06xlN V2N1 pirical results that the form of the bound is actually 2-11 IIXII.· . A a fairly good description of the form of the observed The proof is completed by observing that II X II. = errors, although the numerical constant is somewhat VNII X II •• larger than is typical of actual trials. Theorem: If we use the fast Fourier transform, ~ Theorem: H we use the defining formula to Fourier toring N into n1na .•. nil, to transform a sequence transform a sequence X(t) of length N in a machine X (t) ~ a machine using floating point arithmetic us.iug floating point arithmetic and a fixed word and a fixed word length with a b bit mantissa, then, length with a b bit mantissa, then IItl

IIfJ(X)-XII. = VNlldMJJMlt-1' . . tJ,MIX - M~lt-l' .. M1X II. . = VN IItlM,J,M1c-l' . •t£MIX - MJI,Mlt-J!,M/t-a .• . t/,MIX + .MJj,M1c-l' . . t,tMIX - M~Jg-J},M~ . • •V41X

... + M,Mlt-1" .f/)J1X - M,MJg-l" .M1XII. 11 ~ ~ ~ VN IIM~ .. .M J+I (f!,MJt£MJ-l" .I/,M1X - MJI/,MJ-t •.• M 1X ) II. J-1 t.t but since the M J are orthogonal

= ¥N. i IIf/,MJ(f/,MJ-1••• f/,M1X) - M J(f},MJ-1' • ·tlM1X) II. /=1 FAST FOUlUBR TRANSFORMS-FOR FUN AND PROFIT S71

When we compute the bound for IIf,l(M,Y)- TABLE m AI,YII. find tliat since M may be partitioned into we J OBSERVED RATIOS OF ERROR NORM TO Nln, disjoint bloCks~ we can bound the error from SEQUENCE NORM FOR THREE RANDOM ". each block separately, getting, since In 'real arithme- SEQUENCES AT EACH LENGTH. tic each block is 2nJ square, and the norm of the (in units of 10-8) block is V1iii. a bound of 1.06(2n/)8/12~11Y.1I. where Y. is the aPProPriate part of Y. By using Radix 4+2 Pytbagoraa' theorem this yields Radix Radix with Goertzel's Defining LogsN 2 4+2 rounding method formula I/,lcMJY) - M1YI/. < 1.06(2n/)a/ll~IIY/l. 1.03 (1.03) .46 2.60 4.27 Finally we observe that except for error of order .53 (.53) .13 3.91 I.S4 Nl-l/lXI/. .S7 (.S7) .00 2.73 4.14 2 3.43 1.09 .92 9.73 S.lS 1I11M,-JJ,M,-3 • •• fJ.M1XIIB = IIM1••• M 1XIIR 2.23 .92 1.0S 12.6 3.31 1.89 1.96 ].07 IS.8 4.14 Immediately we have 11M, ... M,X!lR = IIXII. be- 3 4.04 4.99 2.6S 32.8 9.7S cause of the orthogonality, so S.S7 S.S8 2.69 16.6 13.0 4.51 S.OI 2.39 34.4 7.83 A A " ,. IItl(X)-XII. < 1.06 VN ~ (2n,)8/22-11 IlX/lR 4 9.14 7.11 2.12 98.9 17.7 1_1 8.64 5.92 2.91 9]'S 18.4 7.76 6.62 2.71 121. 17.1 CGrol1tuy: If a sequence is .transformed and then the S 10.7 11'.0 4.S8 202. 33.8 iamse transform applied to the result, the norm of 12.7 12.2 4.44 2S8. 36.7 die dlfrerence between the initial and final sequences 11.9 11.4 5.40 198. 36.2 6 13.2 11.2 3.38 S48. 69.1 can be bounded by 2x 1.06!2N>,a/·2~IIXII. if we 14.4 12.0 3.70 787. 15.2 _ ~ definition. or 2X ~.06 · ~ (2n/)8/12~IIXII. if 14.0 10.2 3.59 806. 63.8 1 7 17.6 17.0 6.24 1290. . 143. .. _ the fast Fourier tianstorm. 16.9 16.3 6.74 1990. 141. WIth tbts coron!U'Y in Diilid, an experiment we can 17.3 16.8 6.78 1900. 135. nadDy do is to tilee a sequence, transform, inverse 8 20.0 16.3 4.82 7050. 286. 20.1 16.6 5.09 3490. 288. ~ and then compute the norm of the differ- 20.7 16.2 4.66 S090. 269. .. between the ori~a1 and resultant sequences, 9 22.8 21.6 7.61 13700. S79. dMdIng this by the norm of the original sequence. 22.8 21.4 7.81 10700. 578. 'hb1e m gives these ratios for random Gaussian 22.9 21.7 7.91 l1S00. 561. 111 10 2S.2 21.0 5.82 32100. 1170. eequences of length 2 , k 1, .•. ,12, when the = .1160. aalllforms were Computed using: (1) a radix 2 trans- 2S.6 21.0 S.44 27600. 2S.8 21.0 S.44 29900. 1140. _ (2) a radix 4 + 2 transform; (3) a radix 4 + 2 11 28.1 26.2 8.70 tlDdmn with rounding instead of truncation; (4) a 28.1 26.1 8.S6 Goatzers method transform; and (5) a transform 28.5 26.5 8.S3 ---- Iddab directly implemented the definition. The re- 12 30.7 2S.5 6.30 30.7 2S.5 6.21 ..... of three repHcations of the experim~t are 31.1 2S.7 6.21 ... FIgure 1 fa a plot of the average ratio divided " IosaN apipit Jog.N to show the essentially linear depIIldence (tor ~e factor~ transforms) of the ratio ing solutions of partial difterential equations. Hock- "logaN. Attention is drawn to the common scaling ney 4 gives a discussion ¢ numeric~ solutions of .... of 1()-1 and the consistency of the ta~led ratios partial differential equations. He considers the prob- IIr die larger transforms. lem o ~ :%

Least Squares ApproXimation by Trigonometric Polynomials o RADIX 2 .. RADIX 4.2 Often one may want to approximate a given so­ quence by a band-limited sequence, that lsi a so­ a RADIlC 4+2 WITH ROUNDING INSTEAD quence generated by a low order trigonometric OF TRUNCATION polynomial. A possible example could be the deter­ .. GOERTZEC.s METHOD mination of filter coefficients to approximate a d~ • DEFINING FORMULA sired transfer function.· The use of expansions in terms of orthogonal functions to obtain least square approximations is well known. The Fourier trans­ form of a sequence gives its expansion in terms of the mutually orthogonal complex exponentials, hence to obtain the coefficients for the approximation, we simply retain the low order coefficients from the Fourier transform. We may compare the approxima­ tion to the original by looking at the inv~~ . Fourier transform of the coefficients after repJacJna tho UIl­ wanted coefficients by 7.ClOs. H we had wanted a positive definito .pproxima­ Fa • 7 • • 10 11 12 LOGaN tion, we could have approximated the sq~ root of the original sequence. The square of the approxima­ FJaure 1. Observed values of relative error normILopN. tion to the root is then the positive approximation. Squaring a sequence like this has the eft~ of con­ onto an nXn grid (Hockney considers n :: 48, one volving the Fourier coefficients. Other variations' in case of n 12X29, q 0,1,2, ... ). A Fourier = = the appro~ating problem may be handled by sim- transform with respect to x reduces the problem to ilar specialized techniques. . the solution of a system of equations for the c0- y A caution: Gibbs phenomenon is a persistent ordinate. By using a 12 point formula from Whit­ problem when using least squares, particularly in the 8 taker and Robinson, Hockney is able to achieve a case of trigonometric approximation. In some situa­ Fourier transform in "s/36 operations.· H rather he tions, the problem may be sufficiently bad to warrant were to use a fast Fourier transform for n = 2", the use of other approximation criteria. p = 0,1,2, ... he could achieve a Fourier transform . ~__ 1 3 . m approAUUate y - n log.n operations. This estimate Numerical Convolutions 2 of o~ons leads to a net count of 6n1loi2n for To date, the most in;tportant uses of the fast Hockney's method. Under this accounting, Hock­ Fourier transform have been in connection witti the ney's method is much superior to the other methods convolution theorem of Table I. Some uses of nu- he considers. It must be cautioned that the factoring merical convolutions are the following: , . Auto- and Cross-Covarlancea: 1. METHOD AND· ·Operation in this section Imeans a real operatioD of a multiplication and an addttiOD. TIMING CoNSIDERATIONS. In the analysis of time ,FAST FOURIER TRANSFORM~FOR FUN AND PROFIT 573 series by digital spectrum analysis, there are many N'=l H'=1 computational advantages to the "indirect" method ~ ~ of spectral estimation by COllij)uting the autocovari­ '=0 .=0 anee ant\ then F.ourier transforming rather 'than the "~' method by obtaining the periodogr~ and then smootbipg.l ,F~~ example, it is more practical to consider examJning a spectrum through several dif­ ferent spectral windows by the indirect method. Sim­ Darly, wh~ analyzing a pair of series with co­ and quadrature-spectra, we often find the cross­ {X'(/) }*Y'(s) ~ i(S-~-:T) ) covariance of the two series computationaIIy con- e( venient ' The first major use of the convolution theorem, by which by the orthogonality condition of Table I gives 8 Sande, was to compute the auto-covariance N'-1 N'-l 1 = l: l: {X'(t) )*Y'(S)8N,(S-t-T) Rzz(T) = N ~ {X(/)}*X(/+T) '=0 .=0 N'-l T = 0;±1,±2, ... ,±L = ~ {X"(/)}*Y'(/+T) and cross-covariance 1 RecaII that the indices must be interpreted modulo Rn(T) = N ~ {X(/)}*Y(/+T) N'. For T < N' - N, this expression is just N times T = 0,±i,~2, ... ±L. the cross-covariance RJCY because then the extraneous products are precisely zero. where the summations are overaII values of I for Note that the auto-covariance is precisely the wbtch 1I1e proc:hicts are defined. (These are not in the cross-covariance of a series with itself, so that the form ~f Table I but be readily put into that can outline just given also describes how to compute the form.) convolution theorem may lead to im­ no auto-covariance. On the other hand, we may take provements by a factor of 20 in' computing time as advantage of the fact that in most time series prob­ compared to the summing of lagged products. The two major problems are that these are not defined lems the series are real to compute two auto-covari­ cyclically and that N may.not be convenient for use ances at one time. We use one series as the real part of the fast Fourier transform. Appending zeros to and the other series as the imaginary part in forming one or both ends of the series solves both problems. the sequence which we Fourier transform, using the To see this, extend X to length N' > N + L by symmetry properties from Table I to separate them adding zeros 80 that later. Exactly how much faster the convolution theorem X'(/) = X(/) I = 0,1, .. . ,N-l approach is than the lagged products depends not X'(/) 0 I N, • .. ,N'-l = = only on the length of series and proportion of the BItend tho Y series similarly. Fourier transform available lags desired, but,also on details of the pro- both new sequences to obtain (2N-L) (L+l) , grams used. There are multiplica- 2 X'(f) = ~t X'{/)~~) tions and additions, plus associated ~dexing opera­ tions, in summing lagged products. On the other = ~1 hand, the number of operations required for the r{n Y'{S){;) Fourier transforms (appro~a~ely proportional to N' log N') depends on our choice of N' which can Inverse transform the sequence formed by the vary according to the availability of Fourier ~ans­ pIOduct of the Y~ transform and the complex con­ form routines, a routine which can handle mixed jugate of tho ]{I transform. factors being much more flexible than a radix 4 + 2 1 N'-l A A A A ( routine which restricts N' to be a power of two. Fig­ C{.,>. ~ {X'(/)}*Y'(/)e - ;,i) =Hi ure 2 gives a comparison of direct summing and S72 PROCEEDINGS-FALL JOINT COMPUTER CONFERENCE, 1966

of 12X22 which Hockney uses for his Fourier ttans­ form will be equivalent to the fast Fourier transform: The apparent reduction ooly comes into play as the 18 size, n, is increased past, 48. ~ Hockn~ points out, the calculation of Fourier' coefticlents has long J>een 4+2 ALGORITHM recommended (and avoided, bec;ause of cost) as a method of solving partial difterential equations. Perhaps the fast Fourier transform will render the • solution of partial difterential equatio~ both more e economical and more straightforward.

Least SqUQl'es Approximation 'by Trigonometric Polynomials o RADIX 2 s~ .. RADIX 4+2 Often one may want to approximate • given quence by a band-limited sequence, that la, a s~ a RADIlC 4+ 2 WITH ROUNDING INSTEAD quence generated by a low order trigOnometric OfF TRUNCATION polynomial. A possible example could be the deter­ .. GOERTZ£L.'S METHOD mination of filter coefficients to approJPmate a de­ • DEFINING FORMUL.A sired transfer function~a The use of expansions in terms of orthogonal functions to obtain least square approximations is well known. The llourler trans­ form of a sequence gives its expansion in terr,na of the mutually orthogonal complex exponentials, hence to obtain the coefficients for the approximation, we simply retain the low order coefficients from the Fourier transform. We may compare the ~pprOxima­ tion to the original by looking at the invm;se.}?oprler transform of the coefficients after replaCinj tho un­ wanted coefficients by =as. If we had wanted a positive definite appioxbna­ 8 7 II 10 1\ 12 LOGaN tion, we could have approxitp.ated the square root of the original sequence. The square of th~ approxima­ Figure 1. Observed valuca of relativo error norm/LoBaN. tion to the root is then the positive approximation. Squaring a s~uence like this has the effect of ~n­ onto nXn grid '(Hackney considen n 48, one an = volving the Fourier coefficients. Other variations in case of n = 12X2", q = 0,1,2, ... ). A Fourier the approximating problem may be hancDed by sim­ transform with respect to 'x reduces the problem to ilar specialized techniques. the solution of a system of equations for the y co­ A caution: Gibbs phenomenon is a penistent ordinate. By using a 12 point formula from Whit­ problem when using least squares, particularly in the 8 taker and Robinson, Hackney is able to achieve a case of trigonometric approximation. In some situa­ Fourier transform in n'J /36 operations.· If rather he tions, the problem may be sufficiently bad to warrant were to use a fast Fourier transform for n = 2", the use of other approximation criteria. p = 0,1,2, ... he could achieve a Fourier transform m. approxunate . 1y - 3 n logan operations. This estimate Numerical Convolutions 2 of operations leads to a net count of 6n21oiln for To date, the most in;lportant uses of the fast Hackney's' method. Under this accounting, Hock­ Fourier transform have been in connection with the ney's method is much superior to the other methods convolution theorem of Table I. Some uses of nu­ he considen. It must be cautioned that the factoring merical convolutions are the following: Auto- and Cross-Covariancea: 1. METHOD AND -Operation in this section I means a real operation of a multiplication and an addition . . TIMING CONSIDERATIONS. In the analysis of time FAST FOURIER TRANSFORMS-FOR FUN AND PROFIT 573

series by digital spectrum analysis, there are many 1 . N'=l N'=l N'=l computational advantages to the "indirect" method =- , ~ ~ ~ of spectral estimation by CODU'uting the autocovari­ N' 1:: '=0 .=0 ance and then Fourier transforming rather than the • A ' "direct" method by obtaining the periodogram and {X'(/)}*Y'(s)e( - ~:)e( - ~~)e(;) then smoothing.1 For example, it is more practical to consider examining a spectrum through several dif­ 1 N'-l ""-1 ferent spectral windows by the indirect method. Sim­ N' ~ ~ ilarly, when analyzing a pair of series with co­ '=0 .=0 and quadrature-spectra, we often find the cross­ covariance of the two series computationally con­ venient. The first major use of the convolution theorem, by which by the orthogonality condition of Table I gives 8 Sande, was to compute the auto-covariance N'-l N'-l 1 =~ ~ {X'(/) }*Y'(s)8N ,(S-I-T) RZZ{T) = N ~ {X{I)}*X{I+T) '=0 &=0 N'-l T = O,±l,±2, ... ,±L = ~ {X'(/)}*Y'(/+T) and cross-covariance '=0 Recall that the indices must be interpreted. modulo N'. For T < N' - N, this expression is just N times T = O,±1,±2, ... ±L. the cross-covariance Rzy because then the extraneous products are precisely zero. where the summations are overall values of I for Note that the auto-covariance is precisely the which the products are defined. (These are not in the cross-covariance of a series with itself, so' that the form of Table I but can be readily put into that outline just given also describes how ,to comPute. the form.) The convolution ,theorem may lead to im­ provements by a factor of 20 in computing time as auto-covariance. On the other hand, we may take compared to the summing of lagged products. The advantage of the fact that in most time series prob­ two DUljor problems are that these are not defined lems the series are real to compute two auto-covari:' cyclically BOd that N may not be convenient for use ances at one time. We use one series as the real part of the fast Fourier transform. Appending zeros to and the other series as the imaginary part in forming one or both ends of the series solves both problems. the sequence which we Fourier transform, using the To see this, extend X to length N' > N + L by symmetry properties from Table I to separate them adding zeros so that later. Exactly how much faster the convolution theorem X'{I) = X(I) 1= O,t, .. . ,N-t approach is than the lagged products depends not X'(/) = 0 1= N, ... ,N'-l only on the length of series and proportion of the Extend the Y series similarly. Fourier transform available lags desired, but also on detailS of the pro- both new sequences to obtain (2N-L) (L+l) ' .1 grams used. There are " . multipllca- . 2. X'(t) = ~1 X'(t){;) tions and additions, plus ass~ated ~dexing Qpera­ tions, in summing lagged products. On the other r(f) = ~1 Y'(S){;) hand, the number of operations required for the Fourier tran~forms (appro~a~ly proportional to N' log N') depends on our choice of N' which can Inyerse transform the sequence formed by the vary according to the availabi1i~ of Fourier ~ans­ product of the Y' transform and the complex con­ jugate of the X' transform. form routines, a routine which can \landle mixed factors !'eing much more flexible ~an a radix 4 + 2 C(T) = ~ . ~1 {X'(t)}*Y'(i)e( _ ;, ) routine which restricts N' to be a power of two. fig­ ure 2 gives a comparison of direct ,summing and 574 PROCEEDINGS-FALL JOINT COMPUTER CONFERENCE, 1966

17!1 tions is for series of length N/S. We are thus in­ terested in those products in the hatched area of the ~ 1!l0 diagram. The segments described above are indicated by the dotted lines; they are labeled X o, Xl,. . . X. and YO,Yh ••• Y •. We now see how 'RZY(T) may be '12S calculated, by summing the appropriate' crosS:-Co­ variances of X, with Y/. For inStaDce, for the par­ 100 ,.. ticular value of T illustrated, g ....." 7S RZY(T) - ~ {RZoY.{T) + Rz•yo (T' - ~) ""2 j: SO + RZ1Y1 {T) + RZaY1 (T - ~)

BY CONVOWTION THEDREM

3000 + RZ.Y.(T) +

Figuro 2. Computation times for auto-covariance of a 3000 + RZ'Y.{T)} point aeries, using convolution theorem or sum­ ming lagged products. Filtering: DIGITAL FILTERS. Often, when we have a set of data which is indexed by displacement or time, cpnvolution theorem times us~g our radix 4 + 2 we wish to "smooth" our data to reduce the eft~t of transform routine. Figure 3 shows (for this program) unwanted "noise". If our original data were repre­ the regions in which either direct summing or the sented by {X(t)} then forming convolution ~eorem is more efficient as functions of N andL. yet) = X{t)+X{t+ 1)+X{t+2) 3 2. ExTRA-LONG SERIES. When we have to com­ pute auto- or cross-covariances of extra-long series, would represent one method of $Dloothing' our data. we could' just continue to use the method of the pre­ More generally, we could choose to use any arbitrary vious section, doing the oversize -Fourier transforms weights we pleased so that by the method outlined in the section Fast Fourier Trans/orms Using Hierarchical Store. A simpler and 200.,...---.....,..---...,----r----, oq::asionally more efficient method is also av.wable. CONVOWTION THEOREM 'ASTER What we do ins~ead is to partition the series X and Y into contiguouS disjoint segments, and express the crosa-:c;ovariance Rzy m: terms of the cross-covari­ ances betw,een these various subseries. A simple diagram will illustrate the method. In Fig. 4 w~ have plotted all the products of the ele­ 1/1 ~ ments of series Y against those of series X" the plot­ "oJ ting position corresponding to the values of the re­ spective indices. The cross-covariance R.zy(r) then corresponds to 1/N times the sum over all elements on a line parallel to the main diagonal, but T elements above it. Such a line is shown on the diagram. As an example, we consider the problem of com­ putiug Rj:y for T = O,::::!:l, .. . ±L, when the largest 2000 4000 8000 8000 cross-coviuianee we can compute by the method of N Figure 3. Region ",here using convolutio~ theorem is faster 'the sub-section on Method and Timing Considera- than summing lagged producti. FAST FOUlUBR TRANSFORMS-FOR FUN AND PROFIT 575

(complex) gain of the moving average (or filter) ~etermined by th~ coefficients c(o), . . . ,c(k). Mov­ mg ~verages are linear in the data, for example, the mOVIng average of U(t) + yet) is the same as the sum of the separate moving averages of U(t) · and Y.(t). Thus if we consider our data as beuig com­ posed of many Fourier components, we see that the moving average affects each of the Fourier com­ ponents separately. All of these considerations can be com~actly stated using the terminology we have at hand. Taking the moving average is a convolution with a set of weights. The complex gain of the moving average is the Fourier transform of' the coefficients. Filtering corresponds to multiplying by the complex gain in the frequency domain. This has given but a small introduction to the leo Xt • x. analysis of linear filters. It does not tell us how to x SERIES design a filter to perform a desired task. For this Figure 4. Diasram of crOIll products for computing cross­ probJem, the reader is referred to general texts such covariance. as Hamming,lI or Blackman and Tukcy,l or to more specialized works such as Kaiser.lI yet) = c(k)X(t) + c(k-l)X(t+l) + ... 2. SECTIONING. After we have chosen the weights II + c(o)X(t+k) = ~ c(k-j)X(t+j). with which we would like to filter, we still must 1=0 choose how to implement the filter. A typical situa­ tion might be to have 15,000 data points which we H, rather than smoothing, we had desired to estimate wish to filter with 50 weights. We could do this by moving short-term integrals (possibly with a kernel) computing the convolution directly. We might also we would again be able to express the result as a extend our data to 16K by adding zeros and our summation over the data .points weighted by suitable filter to 16K weights by adding zeros and then doing coefficients. The estimation of derivatives also leads two Fourier transforms to get our convolution. For to a similar expression. this choice of numbers the decision is not dramati- Let us consider the application of such a moving caUy in favor of one or the other method. . average to a complex exponential of a fixed fre­ Applying a 16K Fourier transform to only 50 quency. Thus our data would be X(t) = e'III' = weights seems rather extravagant Perhaps we could e(ft) and the result of the moVing average would be do better if we were to filter only sections of the yet) = c(k)X(t) + ... + c(o)X(t+k) original data at anyone time as first suggested by Stockham.' The problem is then to find a sensible size for the various sections. Let us say that we have " c(k-j)X(t+i> - :I1-0 D data points and F filter weights. We want to find N (the section length) so as to minimize the total time. " c(k-J)e(/x {t+J}) This scheme will take about D/(N-F) Fourier trans­ - :I1=0 forms (transforming two real sequences at one time). The total time will then be t ..!!...- cNJ,n(N) e(tf) ~" c(k-j)e(fj). N-F . 1=0 where c may have a small dependency On N which We can thus represent yet) as the product of e(lt) we will ignore. By assuming that F < < D and that N is continuous we may find the minimum of t by and A (I) = ~" c(k-j)e(jj). A (f) is called the 1=0 differentiating with respect to N. Thus 576 PROCEEDINGS-PALL JOINT COMPUTER CONFERENCE, 1966

.... '" ~ ~( N) " N ~ = Dc ~ (Nj,n(N») (n) Z(t) = A (t} - 2" < t < 2'

'ON 'ON N-F .A. " A (b) Z( -N/2) = Z(N/2) = lI2X(N /2) if N even

...... A ~ , DC{J;~i + NNF~ ' - ~~~~~} (c) Z(s) =Oforallothervaluesofs. When we recall that X(1) has period N and that = Dc {(N-F)/'n(N) + N - F - Nj,n(N) } (~-F)· 2(9) has period NM we identify X( -1) = X(N-!) = Dc {_ Fln(N) + N - F} (N-F)· · , and Z( -!) = Z(NM-!) . ~ = 0 implies N - F(1 +In(N)) = 0 'ON The construction of. 2 corresponds to taking the circle on which the Fourier transfoQll ~cicnts which yields N of X are defined, breaking it at ~, and iilaerting F = 1 + J,n(N) . M(N-l) zeros so that the low frequency Fourier For our case of SO filter weights, this suggests coefficients of ·the two sequences match, and the high using Fourier transforms of length about 300, for a frequency coefficients of ZG> are zero. What is the reduction 'of the ~e required by a factor of two. inverse transform of ZOO?

InlerpolatltJn. oDe of the standard operations in 1 IIN-l A ( (S/NM )8 ) numerical computing is the iriterpolation in a table Z(i ) = MN e Zrs)e - of function values to obtain approximate values on a grid at ~ finer spacing than the original. ' which, we note, is j~t the band-limited interpolation 1 1. BAND-LIMlTBD INTBRPOLATION. The most ob­ of X(t), except for a factor -. For example, when M vious application of these ideas is to the case of s = Mt, band-limited interpOlation, 'sometimes called trigo­ 1 IIN-l j tJ) nometric or 'cosine interpolation. This interpolation Z(Mt) - Mii ~ .2(8)1" - N is done over the whole interval, by fitting a trigo­ nometric polynomial (or compleX exponential series) = 2.. X(t) by the definition of irs) of su1ilbiently high order through all ih~ given values, M a then evaluating it at the new points. But the first Applying this in the case where the original values step of this is exactly the finding of the Fourier trans­ are not at equispaced points merely makes the find­ form of the original data. And the second step, as we ing of the first transform, 2(1), more complicated. shall see, can be accomplished by inverse transform­ Furthermore, the extension of this scheme to multi­ ing the sequence whose low order coefficients are the ple dimensions is immediate. same as the coefficients of the transform of the origi­ We remark ·that a fundamental fact of band­ nil data, and whose high order coefficients are zero. limited interpOlation is that it is periodic, so that the To, justify this last statement, consider the case first points of a sequence inftuence the values in­ where the function is given at N equispaced grid terpolated between the last points. This property, and points. We wish to interpolate in this sequence to the long range effects of single values, may mean we obtain values at M times as many points. The origi! should avoid band-limited interpolations for certain nal sequence of values, X(t), has a Fourier transform uses. It may, however, be extremely useful in other X(I) such that we have the representation circumstances. 2. GENERAL INTERPOLATION RULES AS CONVO­ X(I) = 2. ~ 2(1)e(- tt) LUTIONS. Knowing the limitations of band-limited N N 'cO interpolation, we often prefer to use a local formula Consider the augmented Fourier transform i~) such as Lagrangian polynomial interpolation. Using s = 0,1, .• :,N'm-l such that any interpolation rule such that the interpolated FNrr FOURIER TRANSFORMS-FOR FUN AND PROFIT 577 values are linear combinations of the adjacent known This is a truncated power series evaluated at points values to subtabulate a table 'of equispaced data may on the unit circle in the complex plane. Our convo­ very conveniently be done by convolutions. U we lution theorem is now a statement about multiplying denote the value to be interpolated at I + p, 0 ~ p polynomials and using a Cauchy product representa­ < 1, by Z,(t), then the series Z,(t) is the convolution tion. The inverse Fourier transform gives us a way of of the original series X(/) with an appropriate series evaluating coefficients of polynomials given as values of weights: equispaced on the unit circle. /3 Of the appJications so far suggested, this most Z,(/) = ~ W,(S)X(/-S) deserves the epithet "fun." It is doubtful if this tech­ nique win supplant conventional symbol manipula­ It is possible to arrange that the convolutions for tion techniques as it makes no use of special prop­ aD values of p be done simultaneously, but the com­ erties of the coefficient sequences. It does, however, putational effort may be increased over ~ing them admirably illustrate that maintaining an open mind separately. Since, typically, the series of weights will may wen bring forth new, surprising and occasionally be short CQDlpared to 1I1e length of the series X(t), it benefiCial applications of finite discrete Fouriq may be profitable to' employ the sectioning described analysis. earlier. When applied to problems in more than one cUmension, additional profit may be made if we use Considering Problems in Trans/orm Space interpolation rules which are direct products or di­ It has long been a fact that some problems of rect sums of lower dimensional rules, for example, a physics are more tractable in transform space than two dimensional interpolation rule such that the they are in the regular' coordinates. Our refound weights W H(r,s) are either a product (r)W~(s) or w; ability to switch from one to the other may mean that a sum W~(r) W:(s). The advantage of such rules + some previously intractable problems will now be fa that the higher dimensional convolutions may then solvable. In electromagnetic problems, the return done as a succession of lower 'dimensional con­ be signal can often be represented as a convolution with volutions or equivalently that the Fourier transforms a kernel. Taking the Fourier transform can change of such sequences are the products or sums, respec­ the form of the non-linearity from convolution to tively, of the Fourier transforms of the constituent multiplication ' and may make the problems more parts. manageable. In pattern recognition problems, one Cauchy Products for SymboUc Polynomial Manipu­ may seek measures which are free from the effects tIltlon. The Cauchy product of two infinite series is a of linear transformations. Taking the Fourier trans­ well-known result of college algebra: form may greatly reduce the problem of finding such • • • measures. It has been suggested that quantizing the I ".%'. ~ bl%1 = ~ ~ where Fourier tr~sform of a signal rather than quantizing foCI 1.0 JpoO the signal itself may result in a higher quality trans­ It mission for communication. CIt = ~ Qllb".." ~ere are certainly a wealth of problems in which ..0 Fourier analysis arises as a most natural tool to be IIIbject to some convergence criteria. We readily rec­ employed. Unfortunately it has often been rejected oguize that the {CIt} are convolutions of the {a.} and because of its high computational cost Perhaps the (b,} and that we could use our techniques to evalu­ application of the fast Fourier transform could swing ate these discrete convolutions. the economic baJance to make this time-honored We may arrive at this by considering an alternative technique again competitive. Yiewpoint Let us write z = e(1J) and recognize that ,- =e(nIJ). With this notation the Fourier transform ACKNOWLEDGMENT becomes The authors would like to express their apprecia­ B-1 N-2 tion to those from whose work we have borrowed, X(1) = I X(t)e(tf/N) = ~ X(/)Zl where 1=0 1=0 and whose fruitful conversations have 'helped in the z = e(i!N) preparation of this paper. Tn particular we should 578 PROCEEDINGS-FALL JOINT COMPUTER C9NFBRENCE, 1966 • 1ilce to thank C. Bingham. A. B. Langdon, C. L. 5. J. F. Kaiser. "Some Practical Considerations Mallows, T. G. Stockham, and J. W. Tukey. in the Realization of Linear Digital Filters," Pro­ ceedings of the Third Allerton Conference on Cir­ REFERENCES cuit and System Theory, Monticello, Dlinois, Oc­ tober 1965. 1. R. B. Blackman and J. W. Tukey, The Meas­ 6. G. Sande, "On an Alternative Method of Cal­ urement of Power Spectra. Dover, New York, 1958. culating Covariance Functions," unpublished, Prince- 2. J. W. Cooley and J. W. Tukey, "An Algorithm ton University, 1965. . for the Machine Calculation of Complex Fourier Se­ 7. T. G. Stockham, "High Speed Convolution and Mathematics of Computation, vol. 19, no. 90, pes," Correlation," AFJPS. volume 28, 1966 Spring Joint (1965) pp. 297-301. 3. R. W. Hamming, Numerical Analysis for Sci­ Computer Conference. Spartan Books, Washington, entists and Engineers, McGraw-Hill, New York, 1966. 1962. . 8. Whittaker and Robinson, Calculus of Observa­ 4. R. W. Hockn~y, "A Fast Direct Solution of tions, Blackie & Son, London, 1944. Poisson's Equation USing Fourier Analysis," loU17UJl 9. J. H. Wilkinson, Roundinf EmJrs in Algebraic of the Association of Computing Machinery, vol. 12, Processes. Prentice-Hall, Englewood CIi1fs, New no. 1, (1965) pp. 95-113 . . Jersey, 1963.