Convolution Demystified

TERRY A. LORING

1. You’ve convolved before Convolution is made mysterious, but it comes up in a familiar place: polynomial multiplication. 2 3 2 Consider f(x) = a0 + a1x + a2x + a3x and g(x) = b0 + b1x + b2x . The annoying way to compute and fg, which is a polynomial, and to ﬁnd all its coeﬃcients, is like this:

[fg](x) = f(x)g(x) ¡ 2 3¢ ¡ 2¢ = a0 + a1x + a2x + a3x b0 + b1x + b2x ¡ 2¢ = a0 b0 + b1x + b2x ¡ 2¢ + a1 b0 + b1x + b2x ¡ 2¢ + a2 b0 + b1x + b2x ¡ 2¢ + a3 b0 + b1x + b2x 2 5 = a0b0 + a0b1x + a0b2x + ... + a3b2x which is past my pain threshold. You’ve all noticed long ago that the way a degree-n monomial is created is when a degree-r and a degree-s monomial come together and r + s = n. The coeﬃcient of xn is thus

Date: September 5, 2003. X arbs. Thus the quadratic term in fg is (a0b2 + a1b1 + a2b0) . Indeed r+s=n

[fg](x) = (a0b0)

= (a0b1 + a1b0) x 2 = (a0b2 + a1b1 + a2b0) x 3 = (a1b2 + a2b1 + a3b0) x 4 = (a2b2 + a3b1) x 5 = (a3b2) x

X The meaning of a sum like arbs is to create all possible pairs r+s=n (r, s) add these all together. There are only the order in a finite sum of reals in irrelevant, and yet most people feel better if an order is forced. Thus such a sum is usually rewritten to imply an order: X Xn arbs = arbn−r r+s=n r=0 We’re almost to convolution. Next we pad the polynomials f and g with zeros. For j ≤ −1 and for j ≥ 4, set aj = 0. For k ≤ −1 and k ≥ 3 set bk = 0. Then (for all real values of x, except perhaps zero) we have X∞ j f(x) = ajx j=−∞ and X∞ k g(x) = bkx k=−∞ (You can ignore ² and δ for these sums. Just toss the terms that equal zero and sum the finitely many terms that remain.) What’s more, we see that X∞ n [fg](x) = cnx n=−∞ where, for all n, X∞ cn = arbn−r r=−∞ 2 2. Convolving sequences We’ll consider all types of bi-infinite sequences later. Here’s a good place to start:

Definition 1. A doubly-infinite sequence x = hxni (index set the integers), has finite support is there are integers C1 and C2 such that

(n < C1 or n > C2) ⇒ xn = 0. (Equivalently, all but ﬁnitely many of the terms in the sequence are zero.) A good convention with a doubly inﬁnite sequence is to mark the term of index zero with an underline. Thus z = h... 0, 0, −1, 2, 1, 0, 0,...i is the sequence that has all terms zero except

z−1 = −1, z0 = 2, z1 = 1. To give x a playmate, we let w = h... 0, 0, 1, 0, 1, 0, 0,...i. Definition 2. Suppose x and y are double-infinite sequences of finite support. Then the convolution x∗y is again a double-infinite sequence, where X∞ [x ∗ y]n = xryn−r r=−∞ So, for example, z ∗ w = h... 0, −1, 2, 0, 2, 1, 0,...i. It is no coincidence that the following if true for all real x 6= 0 : ¡ ¢ ¡ ¢ −x−1 + 2 + x x−1 + 0 + x = −x−2 + 2x−1 + 0 + 2x + x2

3. A homorphism Mathematicians are always excited when one operations is turned into another by a function. For example, the log function is beloved precisely because of what it does to products: ln(ab) = ln(a) + ln(b) for positive a and b. 3 A pure mathematician is excited because the structure of the product operation can be explained via the structure of addition. An applied mathematician gets excited an invents the sliderule. All the variations on the Fourier transform perform a similar trick, converting regular multiplication (pointwise multiplication) of functions into convolution, and vice-versa. We found something very close to the Fourier transform above. Consider the set P of all functions from (−∞, 0] ∪ [0, ∞) to(−∞, ∞) that are polynomials in x and x−1. Let S denote the set of all doubly infinite real sequences of finite support. We make the following simple definition: If −2 −1 1 2 f(x) = ... + a−2x + a−1x + a0 + a1x + a2x ... then Φ(f) = ha−2, a−1, a0, a1, a2, i. (It is simple once you look past the notation. It just strips off the coefficients.) Here’s the type of formula that creates all the fuss: (†) Φ(fg) = Φ(f) ∗ Φ(g). We would rather have the reverse formula, that explains the convolution of sequences in terms of the pointwise product of functions. That is not really possible if we stick with real variables. But for a pure mathematician, (†) is inspiration enough to pursue this subject.

4. Differentiation made “easy” We will see that the Fourier Transform for R takes as input certain functions F : R → R and returns a new function Φ(f): R → R. Of the many formulas that hold, here’s one: Φ(f 0)(x) = xΦ(f)(x) In words, the Fourier transform converts differentiation into multiplication by x. (Multiplying by x is easier than taking derivatives, hence “differentiation made easy.” Of course, the Fourier Transform is harder than differentiation, so this is a bit of a mirage.) This formula opens lots of doors. Using the inverse of the Fourier Transform, Φ−1, we can get this: Dnf(x) = Φ−1(xnΦ(f)(x)) 4 for n = 1, 2,... Now, just because we can, we put in a non-integer for n and call this a definition. In particular, for n = 1/2, we define D1/2(f) = Φ−1(x1/2Φ(f)(x)). We now have the order “1/2” derivative of (some) functions! Study- ing this is outside the scope of this class. I just present this to you to give you more of a sense of why the Fourier Transform is so essential in pure math.