A Short Course on Approximation Theory N. L. Carothers Department of Mathematics and Statistics Bowling Green State University ii Preface These are notes for a topics course offered at Bowling Green State University on a variety of occasions. The course is typically offered during a somewhat abbreviated six week summer session and, consequently, there is a bit less material here than might be associated with a full semester course offered during the academic year. On the other hand, I have tried to make the notes self-contained by adding a number of short appendices and these might well be used to augment the course. The course title, approximation theory, covers a great deal of mathematical territory. In the present context, the focus is primarily on the approximation of real-valued continuous functions by some simpler class of functions, such as algebraic or trigonometric polynomials. Such issues have attracted the attention of thousands of mathematicians for at least two centuries now. We will have occasion to discuss both venerable and contemporary results, whose origins range anywhere from the dawn of time to the day before yesterday. This easily explains my interest in the subject. For me, reading these notes is like leafing through the family photo album: There are old friends, fondly remembered, fresh new faces, not yet familiar, and enough easily recognizable faces to make me feel right at home. The problems we will encounter are easy to state and easy to understand, and yet their solutions should prove intriguing to virtually anyone interested in mathematics. The techniques involved in these solutions entail nearly every topic covered in the standard undergraduate curriculum. From that point of view alone, the course should have something to offer both the beginner and the veteran. Think of it as an opportunity to take a grand tour of undergraduate mathematics (with the occasional side trip into graduate mathematics) with the likes of Weierstrass, Gauss, and Lebesgue as our guides. Approximation theory, as you might guess from its name, has both a pragmatic side, which is concerned largely with computational practicalities, precise estimations of error, and so on, and also a theoretical side, which is more often concerned with existence and uniqueness questions, and \applications" to other theoretical issues. The working profes- sional in the field moves easily between these two seemingly disparate camps; indeed, most modern books on approximation theory will devote a fair number of pages to both aspects of the subject. Being a well-informed amateur rather than a trained expert on the subject, however, my personal preferences have been the driving force behind my selection of topics. Thus, although we will have a few things to say about computational considerations, the primary focus here is on the theory of approximation. By way of prerequisites, I will freely assume that the reader is familiar with basic notions from linear algebra and advanced calculus. For example, I will assume that the reader is familiar with the notions of a basis for a vector space, linear transformations (maps) defined on a vector space, determinants, and so on; I will also assume that the reader is familiar with the notions of pointwise and uniform convergence for sequence of real-valued functions, iii iv PREFACE \"-δ" and \"-N" proofs (for continuity of a function, say, and convergence of a sequence), closed and compact subsets of the real line, normed vector spaces, and so on. If one or two of these phrases is unfamiliar, don't worry: Many of these topics are reviewed in the text; but if several topics are unfamiliar, please speak with me as soon as possible. For my part, I have tried to carefully point out thorny passages and to offer at least a few hints or reminders whenever details beyond the ordinary are needed. Nevertheless, in order to fully appreciate the material, it will be necessary for the reader to actually work through certain details. For this reason, I have peppered the notes with a variety of exercises, both big and small, at least a few of which really must be completed in order to follow the discussion. In the final chapter, where a rudimentary knowledge of topological spaces is required, I am forced to make a few assumptions that may be unfamiliar to some readers. Still, I feel certain that the main results can be appreciated without necessarily following every detail of the proofs. Finally, I would like to stress that these notes borrow from a number of sources. Indeed, the presentation draws heavily from several classic textbooks, most notably the wonderful books by Natanson [41], de La Vall´eePoussin [37], and Cheney [12] (numbers refer to the References at the end of these notes), and from several courses on related topics that I took while a graduate student at The Ohio State University offered by Professor Bogdan Baishanski, whose prowess at the blackboard continues to serve as an inspiration to me. I should also mention that these notes began, some 20 years ago as I write this, as a supplement to Rivlin's classic introduction to the subject [45], which I used as the primary text at the time. This will explain my frequent references to certain formulas or theorems in Rivlin's book. While the notes are no longer dependent on Rivlin, per se, it would still be fair to say that they only supplement his more thorough presentation. In fact, wherever possible, I would encourage the interested reader to consult the original sources cited throughout the text. Contents Preface iii 1 Preliminaries 1 Introduction . 1 Best Approximations in Normed Spaces . 1 Finite-Dimensional Vector Spaces . 4 Problems . 9 2 Approximation by Algebraic Polynomials 11 The Weierstrass Theorem . 11 Bernstein's Proof . 12 Landau's Proof . 14 Improved Estimates . 16 The Bohman-Korovkin Theorem . 18 Problems . 20 3 Trigonometric Polynomials 23 Introduction . 23 Weierstrass's Second Theorem . 27 Problems . 29 4 Characterization of Best Approximation 31 Introduction . 31 Properties of the Chebyshev Polynomials . 37 Chebyshev Polynomials in Practice . 41 Uniform Approximation by Trig Polynomials . 42 Problems . 45 5 A Brief Introduction to Interpolation 47 Lagrange Interpolation . 47 Chebyshev Interpolation . 53 Hermite Interpolation . 54 The Inequalities of Markov and Bernstein . 57 Problems . 60 6 A Brief Introduction to Fourier Series 61 Problems . 72 v vi CONTENTS 7 Jackson's Theorems 73 Direct Theorems . 73 Inverse Theorems . 76 8 Orthogonal Polynomials 79 The Christoffel-Darboux Identity . 86 Problems . 88 9 Gaussian Quadrature 89 Introduction . 89 Gaussian-type Quadrature . 92 Computational Considerations . 94 Applications to Interpolation . 95 The Moment Problem . 96 10 The M¨untz Theorems 101 11 The Stone-Weierstrass Theorem 107 Applications to C2π ................................111 Applications to Lipschitz Functions . 113 Appendices A The `p Norms 115 B Completeness and Compactness 119 C Pointwise and Uniform Convergence 123 D Brief Review of Linear Algebra 127 Sums and Quotients . 127 Inner Product Spaces . 128 E Continuous Linear Transformations 133 F Linear Interpolation 135 G The Principle of Uniform Boundedness 139 H Approximation on Finite Sets 143 Convergence of Approximations over Finite Sets . 146 The One Point Exchange Algorithm . 148 References 151 Chapter 1 Preliminaries Introduction In 1853, the great Russian mathematician, P. L. Chebyshev (Cebyˇsev),whileˇ working on a problem of linkages, devices which translate the linear motion of a steam engine into the circular motion of a wheel, considered the following problem: Given a continuous function f defined on a closed interval [ a; b ] and a posi- Pn k tive integer n, can we \represent" f by a polynomial p(x) = k=0 akx , of degree at most n, in such a way that the maximum error at any point x in [ a; b ] is controlled? In particular, is it possible to construct p so that the error max a≤x≤b jf(x) − p(x)j is minimized? This problem raises several questions, the first of which Chebyshev himself ignored: { Why should such a polynomial even exist? { If it does, can we hope to construct it? { If it exists, is it also unique? R b 2 { What happens if we change the measure of error to, say, a jf(x) − p(x)j dx? Exercise 1.1. How do we know that C[ a; b ] contains non-polynomial functions? Name one (and explain why it isn't a polynomial)! Best Approximations in Normed Spaces Chebyshev's problem is perhaps best understood by rephrasing it in modern terms. What we have here is a problem of best approximation in a normed linear space. Recall that a norm on a (real) vector space X is a nonnegative function on X satisfying kxk ≥ 0, and kxk = 0 if and only if x = 0, kαxk = jαjkxk for any x 2 X and α 2 R, kx + yk ≤ kxk + kyk for any x, y 2 X. 1 2 CHAPTER 1. PRELIMINARIES Any norm on X induces a metric or distance function by setting dist(x; y) = kx − yk. The abstract version of our problem(s) can now be restated: Given a subset (or even a subspace) Y of X and a point x 2 X, is there an element y 2 Y that is nearest to x? That is, can we find a vector y 2 Y such that kx − yk = min z2Y kx − zk? If there is such a best approximation to x from elements of Y , is it unique? It's not hard to see that a satisfactory answer to this question will require that we take Y to be a closed set in X, for otherwise points in Y n Y (sometimes called the boundary of the set Y ) will not have nearest points.