A Short Course on

Approximation Theory

Math Summer

N L Carothers

Department of Mathematics and Statistics

Bowling Green State University

Table of Contents

Preliminaries

Problem Set Function Spaces

Approximation by Algebraic Polynomials

Problem Set Uniform Approximation byPolynomials

Trigonometric Polynomials

Problem Set Trigonometric Polynomials

Characterization of Best Approximation

Problem Set Chebyshev Polynomials

Examples Chebyshev Polynomials in Practice

A Brief Intro duction to Interp olation

Problem Set Lagrange Interpolation

Approximation on Finite Sets

A Brief Intro duction to Fourier Series

Problem Set Fourier Series

Jacksons Theorems

Orthogonal Polynomials

Problem Set Orthogonal Polynomials

Gaussian Quadrature

The Mun tz Theorems

The StoneWeierstrass Theorem

A Short List of References

Preface

These are notes for a six week summer course on approximation theory that I oer o c

casionally at Bowling Green State University Our summer classes meet for minutes

vedaysaweek for a total of hours But the pace is somewhat leisurely and there is

probably not quite enough material here for a regulation one semester hour course

On the other hand there is more than enough material here for a one quarter hour

course and evidently enough for a veorsixweek summer course

I should stress that my presentation here is by no means original I b orrow heavily

from a number of well known texts on approximation theory see the list of references at

the end of these notes I use T J Rivlins b o ok AnIntroduction to the Approximation

of Functions as a complementary text and thus you will see many references to Rivlin

throughout the notes Also a few passages here and there are taken from my b o ok Real

Analysis In particular large p ortions of these notes are based on copyrighted material

They are oered here solely as an aid to teachers and students of approximation theory

and are intended for limited p ersonal use only I should also p oint out that I am not an

exp ert in approximation theory and I make no claims that the material presented here is

in current fashion among exp erts in the eld

My interest in approximation theory stems from its b eauty its utility and its rich

history There are also many connections that can b e drawn to questions in b oth classical

and mo dern analysis For the purp oses of this short intro ductory course I fo cus on a

handful of classical topics with a little bit of mo dern terminology here and there and

name theorems Indeed the Weierstrass approximation theorem along with its various

relativesisthecentral theme of the course

In terms of prerequisites I assume at least a one semester course in advanced calcu

lus or real analysis compactness completeness uniform convergence uniform continuity

normed spaces etc along with a course in linear algebra The rst chapter entitled

Preliminaries contains four brief app endices that provide an all to o brief review of such

topics they are included in order to make the notes as selfcontained as p ossible The

course is designed for b eginning masters students in b oth pure and applied mathemat

ics but should b e largely accessible to advanced undergraduates From my exp erience

there are plenty of topics here that even advanced PhD students will nd entertaining

Math Preliminaries

Intro duction

In the great Russian mathematician P L Chebyshev Cebysev while working on a

problem of linkages devices which translate the linear motion of a steam engine into the

circular motion of a wheel considered the following problem

Given a continuous function f dened on a closed interval a b and a p ositive

P

n

k

integer n can we represent f by a p olynomial px a x of degree

k

k

at most n in sucha way that the maximum error at any p oint x in a b is

controlled In particular is it p ossible to construct p in sucha way that the error

max jf x pxj is minimized

axb

This problem raises several questions the rst of which Chebyshev himself ignored

Why should such a p olynomial even exist

Ifitdoescanwe hop e to construct it

If it exists is it also unique

R

b

What happ ens if wechange the measure of the error to say jf x pxj dx

a

Chebyshevs problem is p erhaps b est understo o d by rephrasing it in mo dern terms

What wehave here is a problem of linear approximation in a normed linear space Recall

that a norm on a real vector space X is a nonnegative function on X satisfying

kxk and kxk x

kxk jjkxk for R

kx y k kxk ky k for any x y X

Any norm on X induces a metric or distance function by setting distx y kx y kThe

abstract version of our problems can now b e restated

Given a subset or even a subspace Y of X and a p oint x X is there an

element y Y which is nearest to x that is can we nd a vector y Y such

that kx y k infkx z k If there is such a b est approximation to x from

z Y

elements of Y is it unique

Preliminaries

Examples

P

n

n n

k In X R with its usual norm kx jx j the problem has

k k

k

k

a complete solution for any subspace or indeed any closed convex set Y This

problem is often considered in Calculus or Linear Algebra where it is called least

squares approximation A large part of the current course will b e taken up with

leastsquares approximations to o For now lets simply note that the problem changes

n

character dramatically if we consider a dierent norm on R

Consider X R under the norm kx y k maxfjxj jy jg and consider the

subspace Y fyy Rg ie the y axis Its not hard to see that the p oint

R has innitely many nearest p oints in Y indeed every p ointy x

y is nearest to x

n

There are manynormswemight consider on R Of particular interest are the

p

norms that is the scale of norms

p

n

X

n p

kx k p jx j

i p k

i

k

and

n

kx k maxjx j

i i

i

in

Its easy to see that kk and kk dene norms The other cases take a bit more

work well supply full details later

a b the space of all continuous functions Our original problem concerns X C

f a b R under the uniform norm kf k max jf xj The word uniform is

axb

used b ecause convergence in this norm is the same as uniform convergence on a b

kf f k f f on a b

n n

In this case were interested in approximations byelements of Y P thesubspace

n

of all p olynomials of degree at most n in C a b Its not hard to see that P is a

n

nitedimensional subspace of C a b of dimension exactly n Why

If we consider the subspace Y P consisting of all p olynomials in X C a b

we readily see that the existence of b est appro ximations can b e problematic It follows

Preliminaries

from the Weierstrass theorem for example that each f C a b has distance

from P but since not every f C a b is a p olynomial why we cant hop e for

a b est approximating p olynomial to exist in every case For example the function

f xx sinxiscontinuous on but cant p ossibly agree with any p olynomial

on Why

The key to the problem of p olynomial approximation is the fact that each P is

n

nitedimensionalTo see this it will b e most ecient to consider the abstract setting of

nitedimensional subspaces of arbitrary normed spaces

Soft Approximation

Lemma Let V b e a nitedimensional vector space Then all norms on V are equivalent

That is if kk and jjjj jj are norms on V then there exist constants AB such

that

A kxkjj jx jjj B kxk

for all v ectors x V

Proof Supp ose that V is ndimensional and that kk is a norm on V Fix a basis

e e for V and consider the norm

n

n n

X X

n

a e k ja j ka

i i i i

i

i i

P

n

a e V Since e e is a basis for V its not hard to see that k k is for x

i i n

i

indeed a norm on V Itnow suces to show that kk and kk are equivalent Why

One inequalityiseasytoshow indeed notice that

n n n n

X X X X

a e ja j B max ke k ja jke k a e

i i i i i i i i

in

i i i i

The real work comes in establishing the other inequality

n

To b egin notice that weve actually setup a corresp ondence b etween R and V

P

n

n

sp ecically the map a a e is obviously b oth onetoone and onto Moreover

i i i

i

i

n

this corresp ondence is an isometry betw een R kk and V kk

Preliminaries

Now the inequalityweve just established shows that the function x kxk is contin

uous on the space V kk since

kxkky k kx y kB kx y k

for any x y V Thus kk assumes a minimum value on the compact set

S fx V kxk g

WhyisS compact In particular there is some A suchthat kxkA whenever

kxk Whycanwe assume that A The inequalitywe need now follows from

the homogeneit y of the norm

x

A kxkA kxk

kxk

Corollary Given abxed and a p ositiveinteger n there exist A B

n n

constants whichmay dep end on n such that

n n n

X X X

k

B ja j a x A ja j max

n k k n k

axb

k k k

Exercise

Find explicit formulas for A and B above This can b e done without any fancy

n n

theorems If it helps you may consider the case a b

Corollary Let Y b e a nitedimensional normed space and let M Then any closed

ball f y Y ky kM g is compact

Proof Again supp ose that Y is ndimensional and that e e is a basis for Y From

n

our previous lemma we know that there is some constant Asuchthat

n n

X X

a e A ja j

i i i

i i

P

n

for all x a e Y In particular

i i

i

n

X

M

M ja j a e for i n A ja j

i i i i

A

i

Preliminaries

Thus fy Y ky k M g is a closed subset why of the compact set

n

X

M

x i n a e ja j

i i i

A

i

Corollary Every nitedimensional normed space is complete In particular if Y is a

nitedimensional subspace of a normed linear space X then Y is a closed subset of X

Theorem Let Y b e a nitedimensional subspace of a normed linear space X and let

x X Then there exists a not necessarily unique y Y such that

kx y k minkx y k

y Y

for all y Y That is there is a b est approximation to x byelements of Y

Proof First notice that since Y weknow that a nearest p oint y will satisfy

kx y kkxk kx k Thus it suces to lo ok for y among the vectors y Y

satisfying kx y k kxk It will b e convenient to use a slightly larger set of vectors

though By the triangle inequality

kx y kkxk ky kkx y k kxkkxk

Thus wemay restrict our attention to those y s in the compact set

K fy Y ky kkxkg

To nish the pro of we need only notice that the function f y kx y k is continuous

kx y kkx z k ky z k jf y f z j

hence attains a minimum value at some p oint y K

Corollary For each f C a b and eachpositiveinteger n there is a not necessarily

P suchthat unique p olynomial p

n

n

kf pk kf p k min

n

pP n

Preliminaries

Corollary Given f C a b and a xed p ositiveinteger n there exists a constant

R such that if

n

X

k

f a x kf k

k

k

then max ja jR

k

k n

Examples

Nothing in our Corollary says that p will b e a p olynomial of degree exactly nrather a

n

p olynomial of degree at most nFor example the b est approximation to f xx bya

p olynomial of degree at most is of course pxxEven examples of nonp olynomial

functions are easy to come by for instance the b est linear approximation to f xjxj

is actually the constant function px and this makes for an entertaining on

exercise

Before weleave these soft arguments b ehind lets discuss the problem of uniqueness

of b est approximations First lets see whywewant b est approximations to b e unique

Lemma Let Y b e a nitedimensional subspace of a normed linear space X and supp ose

that each x X has a unique nearest p oint y Y Then the nearest p ointmapx y

x x

is continuous

Proof Lets write P xy for the nearest p oint map and lets supp ose that x x

x n

in X Wewant to show that P x P x and for this its enough to show that there is

n

a subsequence of P x which converges to P x Why

n

Since the sequence x is b ounded in X say kx k M for all nwehave

n n

kP x kkP x x k kx kkx kM

n n n n n

Thus P x is a b ounded sequence in Y a nitedimensional space As such by passing

n

to a subsequence wemay supp ose that P x converges to some element P Y How

n

Nowwe need to show that P P x But

kP x x kkP x x k why

n n n

for any n and hence letting n

kP xkkP x xk

Preliminaries

Since nearest p oints in Y are unique wemust have P P x

Exercise

Let X b e a normed linear space and let P X X Show that P is continuous at x X

if and only if whenever x x in X some subsequence of P x converges to P x

n n

Hint The forward direction is easy for the backward implication supp ose that P x

n

fails to converge to P x and work toward a contradiction

It should b e p ointed out that the nearest p oint map is in general nonlinear and as

such can b e very dicult to work with Later well see at least one case in which nearest

p oint maps always turn out to b e linear

We next observe that the set of b est approximations is always pretty reasonable

Theorem Let Y b e a subspace of a normed linear space X and let x X The set Y

x

consisting of all b est approximations to x out of Y is a b ounded conv ex set

Proof As weve seen the set Y is a subset of fy X ky kkxkg and hence is

x

b ounded

Now recall that a subset K of a vector space V is said to b e convex if K contains the

line segment joining any pair of its p oints Sp ecically K is convex if

x y K x y K

Now y y Y means that

x

kx y k kx y k minkx y k

y Y

set y y y Wewant to showthat y Y but notice Next given

x

that we at least have y Y Finallywe estimate

kx y k kx y y k

kx y x y k

kx y k kx y k

minkx y k

y Y

Preliminaries

Hence kx y k minkx y kthatisy Y

x

y Y

Exercise

If in addition Y is nitedimensional show that Y is closed hence compact

x

If Y contains more than one p oint then in fact it contains an entire line segment

x

Thus Y is either emptycontains exactly one p oint or contains innitely many p oints

x

This observation gives us a sucient condition for uniqueness of nearest p oints If our

normed space X contains no line segments on any sphere fx X kxk r g then any

b est approximation out of any set will b e unique

A norm kk on a vector space X is said to b e strictly convex if for any x y X

with kxk r ky kwe always have kx y k r for any That is

the op en line segmentbetween any pair of p oints on the surface of the ball of radius r in

X lies entirely inside the ball We often simply say that the space X is strictly convex

with the understanding that a prop erty of the norm in X is implied Heres an immediate

corollary to our last result

Corollary If X has a strictly convex norm then for any subspace Y of X and any p oint

x X there can b e at most one b est approximation to x out of Y ThatisY is either

x

empty or consists of a single p oint

In order to arrive at a condition thats somewhat easier to check lets translate our

original denition into a statement ab out the triangle inequalityin X

Lemma X has a strictly convex norm if and only if the triangle inequality is strict on

nonparallel vectors that is if and only if

x y y x all R kx y k kxk ky k

Proof First supp ose that X is strictly convex and let x and y b e nonparallel vectors

in X Then in particular the vectors xkxk and yky k must b e dierent Why Hence

x y kxk ky k

kxk ky k kxk kxk ky k ky k

That is kx y k kxk ky k

Preliminaries

Next supp ose that the triangle inequality is strict on nonparallel vectors and let

x y X with kxk r ky k If x and y are parallel then wemust have y x

Why In this case

kx y k j jkxk r

since j j whenever Otherwise x and y are nonparallel In this case

for any the vectors x and y are likewise nonparallel Thus

kx y k kxk ky k r

Examples

The usual norm on C a b isnot strictly convex and so the problem of uniqueness

of b est approximations is all the more interesting to tackle For example if f xx

and g xx in C then kf k kg k f g while kf g k Why

n

The usual norm on R is strictly convex as is any one of the norms kk p

p

Well prove these facts shortly The norms k k and kk on the other hand are

not strictly convex Why

App endix A

For completeness we supply a few of the missing details concerning the norms We

p

b egin with a handful of classical inequalities of indep endentinterest First recall that we

n

have dened a scale of norms on R by setting

p

n

X

p

k xk jx j p

p i

i

and

kxk maxjx j

i

in

n n

where x x R Please note that the case p gives the usual Euclidean norm

i

i

n n

on R and that the cases p and p clearly give rise to legitimate norms on R

Common parlance is to refer to these expressions as norms and to refer to the space

p

n n

R kk as The space of all innite sequences x x for which the analogous

p n

p n

Preliminaries

innite sum or supremum kxk is nite is referred to as Whats more there is a

p p

continuous analogue of this scale Wemight also consider the norms

p

Z

b

p

kf k jf xj dx p

p

a

and

kf k sup jf xj

axb

where f is in C a b or is simply Leb esgue integrable The subsequent discussion actually

n

covers all of these cases but we will settle for writing our pro ofs in the R setting only

Lemma Youngs inequality Let p and let q b e dened by

p

that is q Then for any a b wehave

p q p

p q

a b ab

p q

p q

Moreover equality can only o ccur if a b We refer to p and q as conjugate exp onents

q

note that p satises p Please note that the case p q yields the familiar

q

arithmeticgeometric mean inequality

Proof A quick calculation b efore we b egin

p p p

q

p p p

p

Nowwe just estimate areas for this you might nd it helpful to draw the graph of y x

q

or equivalently the graph of x y Comparing areas we get

Z Z

a b

p q p q

ab x dx y dy a b

p q

p q

The case for equality also follows easily from the graph of y x or x y since

p pq p q

b a a means that a b

Corollary Holders inequality Let p and let q b e dened by

Then for any a a and b b in R wehave

n n

p q

p q

n n n

X X X

p q

ja b j ja j jb j

i i i i

i i i

Please note that the case p q yields the familiar CauchySchwarz inequality

Moreover equalityinHolders inequality can only o ccur if there exist nonnegative

p q

jb j for all i n scalars and such that ja j

i i

Preliminaries

P P

p q

n n

p q

Proof Let A ja j and let B jb j Wemay clearly assume

i i

i i

that A B why and hence wemay divide and app eal to Youngs inequality

p q

ja b j ja j jb j

i i i i

p q

AB pA qB

Adding we get

n n n

X X X

p q

ja b j ja j jb j

i i i i

p q

AB pA qB p q

i i i

P

n

That is ja b jAB

i i

i

The case for equalityinHolders inequalityfollows from what weknowaboutYoungs

inequality EqualityinHolders inequality means that either A orB or else

p p q q

ja j pA jb j q B for all i n In short there must exist nonnegative scalars

i i

p q

and such that ja j jb j for all i n

i i

Notice to o that the case p q works and is easy

n n

X X

ja b j ja j max jb j

i i i i

in

i i

Exercise

When do es equality o ccur in the case p q

Finally an application of Holders inequality leads to an easy pro of that kk is actually

p

a norm It will help matters here if werstmake a simple observation If p and

p

if q notice that

p

pp

n

X

p p p n

ja j kak ja j

i i

p i

q

i

n n n

Lemma Minkowskis inequality Let p and let a a b b R

i i

i i

Then ka bk kak kbk

p p p

Proof In order to prove the triangle inequality we once again let q b e dened by

and nowwe use Holders inequality to estimate

p q

n n

X X

p p

ja b j ja b jja b j

i i i i i i

i i

Preliminaries

n n

X X

p p

jb jja b j ja jja b j

i i i i i i

i i

p n p n

k k ky k k ja b j kak k ja b j

q q p i i p i i

i i

p

ka bk kak kb k

p p

p

p p

That is ka bk ka bk kak kbk and the triangle inequality follows

p p

p p

If p then equalityinMinkowskis inequality can only o ccur if a and b

are parallel that is the norm is strictly convex for p Indeed if ka bk

p p

kak kbk then either a orb orelsea b and wehave equalityateach stage of

p p

our pro of Now equality in the rst inequality means that ja b j ja j jb j which easily

i i i i

implies that a and b have the same sign Next equality in our application of Holders

i i

p p

inequality implies that there are nonnegative scalars C and D suchthat ja j C ja b j

i i i

p p

and jb j D ja b j for all i n Thus a Eb for some scalar E and all

i i i i i

i n

Of course the triangle inequality also holds in either of the cases p orp

with much simpler pro ofs

Exercises

When do es equality o ccur in the triangle inequality in the cases p orp In

particular show that neither of the norms kk or kk is strictly convex

App endix B

Next weprovide a brief review of completeness and compactness Suchreviewisdoomed

to inadequacy the reader unfamiliar with these concepts would b e well served to consult

a text on advanced calculus suchas Analysis in Euclidean Spaces by K Homan or

Principles of Mathematical Analysis by W Rudin

n

A of normed space X suchasR or R is said to b e To b egin we recall that a subset

closed if A is closed under the taking of sequential limits That is A is closed if whenever

a is a sequence from A converging to some p oint x X wealways have x A Its not

n

hard to see that anyclosedinterval suchasa b ora is indeed a closed subset of

R in this sense There are however much more complicated examples of closed sets in R

Preliminaries

A normed space X is said to b e complete if every Cauchy sequence from X converges

n

to a p ointinX It is a familiar fact from Calculus that R is complete as is R In fact

the completeness of R is often assumed as an axiom in the form of the least upp er b ound

axiom There are however many examples of normed spaces which are not complete

that is there are examples of normed spaces in which Cauchy sequences need not converge

Wesay that a subset A of a normed space X is complete if every Cauchy sequence

from A converges to a p ointinA Please note here that we require not only that Cauchy

sequences from A converge but also that the limit b e backinAAsyou might imagine

the completeness of A dep ends on prop erties of b oth A and the containing space X

First note that a complete subset is necessarily also closed Indeed since every con

vergent sequence is also Cauchy it follows that a complete subset is closed

Exercise

If A is a complete subset of a normed space X show that A is also closed

If the containing space X is itself complete then its easy to tell whic h of its subsets

are complete Indeed since every Cauchy sequence in X converges somewhere all we

need to know is whether the subset is closed

Exercise

Let A b e a subset of a complete normed space X Show that A is complete if and only if

n

A is a closed subset of X In particular please note that every closed subset of R or R

is complete

Finallywe recall that a subset A of a normed space X is said to b e compact if every

sequence from A has a subsequence which converges to a p ointin A Again since we

have insisted that certain limits remain in A its not hard to see that compact sets are

necessarily also closed

Exercise

If A is a compact subset of a normed space X show that A is also closed

Moreover since a Cauchy sequence with a convergent subsequence must itself converge

Preliminaries

why we actually have that every compact set is necessarily complete

Exercise

If A is a compact subset of a normed space X show that A is also complete

Since the compactness of a subset A has something to do with every sequence in A

its not hard to b elieve that it is a more stringent prop erty than the others weve considered

so far In particular its not hard to see that a compact set must b e b ounded

Exercise

If A is a compact subset of a normed space X show that A is also b ounded Hint If not

then A would contain a sequence a withka k

n n

Now it is generally not so easy to describ e the compact subsets of a particular normed

n

space X however it is quite easy to describ e the compact subsets of R or R This

wellknown result go es bymany names we will refer to it as the HeineBorel theorem

n

Theorem A subset A of R or R is compact if and only if A is b oth closed and

b ounded

Proof One direction of the pro of is easy As weve already seen compact sets in R

are necessarily closed and b ounded For the other direction notice that if A is a b ounded

subset of R then it follows from the BolzanoWeierstrass theorem that every sequence

from A has a subsequence which converges in RIfA is also a closed set then this limit

must in fact b e backinAThus every sequence in A has a subsequence converging to a

p ointinA

App endix C

We next oer a brief review of p ointwise and uniform convergence We b egin with an

elementary example

Example

x

x

a For each n consider the function f xe for x R Note that

n

n

x

converges to f xe b ecause for each xed x the sequence f x

n

n

jxj

jf x f xj as n

n n

Preliminaries

In this case wesay that the sequence of functions f converges p ointwise to the

n

function f on R But notice to o that the rate of convergence dep ends on x In

particular in order to get jf x f x j wewould need to take njxjThus

n

at x the inequality is satised for all n while at x the inequalityis

satised only for n In short the rate of convergence is not uniform in x

b Consider the same sequence of functions as ab ove but now lets supp ose that we

restrict that values of x to the interval Of course we still havethat f x

n

x in in other words we still have that f converges f x for each xed

n

p ointwise to f on But notice that the rate of convergence is now uniform over

x in To see this just rewrite the initial calculation

jxj

jf x f xj for x

n

n n

and notice that the upp er b ound n tends to as n indep endent of the choice

of x In this case wesay that f converges uniformly to f on The p oint

n

here is that the notion of uniform convergence dep ends on the underlying domain as

well as on the sequence of functions at hand

With this example in mind wenow oer formal denitions of p ointwise and uniform

convergence In b oth cases we consider a sequence of functions f X R n

n

each dened on the same underlying set X and another function f X R the candidate

for the limit

Wesay that f converges p ointwise to f on X if for each x X wehave f x

n n

f xasn thus for each x X and each we can nd an integer N which

dep ends on and whichmay also dep end on xsuch that jf x f xj whenever

n

nN A convenient shorthand for p ointwise convergence is f f on X or if X is

n

understo o d simply f f

n

Wesay that f converges uniformly to f on X if for each we can nd

n

such that jf x f xj for each an integer N which dep ends on but not on x

n

x X provided that nN Please notice that the phrase for each x X now o ccurs

well after the phrase for each and in particular that the rate of convergence N

do es not dep end on x It should b e reasonably clear that uniform convergence implies

Preliminaries

p ointwise convergence in other words uniform convergence is stronger than p ointwise

convergence For this reason we sometimes use the shorthand f f on X or if X is

n

understo o d simply f f

n

The denition of uniform convergence can b e simplied by hiding one of the quan

tiers under dierent notation indeed note that the phrase jf x f xj for any

n

x X is essentially equivalent to the phrase sup jf x f xj Thus our

n

xX

denition may b e reworded as follows f converges uniformly to f on X if given

n

jf x f xj for all nN there is an integer N such that sup

n

X x

The notion of uniform convergence exists for one very go o d reason Continuityis

preserved under uniform limits This fact is well worth stating

Exercise

Let X b e a subset of R let f f X R for n andletx X If each f

n n

is continuous at x andiff f on X then f is continuous at x In particular if each

n

f is continuous on all of X then so is f Give an example showing that this result may

n

fail if we only assume that f f on X

n

App endix D

Lastlywe discuss continuity for linear transformations b etween normed vector spaces

Throughout this section we consider a linear map T V W between vector spaces V

and W thatiswe supp ose that T satises T x yT xT y for all x y V

and all scalars Please note that every linear map T satises T If we further

supp ose that V is endowed with the norm kk and that W is endowed with the norm

jjj jj j the wemay consider the issue of continuity of the map T

The key result for our purp oses is that for linear maps continuityeven at a single

p ointis equivalent to uniform continuity and then some

kk and W jjjj jj b e normed vector spaces and let T V W be a Theorem Let V

linear map Then the following are equivalent

i T is Lipschitz

ii T is uniformly continuous

Preliminaries

iii T is continuous everywhere

iv T is continuous at V

v there is a constant C such that jjj T x jjj C kxk for all x V

Proof Clearly i ii iii iv We need to show that iv v and

that v i for example The second of these is easier so lets start there

v i If condition v holds for a linear map T then T is Lipschitz with constant

C since jjj T x T y jjj jjj T x y jjj C kx y k for any x y V

iv v Supp ose that T is continuous at Then wemaycho ose a sothat

jjj T x jjj jjj T x T jjj whenever kxk kx k How

Given x V wemay scale by the factor kxk to get xkxk Hence

xk But T xkxk kxk T x since T is linear and so we get T xk

jjj T x jjj kxk That is C works in condition v Note that since condition

v is trivial for x we only care ab out the case x

A linear map satisfying condition v of the Theorem ie a continuous linear map

is often said to b e b ounded The meaning in this context is slightly dierent than usual

Here it means that T maps b ounded sets to b ounded sets This follows from the fact

that T is Lipschitz Indeed if jjj T x jjj C kxk for all x V then as weveseen

jjj T x T y jjj C kx y k for any x y V and hence T maps the ball ab out x of

radius r into the ball ab out T x of radius CrInsymbols T B x B T x More

r Cr

generally T maps a set of diameter d into a set of diameter at most Cd Theres no danger

of confusion in our using the word b ounded to mean something new here the ordinary

usage of the word as applied to functions is uninteresting for linear maps A nonzero

linear map always has an unb ounded range Why

The smallest constant that works in v is called the norm of the op erator T and is

usually written kT k In symbols

jjj Txjjj

jjj Txjjj sup kT k sup

kxk

kxk x

Thus T is b ounded continuous if and only if kT k

Preliminaries

The fact that all norms on a nitedimensional normed space are equivalentprovides

a nal rather sp ectacular corollary

Corollary Let V and W b e normed vector spaces with V nitedimensional Then every

linear map T V W is continuous

P P

n n

Proof Let x x b e a basis for V and let k x k j j as b efore

n i i i

i i

From the Lemma on page we know that there is a constant B such that kxk

B kxk for every x V

Nowif T V kk W jjjj jj is linear we get

n n

X X

T x T x

i i i i

i i

n

X

j j jjj T x jjj

i i

i

n

X

max jj j T x jjj j j

j i

j n

i

n

X

x B max jjj T x jjj

i i j

j n

i

That is jjj T x jjj C kxk where C B max jjj T x jjj a constant dep ending only on T

j

j n

and the choice of basis for V From our last result T is continuous b ounded

Math Problem Set Function Spaces

Problems marked are essential to a full understanding of the course we will discuss

most of these in class Problems marked are of general interest and are oered as a

contribution to your p ersonal growth Unmarked problems are just for fun

The most imp ortant collection of functions for our purp oses is the space C a b consisting

of all continuous functions f a b R Its easy to see that C a b isavector space

under the usual p ointwise op erations on functions f g xf xg xandf x

f x for R Actuallywe will b e most interested in the nitedimensional subspaces

P of C a b consisting of all algebraic p olynomials of degree at most n

n

The subspace P has dimension exactly n Why

n

consisting of all those f s which Another useful subset of C a b is the collection lip

K

satisfy a Lipschitz condition of order with constantK ie those f s for

which jf x f y jK jx y j for all x y in a b Some authors would say that f is

Holder continuous with exp onent

a Show that lip is indeed a subset of C a b

K

b If show that lip contains only the constant functions

K

p

c Show that x is in lip and that sin x is in lip on

d Show that the collection lip consisting of all those f s which are in lip for

K

some K is a subspace of C a b

e Show that lip contains all the p olynomials

f If f lip for some show that f lip for all

g Given showthat x is in lip on but not in lip for any

We will also want to consider a norm on the vector space C a b wetypically use the

uniform or sup norm Rivlin calls this the Chebyshev norm dened by kf k maxjf xj

axb

Some authors write kf k or kf k

u

Show that P and lip are closed subsets of C a b under the sup norm Is lip

n

K

closed A bit harder Show that lip is b oth rst category and dense in C a b

P

n

n

Fix n and consider the norm kpk ja j for pxa a x a x P

k n n

k

Function Spaces

Show that there are constants A B such that A kpk kpk B kpk

n n n n

where kpk max jpxjDoA and B really dep end on n

n n

axb

We will o ccasionally consider spaces of realvalued functions dened on nite sets that

n

is we will consider R under various norms Why is this the same We dene a scale

P

p

n

n p

of norms on R by kxk jx j where x x x and p we

p i n

i

need p in order for this expression to b e a legitimate norm but the expression makes

p erfect sense for any p and even for pprovided no x is Notice please that

i

n

the usual norm on R is given by kxk

Showthat lim kxk max jx jFor this reason we dene kxk max jx jThus

p i i

p

in in

n

R under the norm kk is the same as C f ng with its usual norm

Assuming x for i n compute lim kxk and lim kxk

i p p

p

p

Consider R under the norm kxk Draw the graph of the unit sphere fx kxk g

p p

for various values of p esp ecially p

Showthat Youngs inequality Let p and let q satisfy

p q

p q p q

ab a b for all a b with equality if and only if a b

p q

Holders inequality Let p and let q satisfy Showthat

p q

P P P

p q

n n n

p q

a ja b j ja j jb j and

i i i i

i i i

q p

R R R

b b b

q p

b jg xj dx jf xj dx jf x g xj dx

a a a

Describ e the case for equality in each inequality What happ ens if p q

Minkowskis inequality For pshow that

P P P

p p p

n n n

p p p

a ja b j ja j jb j and that

i i i i

i i i

p p p

R R R

b b b

p p p

b dx jf xg xj dx jf xj dx jg xj

a a a

Describ e the case for equality in each inequality What happ ens if p

Exercise shows that kk is indeed a norm for pWe write L a b tomean

p p

the vector space of functions on a b for which the integral norm is dened and nite we

n n

write to mean the vector space of sequences of length n that is R supplied with the p

Function Spaces

norm kk and we write to mean the vector space of innite sequences x x

p p n

n

for which kxk In each space the usual algebraic op erations are dened p ointwise

p

or co ordinatewise and the norm is understo o d to b e kk

p

A normed space X kk is said to b e strictly convex if kx y k kxk ky k always

implies that x and y lie in the same direction that is either x y or y x for some

nonnegative scalar Equivalently X is strictly convex if the triangle inequality is always

strict on nonparallel vectors

Prove that the following are equivalent

a X kk is strictly convex

x y kxk ky k

b If x y X are nonparallel then

x y

c If x y X with kxk ky k then

Show that L and are strictly convex for p Show also that this fails in

p p

p

case p Hint This is actually a statement ab out the function jtj p

Strictly convex spaces are of interest when considering the problem of nearest p oints Given

a nonemptysubsetK of a normed space X and a p oint x K we ask whether there is a

b est approximation to x from elements of K that is wewanttoknow if there exist one

or more p oints y K satisfying

kx y k infkx y k distx K

y K

Its not hard to see that a satisfactory answer to this question will require that we take K

to b e a closed set in X for otherwise the p oints in K n K wouldnt have nearest p oints

Less easy to see is that wetypically also want to assume that K is a convex set Recall

that a subset K of a vector space X is said to b e convex if it contains the line segment

joining any pair of its p oints that is K is convex if

x y K x y K

Obviouslyany subspace of X is a convex set and for our purp oses at least this is the

most imp ortant example

Let X b e a normed space and let B fx X kxkgShow that B is a closed

convex set

Function Spaces

Consider R under the norm kk Let B fy R ky k g and let x

Show that there are innitely many p oints in B nearest to x

a Let K ff L f and kf k g Show that K is a closed convex

set in L that K and that every p ointinK is a nearest p ointto

R

f g Again show that b Let K ff C f and K is a closed

convex set in C that K but that no p ointinK is nearest to

Let K be a compact convex set in a strictly convex space X and let x X Show

that x has a unique nearest p oint y K

Let K b e a closed subset of a complete normed space X Prove that K is convex if

and only if K is midp oint convex that is if and only if x y K whenever x

y K Is this result true in more general settings For example can you proveit

without assuming completeness Or for that matter is it true for arbitrary sets in

anyvector space ie without even assuming the presence of a norm

Math Approximation by Algebraic Polynomials

Intro duction

Lets b egin with some notation Throughout were concerned with the problem of b est

uniform approximation of a given function f C a b by elements from P the subspace

n

of algebraic p olynomials of degree at most n in C a b Weknow that the problem has a

solution p ossibly more than one whichwevechosen to write as p Weset

n

k E f minkf pk kf p

n

n

pP

n

Since P P for each n its clear that E f E f for each n Our goal in this

n n n n

chapter is to prove that E f Well accomplish this byproving

n

Theorem The Weierstrass Approximation Theorem Let f C a b Then

for every there is a p olynomial p such that kf pk

It follows from the Weierstrass theorem that p f for each f C a b Why

n

This is an imp ortant rst step in determining the exact nature of E f as a function of

n

f and nWell lo ok for much more precise information in later sections

Now there are many pro ofs of the Weierstrass theorem a mere three are outlined in

the exercises but there are hundreds and all of them start with one simplication The

underlying interval a b is of no consequence

Lemma If the Weierstrass theorem holds for C then it also holds for C a b and

tical They conversely In fact C and C a b are for all practical purp oses iden

are linearly isometric as normed spaces order isomorphic as lattices and isomorphic as

algebras rings

Proof Well settle for proving only the rst assertion the second is outlined in the

exercises and uses a similar argument

Given f C a b notice that the function

g xf a b ax x

Algebraic Polynomials

denes an elementofC Now given supp ose that we can nd a p olynomial p

such that kg pk in other words supp ose that

f a b ax px max

x

Then

t a

f t p max

atb

b a

ta

Why But if px is a p olynomial in x then q tp is a p olynomial in t again

ba

why satisfying kf q k

The pro of of the converse is entirely similar If g x isanelementof C then

ta

a t b denes an elementof C a b Moreover if q t is a p olynomial f tg

ba

in t approximating f t then pxq a b ax is a p olynomial in x approximating

g x The remaining details are left as an exercise

The p oint to our rst result is that it suces to prove the Weierstrass theorem for

anyinterval we like and are p opular choices but it hardly matters which

interval we use

Bernsteins Pro of

The pro of of the Weierstrass theorem wepresent here is due to the great Russian math

ematician S N Bernstein in Bernsteins pro of is of interest to us for a varietyof

reasons p erhaps most imp ortant is that Bernstein actually displays a sequence of p olyno

mials that approximate a given f C Moreover as well see later Bernsteins pro of

generalizes to yield a p owerful unifying theorem called the BohmanKorovkin theorem

If f is any bounded function on we dene the sequence of Bernstein p olynomials

for f by

n

X

k n

k nk

f B f x x x x

n

n k

k

Please note that B f is a p olynomial of degree at most n Also its easy to see that

n

B f f and B f f In general B f xisanaverage of

n n n

the numbers f kn k n Bernsteins theorem states that B f f for each

n

f C Surprisingly the pro of actually only requires that wecheck three easy cases

f x f xx and f xx

Algebraic Polynomials

This and more is the content of the following lemma

Lemma i B f f and B f f

n n

ii B f f f and hence B f f

n n

n n

n

X

k n x x

k nk

iii x if x x x

n n n k

k

iv Given and x letF denote the set of k s in fng for which

X

k n

k nk

x Then x x

n n k

k F

Proof That B f f follows from the binomial formula

n

n

X

n

k nk n

x x x x

k

k

To see that B f f rst notice that for k wehave

n

n k n n

k n k k n k

Consequently

n n

X X

n k n

k nk k nk

x x x x x

k k n

k k

n

X

n

j nj

x x x x

j

j

Next to compute B f we rewrite twice

n

k n k n n n n k

if k

n k n k n n k n k

n n

if k

k k n n

Algebraic Polynomials

Thus

n

X

n k

k nk

x x

n k

k

n n

X X

n n

k nk k nk

x x x x

n k n k

k k

x x

n n

which establishes ii since kB f f k kf f kasn

n

n

To prove iii we combine the results in i and ii and simplify Since kn x

kn xknx we get

n

X

n k

k nk

x x x x x x x

n k n n

k

x x

n n

for x

Finally to prove iv note that kn x for k F and hence

X X

n n k

k nk k nk

x x x x x

k k n

k F k F

n

X

k n

k nk

x x x

n k

k

from iii

n

Nowwere ready for the pro of of Bernsteins theorem

Proof Let f C and let Then since f is uniformly continuous there is

a such that jf x f y j whenever jx y j Nowwe use the previous

n

k nk

lemma to estimate kf B f k First notice that since the numbers x x are

n k

Algebraic Polynomials

nonnegative and sum to wehave

n

X

n k

k nk

jf x B f xj x x f f x

n

n k

k

n

X

n k

k nk

x x f x f

n k

k

n

X

k n

k nk

f x f x x

n k

k

Nowxn to b e sp ecied in a moment and let F denote the set of k s in fng for

which jkn xj Then jf x f knj for k F whilejf x f knjkf k

for k F Thus

f x B f x

n

X X

n n

k nk k nk

x x kf k x x

k k

k F

kF

kf k from iv of the Lemma

n

provided that nkf k

Landaus Pro of

Just b ecause its go o d for us lets give a second pro of of Weierstrasss theorem This one

is due to Landau in First given f C notice that it suces to approximate

f p where p is any p olynomial Why In particular by subtracting the linear function

f xf f wemay supp ose that f f and hence that f outside

That iswemay supp ose that f is dened and uniformly continuous on all of R

Again we will display a sequence of p olynomials that converge uniformly to f this

time we dene

Z

n

L xc f x t t dt

n n

where c is chosen so that

n

Z

n

c t dt

n

Note that by our assumptions on f wemay rewrite this expression as

Z Z

x

n n

L xc f x t t dt c f t t x dt

n n n

x

Algebraic Polynomials

Written this way its clear that L is a p olynomial in x of degree at most n

n

We rst need to estimate c An easy induction argumentwillconvince you that

n

n

t nt and so we get

p

Z Z

n

n

p p

t dt nt dt

n n

p

n In particular for any from which it follows that c

n

Z

p

n n

c t dt n n

n

which is the inequalitywell need

Next let begiven and cho ose such that

jf x f y j whenever jx y j

n

Then since c t and integrates to we get

n

Z

n

c jL x f xj f x t f x t dt

n n

Z

n

c jf x t f xj t dt

n

Z Z

n n

c t dt kf k c t dt

n n

p

n

kf k n

provided that n is suciently large

A third pro of of the Weierstrass theorem due to Leb esgue in is outlined in the

exercises Leb esgues pro of is of particular interest since it inspired Stones version of the

Weierstrass theorem well discuss the StoneWeierstrass theorem a bit later in the course

Before we go on lets stop and make an observation or two While the Bernstein

p olynomials B f oer a convenient and explicit p olynomial approximation to f they

n

are by no means the b est approximations Indeed recall that if f xx and f xx

then B f f f f Clearly the b est approximation to f out of P

n n

n n

should b e f itself whenever n On the other hand since wealways have

E f kf B f k why

n n

Algebraic Polynomials

a detailed understanding of Bernsteins pro of will lend insightinto the general problem of

p olynomial approximation Our next pro ject then is to improve up on our estimate of the

error kf B f k

n

Improved Estimates

To b egin we will need a bit more notation The mo dulus of continuity of a b ounded

function f on the interval a b is dened by

a b sup jf x f y j x y a b jx y j

f f

for any Note that is a measure of the that go es along with in the

f

denition of uniform continuity literallywehave written as a function of

f

Here are a few easy facts ab out the mo dulus of continuity

Exercises

Wealways have jf x f y j jx y j for any x y a b

f

If then

f f

f is uniformly continuous if and only if as

f

If f exists and is b ounded on a b then K for some constant K

f

More generallywesay that f satises a Lipschitz condition of order with constant

K where and Kifjf x f y jK jx y j for all x y We

abbreviate this statementby the symb ols f lip Check that if f lip then

K K

K for all

f

For the time b eing we actually only need one simple fact ab out

f

Lemma Let f b e a b ounded function on a b and let Then n n

f f

for n Consequently for any

f f

Proof Given xy with jx y jn split the interval x y into n pieces eachof

length at most Sp ecicallyifwe set z x k y xnfork n then k

Algebraic Polynomials

jz z j for any k and so

k k

n

X

jf x f y j f z f z

k k

k

n

X

jf z f z j

k k

k

n

f

Thus n n

f f

The second assertion follows from the rst and one of our exercises Given

cho ose an integer n so that n n Then

n n

f f f f

We next rep eat the pro of of Bernsteins theorem making a few minor adjustments

here and there

Theorem For any b ounded function f on wehave

p

kf B f k

n f

n

p

In particular if f C then E f as n

n f

n

Proof We rst do some term juggling

n

X

n k

k nk

x x jf x B f xj f x f

n

n k

k

n

X

n k

k nk

x x f x f

k n

k

n

X

n k

k nk

x x x

f

k n

k

n

X

p

k n

k nk

p

n x x x

f

n k

n

k

n

X

p

k n

k nk

p

n x x x

f

n n k

k

Algebraic Polynomials

p

k

where the third inequality follows from our previous Lemma by taking n x

n

p

and All that remains is to estimate the sum and for this well use Cauchy

n

Schwarz and our earlier observations ab out Bernstein p olynomials Since each of the

n

k nk

x x is nonnegative wehave terms

k

n

X

n k

k nk

x x x

n k

k

n n

X X

k n n

k nk k nk

x x x x x

n k k

k k

p

n n

Finally

p

p p p

jf x B f xj n

n f f

n n n

Examples

If f lip itfollows that kf B f k Kn and hence E f Kn

n n

K

As a particular case of the rst example consider f x x on Then

f lip and so kf B f k n But as Rivlin p oints out see Remark on

n

p of his b o ok kf B f k n Thus we cant hop e to improve on the

n

power of n in this estimate Nevertheless we will see an improvement in our estimate

of E f

n

The BohmanKorovkin Theorem

The real value to us in Bernsteins approach is that the map f B f while providing

n

a simple formula for an approximating p olynomial is also linear and p ositive In other

words

B f g B f B g

n n n

B f B f R

n n

and

B f whenever f

n

T C C is necessarily also continuous As it happ ens anypositive linear map

Algebraic Polynomials

Lemma If T C a b C a b is b oth p ositive and linear then T is continuous

Proof First note that a p ositive linear map is also monotone That is T satises

T f T g whenever f g Why Thus for any f C a b wehave

f f jf j T f Tf T jf j

that is jT f jT jf j But now jf jkf k where denotes the constant function

and so we get

jT f jT jf j kf k T

Thus

kT f kkf kkT k

for any f C a b FinallysinceT is linear it follows that T is Lipschitz with constant

kT k

kT f T g k kT f g k kT kkf g k

Consequently T is continuous

Now p ositive linear maps ab ound in analysis so this is a fortunate turn of events

Whats more Bernsteins theorem generalizes very nicely when placed in this new setting

The following elegant theorem was proved indep endently by Bohman and Korovkin in

roughly

Theorem Let T C C b e a sequence of p ositive linear maps and supp ose

n

that T f f uniformly in each of the three cases

n

f x f xx and f xx

Then T f f uniformly for every f C

n

The pro of of the BohmanKorovkin theorem is essentially identical to the pro of of

Bernsteins theorem except of course wewriteT f in place of B f For full details

n n

see Cheneys b o ok A n Introduction to Approximation Theory Chelsea Rather than

proving the theorem lets settle for a quick application

Algebraic Polynomials

Example

Let f C and for each n let L f b e the p olygonal approximation to f with

n

no des at kn k n That is L f is linear on eachsubinterval k n k n

n

and agrees with f at each of the endp oints L f knf kn Then L f f

n n

uniformly for each f C This is actually an easy calculation all by itself but lets

see why the BohmanKorovkin theorem makes short work of it

That L f is p ositive and linear is nearly obvious that L f f and L f f

n n n

are really easy since in fact L f f for any linear function f We just need to show

n

that L f f But a picture will convince you that the maximum distance b etween

n

L f andf on the interval k n k n is at most

n

k k k

n n n n

That is kf L f kn asn

n

Note that L is a linear pro jection from C onto the subspace of p olygonal

n

functions based on the no des kn k n An easy calculation similar in spirit

to the example ab ove will show that kf L f k n asn for any

n f

f C

Math Problem Set Uniform Approximation byPolynomials

One of our rst tasks will b e to give a constructive pro of of Weierstrasss Theorem stating

that each f C a b is the uniform limit of a sequence of p olynomials As it happ ens the

choice of interval a b is inconsequential If Weierstrasss theorem is true for one then

its true for all

Dene a b by ta tb a for t and dene a transfor

mation T C a b C byT f tf t Prove that T satises

a T f g T f T g and T cf cT f forc R

b T fgT f T g In particular T maps p olynomials to p olynomials

c T f T g if and only if f g

d kT f k kf k

e T is b oth onetoone and onto Moreover T T

The p oint to exercise is that C a b and C are iden tical as vector spaces metric

spaces algebras and lattices For all practical purp oses they are one and the same space

While Bernsteins pro of of the Weierstrass theorem b elow will prove most useful for our

purp oses there are many others two of these in the case of C aresketched b elow

R

n

Landaus pro of For each n and dene I x dx

n

Show that I I asn for any Now given f C with

n n

R

n

f f show that the p olynomial L xI t t x dt f

n n

converges uniformly to f xon as n Hint You may assume that f

outside of To get the result for general f C we simply need to subtract

the linear function f xf f

Leb esgues pro of Given f C rst show that f can b e uniformly approxi

mated byap olygonal function Sp ecicallygiven a p ositiveinteger N dene Lx

x is linear for by the conditions LkN f kN for k NandL

kN x k N show that kf Lk is small provided that N is suciently large

The function Lx can b e written uniquely as a linear combination of the angles

P

N

xjx kN j x kN and x the equation Lx c x can

k N k k

k

P

N

b e solved since the system of equations LkN c kN k N

k k

k

Polynomials

can b e solved uniquely for c c How To nish the pro of we need to show

N

that jxj can b e approximated by p olynomials on anyinterval a b Why

Heres an elementary pro of that there is a sequence of p olynomials P converging

n

uniformly to jxj on

a Dene P recursively by P xP xx P x where P x

n n n n

Clearly each P is a p olynomial

n

p

b Check that P x P x x for x Use Dinis theorem to

n n

p

conclude that P x x on

n

c P x is also a p olynomial and P x jxj on

n n

The result in problem or shows that the p olynomials are dense in C

Using the results in conclude that the p olynomials are also dense in C a b

Howdowe know that there are nonp olynomial elements in C In other words

is it p ossible that every elementof C agrees with some p olynomial on

Let Q b e a sequence of p olynomials of degree m and supp ose that Q converges

n n n

uniformly to f on a b where f is not a p olynomial Show that m

n

or C isaneven function show that f may b e uniformly approxi If f C

mated byeven p olynomials or even trig p olynomials

If f C and if f f show that the sequence of p olynomials

P

n

n

k nk

f kn x x with integer co ecients converges uniformly to f

k

k

where x denotes the greatest integer in x The same trickworks for any f C a b

provided that ab

If p is a p olynomial and prove that there is a p olynomial q with rational

co ecients such that kp q k on Conclude that C is separable

P

n

k

Let x b e a sequence of numbers in such that lim x exists for every

i

i

i

n

n

P

n

f x exists for every f C k Show that lim

i

i

n

n

R

n

x f x dx for each n showthat f Hint If f C and if

R

Using the Weierstrass theorem show that f

The next pro of of the Weierstrass theorem that we consider is quite explicit we actually

Polynomials

display a sequence of p olynomials that converges uniformly to a given f C Given

of Bernstein p olynomials for f by f C we dene the sequence B f

n

n

n

X

k n

k nk

f B f x x x

n

n k

k

Please note that B f is a p olynomial of degree at most n Also its easy to see that

n

B f f and B f f In general B f xisanaverage of

n n n

the numb ers f kn k n Bernsteins theorem states that the sequence B f

n

converges uniformly to f for each f C the pro of is rather simple once wehavea

few facts ab out the Bernstein p olynomials at our disp osal For later reference lets write

f x f xx and f xx

Among other things the following exercise establishes Bernsteins theorem for these three

p olynomials Curiously these few sp ecial cases will imply the general result

i B f f and B f f Hint Use the binomial theorem

n n

ii B f f f and hence B f converges uniformly to f

n n

n n

P

n xx

n

k

k nk

iii x x x if x

k

n n n k

iv Given and x let F denote the set of k s in f ng for which

P

n

k

k nk

x x x Then

k F

n k n

Show that jB f jB jf j and that B f whenever f Conclude that

n n n

kB f kkf k

n

If f is a b ounded function on show that B f x f x at each p ointof

n

continuityof f

Bohman Korovkin Let T b e a sequence of monotone linear op erators on C

n

that is each T is a linear map from C into itself satisfying T f T g

n n n

whenever f g Supp ose also that T f f T f f and T f f

n n n

Prove that T f f for every f C Hint Mimic the pro of of Bernsteins

n

theorem

Find B f for f xx Hint k k k k The same

n

metho d of calculation can b e used to show that B f P whenever f P and

n m m

n m

Polynomials

Let f b e continuously dierentiable on a b and let Show that there is a

p olynomial p such that kf pk and kf p k

Supp ose that f C a b istwice continuously dierentiable and has f Prove

that the b est linear approximation to f on a b isa a x where a f c

c a f af cf ca c and where c is the unique solution to f

f b f ab a

The next several exercises concern the mo dulus of continuity Given a b ounded realvalued

function f dened on some interval I we dene the mo dulus of continuityof f by

f

I sup jf x f y j x y I jx y j

f f

Note for example that if f is uniformly continuous then as Indeed

f

the statement that jf x f y j whenever jx y j is equivalent to the statement

that On the other hand if the graph of f has a jump of magnitude say then

f

for all

f

If f satises the Lipschitz condition jf x f y jK jx y j what can you sayabout

p

x Calculate for g x

f g

If f C a b show that and that as

f f f f

Use this to show that is continuous for Finally show that the mo dulus of

f

continuityof is again

f f

a If x cos where x and if g f cos show that

g

g f

b If g xf ax b for c x d show that c d ac b ad b a

g f

Let f be continuously dierentiable on ShowthatB f converges uniformly

n

to f byshowing that kB f B f k n In order to see

n n f

why this is of interest nd a uniformly convergent sequence of p olynomials whose

derivatives fail to converge uniformly Compare this result with problem

Math Trigonometric Polynomials

Intro duction

A real trigonometric p olynomial or trig p olynomial for short is a function of the form

n

X

a a cos kx b sin kx

k k

k

where a a and b b are real numbers The degree of a trig p olynomial is the

n n

highest frequency o ccurring in any representation of the form thus has degree n

provided that one of a or b is nonzero We will use T to denote the collection of trig

n n n

p olynomials of degree at most nandT to denote the collection of all trig p olynomials

ie the union of the T s

n

It is convenient to take the space of all continuous p erio dic functions on R as the

containing space for T a space wedenoteby C The space C has several equivalent

n

descriptions For one its obvious that C is a subspace of C R the space of all con

tinuous functions on R But we might also consider C as a subspace of C inthe

following way The p erio dic continuous functions on R may b e identied with the set

of functions f C satisfying f f Eachsuch f extends to a p erio dic

elementof C Rinanobvious way and its not hard to see that the condition f f

denes a subspace of C As a third description it is often convenient to identify C

with the collection C T consisting of all the continuous realvalued functions on T where

T is the unit circle in the complex plane C That is we simply maketheidentications

i i

e and f f e

In any case each f C is uniformly continuous and uniformly b ounded on all of Rand

is completely determined by its values on anyinterval of length In particular wemay

and will endow C with the sup norm

kf k max jf xj maxjf xj

x xR

Our goal in this chapter is to prove what is sometimes called Weierstrasss second

theorem also from

Trig Polynomials

Theorem Weierstrasss Second Theorem Let f C Then for every

there exists a trig p olynomial T such that kf T k

Ultimatelywe will give several dierent pro ofs of this theorem Weierstrass gavea

separate pro of of this result in the same pap er containing his theorem on approximation

by algebraic p olynomials but it was later p ointed out by Leb esgue that the two

theorems are in fact equivalent Leb esgues pro of is based on several elementary obser

vations We will outline these elementary facts as exercises with hints supplying a few

pro ofs here and there but leaving full details to the reader

We rst justify the use of the word p olynomial in describing

Lemma cos nx and sinn x sin x can b e written as p olynomials of degree exactly n

in cos x for anyinteger n

x cos x its not Proof Using the recurrence formula cos kx cosk x cosk

hard to see that cos x cos x cos x cos x cos x and cos x cos x

cos x More generallyby induction cos nx is a p olynomial of degree n in cos x

n

with leading co ecient Using this fact and the identitysink x sink x

cos kx sin x along with another easy induction argument it follows that sinn x can

n

b e written as sin x times a p olynomial of degree n in cos x with leading co ecient

k k

Alternatively notice that by writing i sin x cos x wehave

n

X

n

n k nk

cos nx Recos x i sin x Re i sin x cos x

k

k

n

X

n

k nk

cos x cos x

k

k

n

The co ecient of cos x in this expansion is then

n

n

X X

n n

n

k k

k k

n n

All the binomial co ecients together sum to but the even or o dd terms

n

taken separately sum to exactly half this amount since

Trig Polynomials

Similarly

n

X

n

n k nk

sinn x Im cos x i sin x i sin x cos x Im

k

k

n

X

n

k nk

cos x cos x sin x

k

k

k k n

x is where weve written i sin x icos x sin x The co ecient of cos x sin

n

n

X X

n n

n

k k

k k

Corollary Any trig p olynomial may b e written as P cos xQcos xsinx where

P and Q are algebraic p olynomials of degree at most n and n resp ectively If

represents an even function then it can b e written using only cosines

Corollary The collection T consisting of all trig p olynomials is b oth a subspace and

a subring of C that is T is closed under b oth linear combinations and pro ducts In

other words T is a subalgebra of C

Its not hard to see that the pro cedure weve describ ed ab ove can b e reversed that is

each algebraic p olynomial in cos x and sin x can b e written in the form For example

cos x cos x cos x But rather than duplicate our eorts lets use a bit of linear

algebra First the n functions

sin x sin nx g A f cos x cos x cos nx sin x

are linearly indep endent the easiest way to see this is to notice that wemay dene an

inner pro duct on C under which these functions are orthogonal Sp ecically

Z Z

hf gi f x g x dx hf f i f x dx

for any pair of functions f g AWell pursue this direction in greater detail later in

the course Second weve shown that eachelementof A lives in the space spanned by

the n functions

n n

x sin x g B f cos x cos x cos x sin x cos x sin x cos

Trig Polynomials

That is

T span A span B

n

By comparing dimensions wehave

n dimT dimspan A dimspan B n

n

and hence wemust have span A span B Thepoint here is that T is a nitedimensional

n

subspace of C of dimension n and wemay use either one of these sets of functions

as a basis for T

n

Before we leave these issues b ehind lets summarize the situation for complex trig

p olynomials ie the case where weallow complex co ecients in Now its clear that

every trig p olynomial whether real or complex can b e written as

n

X

ik x

c e

k

k n

where the c s are complex that is a trig p olynomial is actually a p olynomial over C in

k

ix ix

z e andz e Converselyevery p olynomial can b e written in the form

using complex a s and b s Thus the complex trig p olynomials of degree n form a vector

k k

space of dimension n over C hence of dimension n when considered as a vector

space over R But not every p olynomial in z andz represents a real trig p olynomial

Rather the real trig p olynomials are the real parts of the complex trig p olynomials To

see this notice that represents a realvalued function if and only if

n n n

X X X

ik x ik x

ik x

c e c e c e

k k k

k n k n k n

that is c c for each k In particular c must b e real and hence

k k

n n

X X

ik x ik x ik x

c e c e c e c

k k k

k k n

n

X

ik x ik x

c c e c e

k k

k

n

X

c cos kx ic c sinkx c c

k k k k

k

n

X

c Rec cos kx Imc sinkx

k k

k

Trig Polynomials

whichisoftheform with a and b real

k k

Conversely given any real trig p olynomial wehave

n n

X X

a ib a ib

k k k k

ik x ik x

a cos kx b sin kx a a e e

k k

k k

which of of the form withc c for each k

k k

Its time we returned to approximation theorySince w evebeenabletoidentify C

with a subspace of C and since T is a nitedimensional subspace of C wehave

n

Corollary Each f C has a b est approximation on all of R out of T Iff is an

n

even function then it has a b est approximation which is also even

Proof We only need to prove the second claim so supp ose that f C is even and

that T T satises

n

kf T k kf T k min

T T

n

e

Then since f is even T xT x is also a b est approximation to f out of T indeed

n

e

kf T k maxjf x T xj

xR

maxjf x T xj

xR

maxjf x T xj kf T k

xR

But now the even trig p olynomial

e

T xT x T xT x

b

T x

is also a b est approximation out of T since

n

e e

f T f T kf T k kf T k

b

kf T k min kf T k

T T

n

We next give de la Vallee Poussins version of Leb esgues pro of of Weierstrasss

second theorem that is we will deduce the second theorem from the rst

Trig Polynomials

Theorem Let f C and let Then there is a trig p olynomial T suchthat

kf T k max jf x T xj

xR

Proof We will prove that Weierstrasss rst theorem for C implies his second

theorem for C

Step Iff is even then f may b e uniformly approximated byeven trig p olynomials

If f is even then its enough to approximate f on the interval In this case we

may consider the function g y f arccos y y in C By Weierstrasss

rst theorem there is an algebraic p olynomial py such that

max jf arccos y py j maxjf x pcos xj

y x

But T xpcos xisaneven trig p olynomialHence

kf T k maxjf x T xj

xR

Lets agree to abbreviate kf T k as f T

f x sin x T x Step Given f C there is a trig p olynomial T such that

Each of the functions f xf x and f x f x sin x is evenThus wemay

cho ose even trig p olynomials T and T such that

f xf x T x and f x f x sin x T x

Multiplying the rst expression bysin x the second bysinx and adding we get

f x sin x T xsin x T xsin x T x

where T x is still a trig p olynomial and where now means within since

j sin x j

Step Given f C there is a trig p olynomial T such that f x cos x T x where

means within

Rep eat Step for f x and translate Werstcho ose a trig p olynomial T x

such that

f x sin x T x

Trig Polynomials

That is

f x cos x T x

where T x is a trig p olynomial

Finallybycombining the conclusions of Steps and we nd that there is a trig

p olynomial T x such that f T x where again means within

Just for fun lets complete the circle and show that Weierstrasss second theorem

for C implies his rst theorem for C Since as well see its p ossible to givean

indep endent pro of of the second theorem this is a meaningful exercise

Theorem Given f C and there exists an algebraic p olynomial p such

that kf pk

Proof Given f C the function f cos xisaneven function in C Byour

Corollary to Weierstrasss second theorem wemay approximate f cos xbyaneven trig

p olynomial

f cos x a a cos x a cos x a cos nx

n

But as weve seen cos kx can b e written as an algebraic p olynomial in cos x Hence there

is some algebraic p olynomial p such that f cos x pcos x That is

max jf cos x pcos xj max jf t ptj

x t

The algebraic p olynomials T x satisfying

n

T cos x cos nx for n

n

are called the Chebyshev p olynomials of the rst kind Please note that this formula

uniquely denes T as a p olynomial of degree exactly n and hence uniquely determines

n

the values of T xforjxj to o The algebraic p olynomials U x satisfying

n n

sinn x

U cos x for n

n

sin x

are called the Chebyshev p olynomials of the second kindLikewise note that this formula

uniquely denes U as a p olynomial of degree exactly n n

Trig Polynomials

We will discover manyintriguing prop erties of the Chebyshev p olynomials in the next

chapter For now lets settle for just one The recurrence formula wegave earlier

cos nx cos x cosn x cosn x

now b ecomes

T xxT x T x n

n n n

where T x and T xx This recurrence relation along with the initial cases T

and T maybetaken as a denition for the Chebyshev p olynomials of the rst kind At

any rate its now easy to list anynumb er of the Chebyshev p olynomials T for example

n

T xx x T xx x and T x the next few are T xx

x x x

Math Problem Set Trigonometric Polynomials

A real trigonometric p olynomial or trig p olynomial for short is a function of the form

n

X

a a cos kx b sin kx

k k

k

where a a and b b are real numbers WewilluseT to denote the collection

n n n

of trig p olynomials of degree at most n considered as a subspace C the space of all

continuous p erio dic functions on R The space C may in turn b e considered as a

subspace of C Indeed the p erio dic continuous functions on R may b e identied

with the subspace of C consisting of those f s which satisfy f f As an

alternate description it is often convenient to instead iden tify C with the collection

C T consisting of all continuous realvalued functions on T where T is the unit circle in

the complex plane C In this case we simply make the identications

i i

e and f f e

a By using the recurrence formulas cos kx cosk x cosk x cos x and

sink x sink x cos kx sin xshow that each of the functions cos kx

and sink x sin x may b e written as algebraic p olynomials of degree exactly

k

k in cos xIneach case what is the co ecient of cos x

b Equivalently use the binomial formula to write the real and imaginary parts of

n

cos x i sin x cos nx i sin nx as algebraic p olynomials in cos x and sin x

Again what are the leading co ecients of these p olynomials

c If P x y is an algebraic p olynomial in twovariables of degree at most n show

that P cos x sin xmay b e written as Qcos xRcos xsinxwhereQ and

R are algebraic p olynomials in one variable of degrees at most n and n

resp ectively

n

d Show that cos x can b e written as a linear combination of the functions cos kx

n

k n and that cos x sin x can b e written as a linear combinations of

the functions sin kx k nThus each p olynomial P cos x sin x in cos x

and sin x can b e written in the form

Trigonometric Polynomials

e If represents an even function show that it can b e written using only cosines

ConverselyifP x y isaneven p olynomial show that P cos x sin x can b e

written using only cosines

Showthat T has dimension exactly n as a vector space over R

n

We might also consider complex trig p olynomials that is functions of the form in

whichwenow allow the a s and b s to b e complex numb ers

k k

a Show that every trig p olynomial whether real or complex may b e written as

n

X

ik x

c e

k

k n

where the c s are complex Thus complex trig p olynomials are just algebraic

k

ix

p olynomials in z andz where z e T

b Show that is realvalued if and only ifc c for any k

k k

c If is a realvalued function show that it may b e written as a real trig p oly

nomial that is it may b e written in the form using only real co ecients

Math Characterization of Best Approximation

We next discuss Chebyshevs solution to the problem of b est p olynomial approximation

from Given that there was no reason to b elieve that the problem even had a solution

let alone a unique solution Chebyshevs accomplishment should not b e underestimated

Chebyshev mightvery well havebeenabletoproveWeierstrasss result years early

had the thought simply o ccurred to him Chebyshevs original pap ers are apparently

rather sketchyItwasnt until that full details were given byKirchb erger Curiously

Kirchb ergers pro ofs foreshadowvery mo dern techniques such as convexity and separation

arguments The presentation well giveowes much to Haar and to de la Vallee Poussin

b oth from around

We b egin with an easy observation

Lemma Let f C a b and let p p b e a b est approximation to f out of P Then

n

n

there are at least two distinct p oints x x a b suchthat

f x px f x px kf pk

That is f p attains b oth of the values kf pk

Proof Lets write E E f kf pk maxjf x pxj If the conclusion of the

n

axb

but Lemma is false then wemightaswell supp ose that f x px E for some x

that

e min f x px E

axb

In particular E e and so q p E eisanelementof P with q pWe

n

claim that q is a b etter approximation to f than p Heres why

E e E e E e

E f x px e

or

E e E e

f x q x

That is

E e

E kf pk kf q k

Best Approximation

a contradiction

Corollary The b est approximating constantto f C a b is

p max f x min f x

axb axb

and

E f max f x min f x

axb axb

Proof Exercise

Now all of this is meantasmotivation for the general case whichessentially rep eats

the observation of our rst Lemma inductively A little exp erimentation will convince you

that a b est linear approximation for example would imply the existence of three p oints

at least at which f p alternates b etween kf p k

A bit of notation will help us set up the argument for the general case Given g in

C a b well saythat x a b is a point for g resp ectivelya p oint for g if

kg k resp ectively g xkg k A set of distinct p oint a x x x b g x

n

will b e called an alternating set for g if the x s are alternately p oints and p oints

i

that is if

jg x j kg k i n

i

and

g x g x i n

i i

Using this notation we will b e able to characterize the p olynomial of b est approximation

Since the following three theorems are particularly imp ortant wewillnumb er them for

future reference Our rst result is where all the ghting takes place

Theorem Let f C a b and supp ose that p is a b est approximation to f out p

n

of P Then there is an alternating set for f p consisting of at least n p oints

n

Proof If f P theres nothing to show Why Thus wemay supp ose that fP

n n

and hence that E E f kf pk n

Best Approximation

Now consider the uniformly continuous function f pWemay partition a b

bywayof a t t t b into suciently small intervals so that

n

jx y j E whenever x y t t

i i

Heres whywed wanttodosuch a thing If t t contains a p ointfor f p

i i

then is p ositive on all of t t Indeed

i i

and xE y E x y t t

i i

Similarlyift t contains a p oint for then is negative on all of t t

i i i i

Consequently no interval t t can contain b oth p oints and p oints

i i

Call t t a interval resp ectivelya interval if it contains a p oint

i i

resp ectivelya p oint for f p Notice that no in terval can even toucha

interval In other words a interval and a interval must b e strictly separated

by some interval containing a zero for

Wenow relab el the and intervals from left to right ignoring the neither

intervals Theres no harm in supp osing that the rst signed interval is a interval

Thus we supp ose that our relab eled intervals are written

I I I intervals

k

I I I intervals

k k k

m

I I I intervals

k k k

m m

where I is the last interval b efore we reach the rst interval I And so on

k k

For later reference we let S denote the union of all the signed intervals t t

i i

S

m

that is S I and weletN denote the union of all the neither intervals t t

k i i

j

j

Thus S and N are compact sets with S N a b note that while S and N arent

quite disjoint they are at least nonoverlappingtheir interiors are disjoint

Our goal here is to show that m n So far weonlyknowthat m Lets

supp ose that mn and see what go es wrong

Best Approximation

Since any interval is strictly separated from anyinterval we can nd p oints

z z N such that

m

z min I max I

k k

max I z min I

k k

max I z min I

k m k

m m

And nowwe construct the oending p olynomial

q xz xz x z x

m

Notice that q P since m n Here is the only use well make of the assumption

n

mn Were going to showthat p q P is a b etter approximation to f than p

n

for some suitable scalar

We rst claim that q and f p have the same sign Indeed q has no zeros in any

of the intervals hence is of constantsignonanysuchinterval Thus qon

I I b ecause eachz x on these intervals qonI I b ecause

k j k k

here z x while z x for j and so on

j

We next nd Let e maxjf x pxj where N is the union of all the subin

xN

tervals t t which are neither intervals nor intervals Then eE Why

i i

Nowcho ose so that kq k minfE e E g We claim that p q is a b etter

approximation to f than p One case is easy If x N then

q xj e kq k E jf x pxq xj jf x pxj j

On the other hand if x N then x is in either a interval or a interval In

particular weknow that jf x pxj E kq k and that f x px and q xhave

the same sign Thus

jf x pxq xj jf x pxjjq xj

E min jq xj E

xS

Best Approximation

since q is nonzero on S This contradiction nishes the pro of Phew

Remarks

It should b e p ointed out that the number n here is actually dim P

n

Notice to o that if f p alternates in sign n times then f p must haveat

n n

least n zeros Thus p actually agrees with f or interp olates f at n p oints

n

Were now ready to establish the uniqueness of the p olynomial of b est approximation

Theorem Let f C a b Then the p olynomial of b est approximation to f out of

P is unique

n

Proof Supp ose that p q P b oth satisfy kf pk kf q k E f E Then

n n

P is also b est kf r k E since f r as weve seen their average r p q

n

f pf q

By Theorem f r has an alternating set x x x containing n p oints

n

Thus for each i

f px f q x E alternating

i i

while

E f px f q x E

i i

But this means that

f px f q x E alternating

i i

for each i Why That is x x x is an alternating set for b oth f p and f q

n

In particular the p olynomial q p f p f q has n zerosSince q p P

n

wemust have p q

Finallywe come full circle

Theorem Let f C a b and let p P Iff p has an alternating set containing

n

n or more p oints then p is the b est approximation to f out of P n

Best Approximation

Proof Let x x x b e an alternating set for f p and supp ose that some q P

n n

is a b etter approximation to f than p that is kf q k kf pk In particular then we

must have

jf x px j kf pk kf q kjf x q x j

i i i i

a b have the for each i n Now the inequality jaj jbj implies that a and

same sign why hence q p f p f q alternates in sign n times b ecause

f p do es But then q p would have at least n zeros Since q p P wemust have

n

q p which is a contradiction Thus p is the b est approximation to f out of P

n

Example taken from Rivlin

While an alternating set for f p is supp osed to have at least n p ointsitmaywell

n

have more than n p oints thus alternating sets need not b e unique For example

consider the function f x sin x on Since there are p oints where f alternates

between it follows that p and that there are dierent alternating

sets consisting of exactly p oints not to mention all those with more than p oints In

addition notice that we actually have p p but that p Why

Exercise

Show that y x is the b est linear approximation to y x on

f Essentially rep eating the pro of given for Theorem yields a lower b ound for E

n

Theorem Let f C a b and supp ose that q P is such that f x q x alternates

n i i

in sign at n p oints a x x x b Then

n

E f min jf x q x j

n i i

in

Proof If the inequality fails then the b est approximation p p would satisfy

n

max jf x px jE f min jf x q x j

i i n i i

in in

Nowwe could rep eat essentially the same argumentusedintheproofofTheoremto

arrive at a contradiction The details are left as an exercise

Best Approximation

Even for relatively simple functions the problem of actually nding the p olynomial

of b est approximation is genuinely dicult even computationally We end this section

by stating two imp ortant problems that Chebyshev was able to solve

Problem

Find the p olynomial p P of degree at most n that b est approximates

n n

n

f xx on the interval This particular choice of interval makes for a tidy

solution well discuss the general situation later

n

Since p is to minimize max jx p xj our rst problem is equivalentto

n n

jxj

Problem

Find the monic p olynomial of degree n which deviates least from on In other

words nd the monic p olynomial of degree n which has smallest norm in C

Well givetwo solutions to this problem whichwe know has a unique solution of

course First lets simplify our notation Wewrite

n

pxx p x the solution

n

and

n

M kpk E x

n

All we know ab out p is that it has an alternating set x x x

n

containing n n p oints that is jpx j M and px px for all i

i i i

Using this tiny bit of information Cheb yshev was led to compare the p olynomials p and

p Watch closely

Step Atany x in wemust have p x b ecause px is a relative extreme

i i i

value for p But p is a p olynomial of degree n and so can have at most n zeros

Thus wemust have

x and p x for i n

i i

in fact x x are all the zeros of p and

n

x p x x p x

n n

Best Approximation

Step Now consider the p olynomial M p P Weknow that M px

n i

for i n and that M p on Thus x x must b e double

n

ro ots at least of M p But this makes for n n ro ots alreadysowe

must have them all Hence x x are double ro ots x and x are simple ro ots and

n n

these are all the ro ots of M p

We know that p has a double ro ot at each Step Next consider p P

n

of x x and no other ro ots hence x p x has a double ro ot at each

n

x x and simple ro ots at x and x Since x p x P weve found all

n n n

of its ro ots

Heres the p oint to all this ro oting

Step Since M px and x p x are p olynomials of the same degree with

the same ro ots they are up to a constantmultiple the same p olynomialIts easy to see

what constant to o The leading co ecientof p is while the leading co ecientof p is

n thus

x p x

M px

n

After tidying up

p x n

p p

x

M px

We really should haveanextra here but weknow that p is p ositiveonsome interval

well simply assume that its p ositiveonx Now up on integrating

px

arccos n arccos x C

M

or

pxM cosn arccos x C

But p M b ecause p so

cosn C C m with n m odd

pxM cosn arccos x

pcos xM cos nx

Lo ok familiar Since weknow that cos nx is a p olynomial of degree n with leading co e

n

cient the nth Chebyshev p olynomial T the solution to our problem must b e

n

n

px T x n

Best Approximation

n

Since jT xjforjxjwhy the minimum norm is M

n

Next we give a fancy solution based on our characterization of b est approximations

Theorem and a few simple prop erties of the Chebyshev p olynomials

n n

Theorem For any n the formula pxx T x denes a p olynomial

n

p P satisfying

n

n n n

max jx pxj max jx q xj

jxj jxj

for any other q P

n

n

Proof We know that T x has leading co ecientandsop P No wset

n n

x cosn k n for k n Then x x x and

k n

nk

T x T cosn k n cosn k

n k n

Since jT xj jT cos j j cos n j for x weve found an alternating set

n n

for T containing n points

n

n n n n

In other words x px T x satises jx pxj and for each

n

n n nk n

k n has x px T x Byourcharacterization

k n k

k

n

of b est approximations Theorem p must b e the b est approximation to x out of

P

n

Corollary The monic p olynomial of degree exactly n having smallest norm in C a b is

n

x b a b a

T

n

n n

b a

Proof Exercise Hint If px is a p olynomial of degree n with leading co ecient

thenp xpx b ab a is a p olynomial of degree n with leading co ecient

n n

b a Moreover max jpxj max jpxj

axb x

Prop erties of the Chebyshev Polynomials

As weve seen the Chebyshev p olynomial T x is the unique real p olynomial of degree

n

n

if n n having leading co ecientifn and such that T cos cosn n

Best Approximation

for all The Chebyshev p olynomials have dozens of interesting prop erties and satisfy all

sorts of curious equations Well catalogue just a few

C T xxT x T x for n

n n n

Proof It follows from the trig identity cos n cos cosn cosn that

T cos cosT cos T cos for all that is the equation T x

n n n n

x But since b oth sides are p olynomials xT x T x holds for all

n n

equalitymust hold for all x

The next two prop erties are proved in essentially the same way

C T xT x T xT x for m n

m n mn mn

C T T x T x

m n mn

p p

n n

x x x C T x x

n

Proof First notice that the expression on the righthand side is actually a p olynomial

p p

n n

x and x x the since on combining the binomial expansions of x

p

odd powers of x cancel Next for x cos

in in

e e T xT cos cosn

n n

n n

cos i sin cos i sin

p p

n n

x x x i x i

p p

n n

x x x x

Weve shown that these two p olynomials agree for jxj hence they must agree for all x

real or complex for that matter

p p

n n

x For real x with jxj the expression x x x equals

coshn cosh x In other words

C T cosh x cosh nx for all real x

n

The next prop erty also follows from prop erty C

Best Approximation

p

n

C T x jxj x for jxj

n

n

An approach similar to the pro of of prop erty C allows us to write x in terms of the

Chebyshev p olynomials T T T

n

n

X

n

n n

T x for n even T should b e replaced by T C For n o dd x

nk

k

k

Proof For x

n n n n i i n

x cos e e

n n

in in in

e e e

n n

in in in

e e e

n n

n n

cos n cosn cosn

n n

T x T x T x

n n n

n

where if n is even the last term in this last sum is T since the central term in the

n

n n

binomial expansion namely T isnt doubled in this case

n n

n

cosk n k n Theyre real simple and C The zeros of T are x

n

k

lie in the op en interval

Proof Just checkBut notice please that the zeros are listed here in decreasing order

b ecause cosine decreases

C Between two consecutive zeros of T there is precisely one ro ot of T

n n

Proof Its not hard to check that

k k k

n n n

n n n

for k n which means that x x x

k k k

C T and T have no common zeros

n n

Best Approximation

Proof Although this is immediate from prop erty C theres another way to see it

T x T x implies that T x by prop erty C Rep eating this

n n n

observation wewould have T x for every kn including k No go o d

k

T x has no zeros

n

k n n g is dense in C The set fx

k

Proof Since cos x is strictly monotone on its enough to knowthattheset

fk ng is dense in and for this its enough to know that fk ng

kn kn

is dense in Why But

k k k

n n n n

for n large that is the set fk ng is dense among the rationals in

kn

n

g can b e esti Its interesting to note here that the distribution of the ro ots fx

kn

k

mated see Natanson Constructive Function TheoryVol I pp For large n the

number of roots of T that lie in an interval x x x is approximately

n

nx

p

x

In particular for n large the ro ots of T are thickest near the endp oints

n

In probabilistic terms this means that if we assign equal probability to each of the

n n

ro ots x x that is if we think of each ro ot as the p osition of a p oint with mass

n

n then the density of this probability distribution or the density of the system of p oint

p

masses at a p oint x is approximately x for large n In still other words this

tells us that the probability that a ro ot of T lies in the interval a b is approximately

n

Z

b

p

dx

x

a

C The Chebyshev p olynomials are mutually orthogonal relativetotheweight w x

x on

Proof For m n the substitution x cos yields

Z Z

dx

p

T x T x cos m cos n d

n m

x

Best Approximation

while for m n weget

Z Z

dx

if n

p

cos n d x T

n

if n

x

C jT xjn for x and jT j n

n n

Proof For xwehave

d

T cos

d n sin n

n

d

T x

n

d

dx sin

cos

d

Thus jT xjn b ecause j sin n jnj sin j which can b e easily checked by induction

n

for example At x weinterpret this derivative formula as a limit as and

j n and nd that jT

n

As well see later each p P satises jp xjkpkn kpk T for x

n

n

and this is of course b est p ossible As it happ ens T x has the largest p ossible rate of

n

growth outside of among all p olynomials of degree n Sp ecically

Theorem Let p P and let kpk max jpxj Then for any x with jx j and

n

x

any k n wehave

k k

jp x jk pkjT x j

n

k

where p is the k th derivativeof p

Well prove only the case k In other words well check that jpx jkpkjT x j

n

The more general case is in Rivlin Theorem p

Proof Since all the zeros of T liein we know that T x Thus wemay

n n

consider the p olynomial

px

q x T x px P

n n

T x

n

If the claim is false then

px

kpk

T x

n

Best Approximation

k

Nowateach of the p oints y coskn k nwehave T y and

k n k

hence

px

k

q y py

k k

T x

n

Since jpy jkpkitfollows that q alternates in sign at these n points In particular

k

q must have at least n zeros in But q x by design and jx j That is

weve found n zeros for a p olynomial of degree n So q that is

px

px T x

n

T x

n

But then

px

jpj kpk

T x

n

since T T cos which is a contradiction

n n

Corollary Let p P and let kpk max jpxj Then for any x with jx jwe

n

x

have

q

n

jpx jkpk jx j x

Rivlins pro of of our last Theorem in the general case uses the following observation

k

C For x and k nwehave T x

n

k

is never zero for Proof Exercise Hint It follows from Rolles theorem that T

n

k

x Why Now just compute T

n

Uniform Approximation byTrig Polynomials

We end this section by summarizing without pro ofs the analogues of Theorems

for uniform approximation by trig p olynomials Throughout f C and T denotes the

n

collection of trig p olynomials of degree at most n

f has a b est approximation T T

n

n or more p oints in Note here f T has an alternating set containing

that n dimT

n

T is unique

Best Approximation

If T T is such that f T has an alternating set containing n or more p oints

n

in then T T

The pro ofs of are very similar to the corresp onding results for algebraic p olyno

mials As you might imagine is where all the ghting takes place and there are a few

technical diculties to cop e with Nevertheless well swallow these facts whole and apply

them with a clear conscience to a few examples

Example

For mn the b est approximation to f xA cos mx B sin mx out of T is

n

Proof Wemay write f xR cos mx x for some R and x How Nowweneed

only display a suciently large alternating set for f in some interv al of length

k

Setting x x km k mwe get f x R cos k R and

k k

x x x Since m n it follows that m n

k

Example

The b est approximation to

n

X

f xa a cos kx b sin kx

k k

k

out of T is

n

n

X

T xa a cos kx b sin kx

k k

k

q

in C b a and kf T k

n n

Proof By our last example the b est approximation to f T out of T is hence T

n

must b e the b est approximation to f Why The last assertion is easy to check Since

p

we can always write A cos mx B sin mx A B cos mx x for some x it

q

b follows that kf T k a

n n

Finally lets make a simple connection b etween the twotyp es of p olynomial approxi mation

Best Approximation

Theorem Let f C and dene C by f cos Then

T

E f min kf pk mink T kE

n

n

pP T T

n n

P

n

k

a x is the b est approximation to f out of P Proof Supp ose that p x

k n

k

b

Then T p cos isinT and clearly

n

max jf x p xj max jf cos p cos j

x

b

Thus kf p k k T k min k T k

T T

n

On the other hand since is evenweknow that T its b est approximation out of T

n

is also even Thus T q cos for some algebraic p olynomial q P Consequently

n

k T k kf q k min kf pk

pP

n

Remarks

Once we know that min kf pk mink T kitfollows that wemust also have

pP T T

n n

T p cos

Each even C corresp onds to an f C by f xarccos x and of

course the conclusions of the Theorem and of Remark hold in this case to o

Whenever we sp eak of even trig p olynomials the Chebyshev p olynomials are lurking

somewhere in the background Indeed let T beaneven trig p olynomial write

x cos as usual and consider the following cryptic equation

n n

X X

a a cos k T T cos pcos

k k k

k k

P

n

where px a T x P

k k n

k

Math Problem Set Chebyshev Polynomials

Weve shown that cos n and sinn sin can b e written as algebraic p olynomials

of degree n in cos we use this observation to dene the Chebyshev p olynomialsThe

Chebyshev p olynomials of the rst kind T x are dened by T cos cosn for

n n

n while the Chebyshev p olynomials of the second kind U x are dened

n

by U cos sinn sin for n

n

Establish the following prop erties of T x

n

xx and T xxT x T xforn i T x T

n n n

n

ii T x is a p olynomial of degree n having leading co ecient for n and

n

containing only even resp o dd p owers of x if n is even resp o dd

iii jT xj for x when do es equality o ccur Where are the zeros of

n

T x Show that b etween two consecutive zeros of T x there is exactly one

n n

zero of T x Can T x and T xhave a common zero

n n n

j n xjn for x and jT iv jT

n n

v T xT x T xT x for mn

m n mn mn

vi T T x T x

m n mn

Z

dx

p

vii Evaluate T x T x

n m

x

viii Show that T is a solution to x y xy n y

n

p p

n n

ix T x x x x x for any x real or complex

n

P P

t cos

n in n

for t that is x Re t e t cos n

n n

t cos t

P

tx

n

t T x this is a generating function for T its closely

n n

n

tx t

related to the Poisson kernel

xi Find analogues of ix if p ossible for U x

n

Show that every p P has a unique representation as p a a T a T

n n n

n

Find this representation in the case pxx

Chebyshev Polynomials

The p olynomial of degree n having leading co ecient and deviating least from on

n

is given by T x On an arbitrary interval a b wewould instead take

n

n

b a x b a

T

n

n

b a

Is this solution unique Explain

If p is a p olynomial on a b of degree n having leading co ecient a then

n

n n

kpk a b a Ifb a then no p olynomial of degree exactly n with

n

integer co ecients can satisfy kpk compare this with problem on the Uniform

Approximation byPolynomials problem set

Given p P showthat jpxjkpkjT xj for jxj

n n

If p P with kpk on and if j px j at n distinct p oint x x

n i n

in show that either p or else p T Hint One approachisto

n

compare the p olynomials p and x p

k k

Compute T for k n where T is the k th derivativeof T For x

n n

n

k

and k nshow that T x n

Math Examples Chebyshev Polynomials in Practice

The following examples are cribb ed from the b o ok Chebyshev PolynomialsbyLFoxand

I B Parker Oxford University Press

As weve seen the Chebyshev p olynomals can b e generated by a recurrence relation By

n

reversing the pro cedure we could solveforx in terms of T T T well do this

n

calculation in class Here are the rst few terms in each of these relations

T x T x

T xx x T x

T xx x T xT x

T xx x x T xT x

T xx x x T xT xT x

T xx x x x T xT xT x

Note the separation of even and o dd terms in each case Writing ordinary garden variety

p olynomials in their equivalent Chebyshev form has some distinct advantages for numerical

computations Heres why

x x x T x T xT x T x T x x

after some simplication Nowwe see at once that we can get a cubic approximation to

x x x x on with error at most by simply dropping the T term on

the righthand side since jT xj whereas simply using x x x as our cubic

approximation could cause an error as big as PrettyslickThis gimmic k of truncating

the equivalentChebyshev form is called economization

As a second example we note that a p olynomial with small norm on mayhave

annoyingly large co ecients

x x x x x x

x x x x x

Chebyshev Polynomials in Practice

but in Chebyshev form lo ok out

T x T x T x T x x

T x T x T x T x

T x T T x

x

The largest co ecientisnow only ab out and the omission of the last three terms

pro duces a maximum error of ab out Not bad

P

n

x k n

As a last example consider the Taylor p olynomial e x k x e n

k

with remainder where x Taking n the truncated series has error no

greater than e But if we economize the rst six terms then

X

k

x k T x T x T x T x

k

T x T x T x

The initial approximation already has an error of ab out so we can certainly drop

the T term without any additional error Even dropping the T term causes an error of

no more than or thereab outs The resulting approximation has a far smaller error

than the corresp onding truncated Taylor series e

The approach used in our last example has the decided disadvantage that wemust rst

decide where to truncate the Taylor serieswhichmight converge very slowly A b etter

x

approachwould b e to write e as a series involving Chebyshev p olynomials directlyThat

P

x

is if p ossible wewanttowrite e a T x If the a s are absolutely summable

k k k

k

it will b e very easy to estimate any truncation error Well get some idea on howtogo

ab out this when we talk ab out leastsquares approximation As it happ ens such a series

is easy to nd its rather likeaFourier series and its partial sums are remarkably go o d

uniform approximations

Math A Brief Intro duction to Interp olation

Our goal in this section is to prove the following result as well as discuss its ramications

In fact this result is so fundamental that we will present three pro ofs

Theorem Let x x x b e distinct p oints and let y y y b e arbitrary p oints

n n

in R Then there exists a unique p olynomial p P satisfying px y i n

n i i

First notice that uniqueness is obvious Indeed if two p olynomials p q P agree at

n

n p oints then p q Why The real work comes in proving existence

First Proof Vandermondes determinant We seek c c c so that px

n

P

n

k

c x satises

k

k

n

X

k

y i n c x px

i k i

i

k

That is we need to solve a system of n linear equations for the c s In matrix form

i

n

x x x

y

c

n

x x x

y

c

n

c

y

x x x

n

n

n

n n

This equation always has a unique solution b ecause the co ecient matrix has determinant

Y

D x x

i j

jin

D is called Vandermondes determinant note that Difx x x Since

n

this fact is of indep endentinterest well sketch a short pro of b elow

Q

Lemma D x x

i j

jin

Proof Consider

n

x x x

n

x x x

V x x x x

n

n

x x x

n

n n

n

x x x

Interp olation

V x x x xisap olynomial of degree n in x and its whenever x x i

n i

Q

n

n Thus V x xc x x by comparing ro ots and degree

i

i

n

However its easy to see that the co ecientof x in V x xisV x x

n

Q

n

Thus V x xV x x x x The result now follows by induction

n i

i

x



x x and the obvious case

x

Second Proof Lagrange interp olation We could dene p immediately if wehad

p olynomials x P i nsuch that x where is Kroneckers

i n i j ij ij

P

n

delta that is fori j and fori j Indeed px y x

ij ij i i

i

would then work as our interp olating p olynomial In short notice that the p olynomials

f g would form a particularly convenient basis for P

n n

Well givetwo formulas for x

i

Y

x x

j

a Clearly x works

i

x x

i j

j i

b Start with W xx x x x x x and notice that the p olynomial we

n

need satises

W x

xa

i i

x x

i

for some a R Why But then x a W x again why that is

i i i i i

W x

x

i

x x W x

i i

Q

Please note that xisamultiple of the p olynomial x x for i nand

i j

j i

that px is then a suitable linear combination of such p olynomials

Third Proof Newtons formula We seek px of the form

pxa a x x a x x x x a x x x x

n n

Please note that x do es not app ear on the righthand side This form makes it almost

n

eortless to solve for the a s by pluggingin the x s i n

i i

y px a

y a

x x a y px a a

x x

Interp olation

Continuing we nd

y a a x x

a

x x x x

y a a x x a x x x x

a

x x x x x x

and so on Natanson Vol I I I gives another formula for the a s

i

Example

As a quick means of comparing these three solutions lets nd the interp olating p olynomial

quadratic passing through and Youre invited to check the following

Vandermonde px x x

Lagrange pxx x x x x x

Newton px x x x

As you mighthave already surmised Lagranges metho d is the easiest to apply by

hand although Newtons formula has much to recommend it to o its esp ecially wellsuited

to situations where weintro duce additional no des We next set up the necessary notation

to discuss the ner p oints of Lagranges metho d

Given n distinct p oints a x x x b sometimes called no des we

n

rst form the p olynomials

n

Y

W x x x

i

i

and

Y

x x W x

j

x

i

x x x x W x

i j i i

j i

The Lagrange interp olation formula is

n

X

L f x f x x

n i i

i

That is L f is the unique p olynomial in P that agrees with f at the x s In particular

n n i

notice that wemust have L pp whenever p P In fact L is a linear pro jection

n n n

from C a b onto P WhyisL f linear in f

n n

Interp olation

Typically were given or construct an array of no des

x

x x

X

x x x

and form the corresp onding sequence of pro jections

n

X

n n

L f x f x x

n

i i

i

An easy but admittedly p ointless observation is that for a given f C a b we can always

nd an array X so that L f p the p olynomial of b est approximation to f out of P

n n

n

since f p has n zeros wemay use these for the x s Thus kL f f k E f

i n n

n

in this case However the problem of convergence changes character dramatically if we

rst cho ose X and then consider L f In general theres no reason to b elieve that L f

n n

converges to f In fact quite the opp osite is true

Theorem Fab er Given anyarray of no des X in a b there is some f C a b

for which kL f f k is unb ounded

n

The problem here has little to do with interp olation and everything to do with pro

jections

Theorem Kharshiladze Lozinski For each nletL be a continuous linear

n

pro jection from C a b onto P Then there is some f C a b for which kL f f k

n n

is unb ounded

Evidently the op erators L arent p ositive monotone for otherwise the Bohman

n

Korovkin theorem and the fact that L is a pro jection onto P would imply that L f

n n n

converges uniformly to f for every f C a b

The pro ofs of these theorems are long and dicultwell save them for another day

Some of you may recognize the Principle of Uniform Boundedness at work here The

real p oint here is that we cant haveeverything A p ositive result ab out convergence of

t interp olation will require that we imp ose some extra conditions on the functions f wewan

to approximate As a rst step in this direction weprove that if f has suciently many

derivatives then the error kL f f k can at least b e measured n

Interp olation

Theorem Supp ose that f has n continuous derivatives on a b Let a x

x x b let p P b e the p olynomial that interp olates f at the x s and let

n n i

Q

n

W x x x Then

i

i

n

kf pk kf kkW k

n

Proof Well prove the Theorem byshowing that given x in a b there is a in a b

with

n

f W x f x px

n

If x is one of the x s then b oth sides of this formula are and were done Otherwise

i

W x and wemay set f x pxW x Now consider

tf t pt W t

Clearly x foreach i n and by our choice of we also have x

i

Here comes Rolles theorem Since has n distinct zeros in a b w emust have

n

for some in a b Why Hence

n n n n

f p W

f x px

n

f n

W x

b ecause p has degree at most n and W is monic and degree n

Observations

Equation is called the Lagrange formula with remainder Compare this result to

Taylors formula with remainder

n

The term f is actually a continuous function of x That is f x pxW x

is continuous its value at an x is f x p x W x why and W x

i i i i i

Q

x x

i j

j i

On any interval a b using any no des the sequence of Lagrange interp olating p oly

x x

nomials for e con In this case verge uniformly to e

c

x x n

ke L e k b a

n

n

Interp olation

x

where c ke k in C a b The same would hold true for any innitely dierentiable

n n

function satisfying say kf kM anyentire function for example

Q

n

x x is minimized by taking x cosi n On the norm of

i i

i

the zeros of the nth Chebyshev p olynomial T Why As Rivlin p oints out the

n

zeros of the Chebyshev p olynomials are a nearly optimal choice for the no des if go o d

uniform approximation is desired

The question of convergence of interp olation is actually very closely related to the

analogous question for the convergence of Fourier seriesand the answer here is nearly

the same Well have more to say ab out this analogy later First lets note that L is

n

continuous b ounded this will give us our rst bit of insightinto Fab ers negativ e result

P

n

Lemma kL f kkf k j xj for any f C a b

n i

i

Proof Exercise

P

n

The numb ers j xj are called the Leb esgue numbers asso ciated to this

n i

i

pro cess Its not hard to see that is the smallest p ossible constant that will work in

n

this inequality in other words kL k Indeed if

n n

n n

X X

j j xj x j

i i

i i

then we can nd an f C a b with kf k and f x sgn x for all i How

i i

Then

n n

X X

j x j kf k sgn x x kL f kjL f x j

i n i i n n

i i

As it happ ens c log n this is where the hard work comes in see Rivlin or Natanson

n

for further details and in particular as n

n

A simple application of the triangle inequality will allowustobringE f backinto

n the picture

Interp olation

Lemma Leb esgues theorem kf L f k E f for any f C a b

n n n

Proof Let p b e the b est approximation to f out of P Then since L p p we

n n

have

kf L f kkf p k kL f p k

n n

kf p k E f

n n n

App endix

Although wewont need anything quite so fancy it is of some interest to discuss more

general problems of interp olation We again supp ose that we are given distinct p oints

x x in a b but nowwe supp ose that we are given an array of information

n

m



y y y y

m

y y y y

m

n

y y y y

n

n

n n

where each m is a nonnegativeinteger Our problem is to nd the p olynomial p of least

i

degree that incorp orates all of this data by satisfying

m



m



p x y p x y p x y

m

m

p x y px y p x y

m

n

m

n

p x y px y p x y

n

n n n n

n

In other words we sp ecify not only the value of p at each x but also the rst m derivatives

i i

of p at x This is often referred to as the problem of Hermite interp olation

i

Since the problem has a total of m m m n degrees of freedom

n

it wont come as any surprise that is has a unique solution p of degree at most N

m m m n Rather than discuss this particular problem any further lets

n

instead discuss the general problem of linear interp olation

The notational framework for our problem is an ndimensional vector space X on

which m linear realvalued functions or linear functionals L L are dened The

m

general problem of linear interp olation asks whether the system of equations

L f y i m

i i

Interp olation

has a unique solution f X for anygiven set of scalars y y R Since a linear

m

functional is completely determined by its values on anybasiswewould next b e led to

consider a basis f f for X and from here it is a small step to rewrite asamatrix

n

equation That is we seek a solution f a f a f satisfying

n n

a L f a L f y

n n

a L f a L f y

n n

a L f y a L f

n m n m m

If we are to guarantee a solution a a for eachchoice of y y thenwell need

n m

to have m n and moreover the matrix L f will have to b e nonsingular

i j

Lemma Let X be an ndimensional vector space with basis f f and let L L

n n

b e linear functionals on X Then L L are linearly indep endent if and only if the

n

matrix L f is nonsingular that is if and only if det L f

i j i j

is singular then the matrix equation Proof If L f

i j

c L f c L f

n n

c L f c L f

n n

c L f c L f

n n n n

has a nontrivial solution c c Thus the functional c L c L satises

n n n

c L c L f i n

n n i

Since f f form a basis for X this means that

n

c L c L f

n n

for all f X That is c L c L the zero functional and so L L are

n n n

linearly dep endent

Converselyif L L are linearly dep endent just reverse the steps in the rst part

n

of the pro of to see that L f is singular

i j

Interp olation

Theorem Let X be an ndimensional vector space and let L L b e linear function

n

als on X Then the interp olation problem

L f y i n

i i

always has a unique solution f X for anychoice of scalars y y if and only if

n

L L are linearly indep endent

n

Proof Let f f b e a basis for X Then is equivalent to the system of equations

n

a L f a L f y

n n

a L f a L f y

n n

a L f a L f y

n n n n n

by taking f a f a f Thus always has a solution if and only if always

n n

has a solution if and only if L f is nonsingular if and only if L L are linearly

i j n

indep endent In any of these cases note that the solution must b e unique

In the case of Lagrange interp olation X P and L is evaluation at x ie L f

n i i i

f x which is easily seen to b e linear in f Moreover L L are linearly indep endent

i n

provided that x x are distinct Why

n

In the case of Hermite interp olation the linear functionals are of the form L f

xk

k

f x dierentiation comp osed with a p ointevaluation If x y thenL and L

xk ym

are indep endent for any k and mifk m then L and L are indep endent How

xk xm

would you check this

Math Problem Set Lagrange Interp olation

Throughout x x x are distinct p oints in some interval a b and V x x x

n n

denotes the Vandermonde determinant

n

x x x

n

x x x

V x x x

n

n

x x x

n

n n

Q

x x x x x Showby induction that V

n i j

jin

Hint In order to reduce to the n n case replace c thej th column by c x c

j j j

starting on the right with j nFactor and use the induction hyp othesis

Let y y y R b e given Show that the p olynomial p P satisfying px y

n n i i

i nmay b e written as

n

x x x

n

y x x x

pxc

n

x y x x

n n

n n

where c is a certain constant Find c and prove the formula

Given f C a b let L f denote the p olynomial of degree at most n that agrees

n

with f at the x s Prove that L is a linear pro jection onto P That is showthat

i n n

f if and only if f P L f g L f L g and that L f

n n n n n

Let x i n denote the Lagrange interp olating p olynomials of degree

i

at most n asso ciated with the no des x x x that is x Showthat

n i j ij

P P

n n

k k

x and more generallythat x xx for k n

i i

i

i i

If and L are as ab ove show that the error in the Lagrange interp olation formula

i n

P

n

is L f f x f x f x x

n i i

i

P

n

With and L as ab ove show that kL f k kf k where j xj

i n n n n i

i

Show that no smaller numb er has this prop erty for all f C a b

Math Approximation on Finite Sets

In this section we consider a question of computational interest Since b est approximations

are often very hard to nd howmightwe approximate the b est approximation The

answer to this question lies in approximations over nite sets Heres the plan

Fix a nite subset X of a b consisting of m distinct p oints a x x b

m m

and nd the b est approximation to f out of P considered as a subspace of C X

n m

In other words if we call the b est approximation p X then

m

n

jf x px jE f X max jf x p X x j min max

i i n m i m i

n

m im pP i

n

Argue that this pro cess converges in some sense to the b est approximation on all

of a b provided that X gets big as m In actual practice theres no need

m

to worry ab out p X converging to p the b est approximation on all of a b

m

n n

rather we will argue that E f X E f and app eal to abstract nonsense

n m n

Find an ecient strategy for carrying out items and

Observations

If m n then E f X That is we can always nd a p olynomial p P

n m n

n or fewer p oints How Of course p wont b e unique if that agrees with f at

m n Why In any case wemightaswell assume that m n In fact as

well see the case m n is all that we really need to worry ab out

If X Y a b then E f X E f Y E f Indeed if p P is the b est

n n n n

approximation on Y then

j E f Y E f X max jf x pxjmax jf x px

n n

xX xY

Consequentlywe exp ect E f X toincrease to E f asX gets big

n m n m

Nowifwewere to rep eat our earlier work on characterizing b est approximations

restricting ourselves to X everywhere heres what wed get m

Finite Sets

Theorem Let m n Then

i p P is a b est approximation to f on X if and only if f p has an alternating set

n m

containing n points out of X that is f p E f X alternatelyonX

m n m m

ii p X is unique

m

n

Next lets see how this reduces our study to the case m n

Theorem Fix n m n and f C a b

i If p P is b est on all of a b then there is a subset X of a b containing n

n

n n

p oints such that p p X Moreover E f X E f E f X

n n n n

n n n n

for any other subset X of a b with equality if and only if p X p

n n

n n

of X suchthat X P is b est on X then there is a subset X ii If p

m m n m

n n

p X p X and E f X E f X For any other X X

m n m n n m

n n n n

wehave E f X E f X E f X with equality if and only if

n n n n m

n

p X p X

n m

n n

p over a b containing exactly n Proof i Let X b e an alternating set for f

n n

p oints Then X is also an alternating set for f p over X That is for x X

n n n n

y j x E f max jf y p f x p

n

n n



y X

n

So by uniqueness of b est approximations on X wemust have p p X and

n n n n

E f E f X The second assertion follows from a similar argument using the

n n

n

uniqueness of p on a b

n

ii This is just i with a b replaced everywhere by X

m

Heres the p oint Through some as yet undisclosed metho d wecho ose X with

m

m n in fact mn such that E f X E f E f X and

n m n n m

then wesearch for the b est X X meaning the largest value of E f X We

n m n n

then take p X as an approximation for p Aswell see momentarily p X can

n n

n n n

b e computed directly and explicitly

Finite Sets

Now supp ose that the elements of X are a x x x blet

n n

n

p p X bepxa a x a x and let

n n

n

E E f X max jf x px j

n n i i

in

E alternatelyand In order to compute p and E we use the fact that f x px

i i

write for instance

f x E px

f x E px

n

f x E px

n n

n

where the E column might instead read E E E That is in order to

nd p and E we need to solve a system of n linear equations in the n unknowns

E a a The determinant of this system is up to sign

n

n

x x

n

x x

A A A

n

n n

x x

n

n

where wehave expanded by cofactors along the rst column and have used the fact that

each minor A is a Vandermonde determinant and hence each A If we apply

k k

Cramers rule to nd E we get

n

f x A f x A f x A

n n

E

A A A

n

n

f x f x f x

n n

P P

n n

i

q x Moreover these same s satisfy where and

i i i i

i i

for every p olynomial q P since E E q X for p olynomials of degree at most

n n n

n and since Cramers rule supplies the same co ecients for all f s

It may b e instructiv e to see a more explicit solution to this problem For this recall

that since wehave n p oints wemayinterp olate exactly out of P Given this our

n

original problem can b e rephrased quite succinctly

Finite Sets

Let p b e the unique p olynomial in P satisfying px f x i n

n i i

i

and let e b e the unique p olynomial in P satisfying ex i n

n i

If it is p ossible to nd a scalar so that p e P then p e p X and

n n

n

jj E f X Why Because f p ee alternatelyonX and so

n n n

j j max jf x px exjThus we need to compare leading co ecients of p

xX

n

and e

Nowif p has degree less than n then p p X andE f X Thus

n n n

n

would do nicely in this case Otherwise p has degree exactly n and the question

is whether e do es to o Now

n

i

X

W x

ex

W x x x

i i

i

Q P

n n

i

where W x x x and so the leading co ecientofe is W x

i i

i i

Well b e done if we can convince ourselves that this is nonzero But

i n

Y Y Y

ni

W x x x x x x x

i i j i j j i

j j i

j i

n i

W x is of constantsign Finally since hence

i

n

X

W x f x

i

px

W x x x

i i

i

P

n

f x W x and its easy to nd the value of p has leading co ecient

i i

i

Conclusion p X p e where

n

n

P

n

n

X

f x W x

i i

i

i

f x

P

i i

n

i

W x

i

i

i

and

jW x j

i

P

i

n

jW x j

j

j

P

n

i

and jj E f X Moreover q x for every q P

n n i i n

i

Finite Sets

Example

Find the b est linear approximation to f xx on X f g

We seek pxa a x and we need only consider subsets of X ofsize

There are four

X f gX f gX f gX f g

In eachcasewe nd a p and a E in our earlier setup For instance in the case of

X wewould solve the system of equations f x px for x

a

a a

a

a a

a

In the other three cases you would nd that and

Since we need the largest were done X or X works and p X xx

xx Recall that the b est approximation on all of is p

Where do es this leaveusWe still need to know that there is some hop e of nding an

initial set X with E f E f X E f and we need a more ecientmeans

m n n m n

m

hing through the subsets X X In order to attack the problem of of searc

n m

n

nding an initial X well need a few classical inequalities Wewont directly attack the

m

second problem instead well outline an algorithm that b egins with an initial set X

n

containing exactly n points which is then improved to some X bychanging only

n

a single p oint

The Inequalities of Markov and Bernstein

In order to discuss the convergence of approximations over nite sets we will need to know

that dierentiation is b ounded on P a fact that is nearly obvious byitself

n

The inequalitywell use is due to A A Markov from

Finite Sets

Theorem If p P and if jpxj for jxjthenjp xjn for jxj

n

Moreover jp xj n can only o ccur at x and only when p T the Chebyshev

n

p olynomial of degree n

Markovs brother V A Markov later improved on this in byshowing that

k

k

jp xjT Weve alluded to this fact already see Rivlin p and even more

n

is true For our purp oses its enough to have some b ound on dierentiation in particular

well only use

kp kn kpk and kp kn kpk

where kk is the norm in C

Ab out years after Markov in Bernstein asked for a similar b ound for the

derivative of a complex p olynomial over the unit disk jz j Now the maximum mo dulus

i

theorem tells us that wemay reduce to the case jz j that is z e and so Bernstein

was able to restate the problem in terms of trig p olynomials

Theorem If S T and if jS j then jS jn Equality is only p ossible for

n

S sin n

Our plan is to deduce Markovs inequality from Bernsteins inequalityby a metho d of

pro of due to Polya and Szego in To b egin lets consider the Lagrange interp olation

formula in the case where x cosi n i n are the zeros of the

i

Chebyshev p olynomial T Recall that wehave x x x

n n n

Lemma Each p olynomial p P may b e written

n

n

q

X

T x

n

i

px x px

i

i

n x x

i

i

Proof We know that the Lagrange interp olation formula is exact for p olynomials of

degree n and weknow that up to a constantmultiple T x is the pro duct W x

n

x x x x All that remains is to compute T x But recall that for x cos

n i

n

wehave

n sin n n sin n n sin n

p p

T x

n

sin

cos x

Finite Sets

But for x cosi n ie for i n it follows that sin n

i i i

i

sini that is

p

i

x

i

T x n

i

n

Lemma For any p olynomial p P wehave

n

p

max jpxj max n x px

x x

p

Proof Tosavewear and tear lets write M max n x px

x

First consider an x in the interval x x that is jxjcosnx In this case

n

p

we can estimate x from b elow

r

q

p

x x cos sin

n n n

b ecause sin for from the mean value theorem Hence for jxj

p

x jpxjM cosn we get jpxjn

Now for xs outside the interval x x we apply our interp olation formula In this

n

case each of the factors x x is of the same signThus

i

p

n

i

X

x T x

n

i

jpxj px

i

n x x

i

i

n n

X X

T x M T x M

n n

n x x n x x

i i

i i

But

n

X

T x

n

T x why

n

x x

i

i

xjn Thus jpxjM and we knowthat jT

n

We next turn our attention to trig p olynomials As usual given an algebraic p oly

nomial p P we will so oner or later consider S pcos In this case S

n

p cos sin is an o dd trig p olynomial of degree at most n and jS j jp cos sin j

Finite Sets

p

jp x x jConverselyif S T is an o dd trig p olynomial then S sin is even

n

and so may b e written S sin pcos for some algebraic p olynomial p of degree at

most n From Lemma

S

max jpcos jn max jpcos sin j n max jS j max

sin

This proves

Corollary If S T is an o dd trig p olynomial then

n

S

max n max jS j

sin

Nowwere ready for Bernsteins inequality

Bernsteins Inequality If S T then

n

max jS jn max jS j

Proof We rst dene an auxiliary function f S S For

xed f is an o dd trig p olynomial in of degree at most n Consequently

f

n max jf jn max jS j

sin

But

f S S

lim S lim

sin

and hence jS jn max jS j

Finallyweprove Markovs inequality

Markovs Inequality If p P then

n

max jp xjn max jpxj

x x

Finite Sets

Proof We know that S pcos is a trig p olynomial of degree at most n satisfying

max jpxj max jpcos j

x

Since S p cos sin is also trig p olynomial of degree at most n Bernsteins in

equality yields

max jp cos sin jn max jpcos j

In other words

p

p x max n max jpxj x

x x

Since p P the desired inequalitynow follows easily from Lemma

n

p

max jp xjn max p x n max jpxj x

x x x

Convergence of Approximations over Finite Sets

In order to simplify things here wewillmakeseveral assumptions For one we will

consider only approximation over the interval I As b efore we consider a xed

f C and a xed integer n For eachinteger m wecho ose a nite

subset X I consisting of m points x x in addition we will

m m

assume that x and x Ifwe put

m

max min jx x j

m i

xI im

then each x I is within of some x If X consists of equally spaced p oints for

m i m

example its easy to see that m

m

Our goal is to prove

Theorem If then E f X E f

m n m n

And wewould hop e to accomplish this in sucha way that is a measurable quantity

m

dep ending on f m and a prescrib ed tolerance E f X E f

n m n

As a rst step in this direction lets bring Markovs inequalityinto the picture

Finite Sets

Lemma Supp ose that n Then for any p P wehave

m n

m

max jpxj max jpx j

m i

x im

and

n max jpx j

p m m m i

im

Proof Take a in with jpaj kpk If a X were done since

m

Otherwise well have a and p a Next cho ose x X

m i m

with ja x j and apply Taylors theorem

i m

x a

i

px pax a p a p c

i i

for some c in Rewriting wehave

m

jpajjpx j jp cj

i

And nowwe bring in Markov

n

m

kpk max jpx j kpk

i

im

which is what we need

The real p oint here is that each p P is Lipschitz with constant n kpk Indeed

n

jps ptj js t p cjjs tjkp k n kpkjs tj

from the mean value theorem and Markovs inequality Thus n kpk and

p

combining this with weget

n kpk n max jpx j

p m m m m i

im

Nowwere ready to compare E f X to E f Our result wont b e as go o d as

n m n

Rivlins he uses a fancier version of Markovs inequality but it will b e a bit easier to

prove As in the Lemma well supp ose that

n

m

m

and well set

n

m

m

m

Note that as wealsohave and

m m m

Finite Sets

Theorem For f C

E f X E f E f X kf k

n m n m n m f m m

Consequentlyif thenE f X E f as m

m n m n

Recall that Proof Let p p X P b e the b est approximation to f on X

m m n

n

max jf x px j E f X E f kf pk

i i n m n

im

Our plan is to estimate kf pk

Let x and cho ose x X with jx x j Then

i m i m

jf x pxj jf x f x j jf x px j jpx pxj

i i i i

E f X

f m n m p m

E f X max jpx j

f m n m m i

im

where weve used from the previous Lemma to estimate All that remains is to

p m

revise this last estimate eliminating reference to pFor this we use the triangle inequality

again

j max jf x px j max jf x j max jpx

i i i i

im im im

E f X kf k

n m

Putting all the pieces together gives us our result

E f E f X E f X kf k

n f m n m m n m

As Rivlin p oints out it is quite p ossible to givealower b ound on m in the case of say

equally spaced p oints which will give E f X E f E f X but this is

n m n n m

surely an inecient approach to the problem Instead well discuss the one p oint exchange

algorithm

The One PointExchange Algorithm

Were given f C n and

Finite Sets

ni

Pick a starting reference X A convenientchoice is the set x cos

n i

n

ni

i n These are the p eak p oints of T that is T x

n n i

and so T is the p olynomial e from our Conclusion

n

Find p p X and by solving a system of linear equations Recall that

n

n

jj jf x px j kf p k kf pk

i i

where p is the b est approximation to f on all of

px and any Find approximately if necessary the error function exf x

p oint where jf p j kf pk According to Powell this can b e accomplished

using lo cal quadratic ts

Replace an appropriate x by so that the new reference X fx x g has the

i

n

prop erties that f x px alternates in sign and that jf x px jjj for all

i i i i

i The new p olynomial p p X and new must then satisfy

n n

jj min jf x px j max jf x p x j j j

i i i i

in in

This is an observation due to de la Vallee Poussin Since f p alternates in sign on

an alternating set for f p it follows that f p increases the minimum error over

this set See the Theorem on page of Characterization of Best Approximation

for a precise statement Again according to Powell the new p and can b e found

quickly through matrix up dating techniques Since wev e only changed one of the

x s only one row of the matrix on page needs to b e changed

i

The new satises j jkf p kkf p k and the calculation stops when

kf p kj j jf p jj j

Math A Brief Intro duction to Fourier Series

The Fourier series of a p erio dic b ounded integrable function f is

X

a

a cos kx b sin kx

k k

k

where the co ecients are dened by

Z Z

f t cos kt dt and b f tsinkt dt a

k k

Please note that if f is Riemann integrable on then each of these integrals is

welldened and nite indeed

Z

ja j jf tj dt

k

and so for example wewould have ja jkf k for f C

k

We write the partial sums of the series as

n

X

a

s f x a cos kx b sin kx

n k k

k

Now while s f need not converge p ointwise to f in fact it mayeven diverge at a given

n

p oint and while s f is not typically a go o d uniform approximation to f itisstilla

n

very natural choice for an approximation to f in the leastsquares sense whichwell

make precise shortly Said in other words the Fourier series for f provides a useful

representation for f even if it fails to converge p ointwise to f

Observations

The functions cos x cos xsinx sin xareorthogonal on That is

Z Z Z

sin mx sin nx dx cos mx sin nx dx cos mx cos nx dx

for any m n and the last equation even holds for m n

Z Z

cos mx dx sin mx dx

Fourier Series

R

dx for any m and of course

P

n



cos kx sin kx then What this means is that if T x

k k

k

Z Z

m

T x cos mx dx cos mx dx

m

for m while

Z Z

T x dx dx

That is if T T then T is actually equal to its own Fourier series

n

The partial sum op erator s f isalinear pro jection from C onto T

n n

P

n



If T x cos kx sin kx is a trig p olynomial then

k k

k

Z Z Z

n

X

k

f x T x dx f x dx f x cos kx dx

k

Z

n

X

k

f xsinkx dx

k

n

X

a

a b

k k k k

k

where a and b are the Fourier co ecients for f This should remind you of the

k k

dot pro duct of the co ecients

Motivated by and we dene the inner pro duct of two elements f g C by

Z

f x g x dx hf gi

Note that from wehave hf s f i hs f s f i for any n Why

n n n

If some f C has a b forall k then f

k k

Indeed by or linearity of the integral this means that

Z

f x T x dx

for any trig p olynomial T But from Weierstrasss second theorem we know that f is

the uniform limit of some sequence of trig p olynomials T Thus

n

Z Z

f x dx lim f x T x dx

n

n

Fourier Series

Since f is continuous this easily implies that f

If f g C have the same Fourier series then f g Hence the Fourier series for

an f C provides a representation for f even if the series fails to converge to f

The co ecients a a a and b b b minimize the expression

n n

Z

a a b f x s f x dx

n n

Its not hard to see for example that

Z

f x s f x cos kx dx

n

a

k

precisely when a satises

k

Z Z

f x cos kx dx a cos kx dx

k

The partial sum s f is the b est approximation to f out of T relativetothe L

n n

norm

Z

q

kf k f x dx hf f i

Be forewarned Some authors prefer in place of That is

kf T k kf s f k min

n

T T

n

Moreover using and wehave

hf s f f s f i kf s f k

n n n

hf f i hf s f i hs f s f i

n n n

kf k ks f k

n

Z

n

X

a

f x dx a b

k k

k

This should remind you of the Pythagorean theorem

It follows from that

Z Z

n

X

a

a b s f x dx f x dx

n

k k

k

Fourier Series

In other symbols ks f k kf k In particular the Fourier co ecients of any

n

f C are square summable Why

If f C then its Fourier co ecients a and b tend to zero as n

n n

It follows from and Weierstrasss second theorem that s f f in the L norm

n

whenever f C Indeed given cho ose a trig p olynomial T suchthat

kf T k Then since s T T for large enough nwehave

n

kf s f k kf T k ks T f k

n n

p p

kf T k kf T k

Compare this calculation with Leb esgues Theorem page

By way of comparison lets give a simple class of functions whose Fourier partial sums

provide go o d uniform approximations

Theorem If f C then the Fourier series for f converges absolutely and uniformly

to f

Proof First notice that integration byparts leads to an estimate on the order of growth

of the Fourier co ecients of f

Z Z Z

sin kx

f xsinkx dx a f x cos kx dx f x d

k

k k

kk as k Nowweintegrate b ecause f is p erio dic Thus ja jkf

k

byparts again

Z Z Z

cos kx

ka f xsinkx dx f x d f xcoskx dx

k

k k

b ecause f is p erio dic Thus ja jkf kk ask More imp ortantly

k

this inequality along with the Weierstrass M test implies that the Fourier series for f is

b oth uniformly and absolutely convergent

X X X

a a

a cos kx b sin kx ja j jb j C

k k k k

k

k k k

Fourier Series

But why should the series actually converge to f Well if we call the sum

X

a

a cos kx b sin kx g x

k k

k

then g C why and g has the same Fourier co ecients as f why Hence by

g f

Our next chore is to nd a closed expression for s f For this well need a couple of

n

trig identities the rst two need no explanation

cos kt cos kx sinkt sin kx cos k t x

cos sin sin sin

sin n

cos cos cos n

sin

Heres a short pro of for the third

n n

X X

sin sin sin k sin k sin n cos k sin

k k

The function

t sin n

D t

n

sin t

is called Dirichlets kernelItplays an imp ortant role in our next calculation

Nowwere ready to rewrite our formula for s f

n

n

X

s f x a a cos kx b sin kx

n k k

k

Z

n

X

cos kt cos kx sinkt sin kx dt f t

k

Z

n

X

cos k t x dt f t

k

Z

sin n t x

f t dt

sin t x

Z Z

f t D t x dt f x t D t dt

n n

Fourier Series

It now follows easily that s f is linear in f b ecause integration against D is linear

n n

that s f T b ecause D T and in fact that s T T In other words

n n n n n m

minmn

s is indeed a linear pro jection onto T

n n

While weknow that s f is a go o d approximation to f in the L norm a b etter

n

understanding of its eectiveness as a uniform approximation will require a b etter under

standing of the Dirichlet kernel D Here are a few p ertinent facts

n

Lemma a D is even

n

Z Z

D t dt D t dt b

n n

c jD tjn and D n

n n

j sin n t j

jD tj for t d

n

t t

Z

e If jD tj dt then log n log n

n n n

Proof a b and c are relatively clear from the fact that

D t cos t cos t cosnt

n

Notice to o that b follows from the fact that s For d we use a more delicate

n

estimate Since sin for it follows that t sint t for

t Hence

j sin n t j t j j sin n

t t

sin t

for t Next the upp er estimate in e is easy

Z Z

t j j sin n

dt jD tj dt

n

sin t

Z Z

n

dt n dt

t

n

n

log logn logn

n

The lower estimate takes some work

Z Z Z

j sin n j sin n t j t j

jD tj dt dt dt

n

t

sin t

Fourier Series

Z Z

n n

j sin x j j sin x j

dx dx

x x

Z

n

k

X

j sin x j

dx

x

k

k

Z

n n

k

X X

j sin x j dx

k k

k

k k

log n

P

n

b ecause log n

k

k

R

The numb ers kD k jD tj dt are called the Leb esgue numbers asso

n n n

ciated to this pro cess compare this to the terminology we used for interp olation The

p oint here is that gives the norm of the partial sum op erator pro jection on C and

n

just as with interp olation as n As a matter of no small curiosity notice

n

that from Observation the norm of s as an op erator on L is

n

Corollary If f C then

Z

js f xj jf x tjjD tj dt kf k

n n n

In particular ks f k kf k log nkf k

n n

If we approximate the function sgn D by a continuous function f of norm one then

n

Z

jD tj dt s f

n n n

Thus is the smallest constant that works in The fact that the partial sum op erators

n

are not uniformly b ounded on C along with the Baire category theorem tells us that

there must b e some f C for which ks f k is unb ounded But as weve seen this has

n

more to do with pro jections than it do es with Fourier series

Theorem Kharshiladze Lozinski For each nlet L b e a continuous linear pro jection

n

from C onto T Then there is some f C for which kL f f k is unb ounded

n n

Although our last Corollary may not lo ok very useful it do es give us some information

ab out the eectiveness of s f as a uniform approximation to f Sp ecicallywehave

n

Leb esgues theorem

Fourier Series

T

Theorem If f C and if weset E f min kf T k then

n

T T

n

T T

f f kf s f k log n E E

n

n n

Proof Let T b e the b est uniform approximation to f out of T Then since s T

n n

T we get

kf s f kkf T k ks T f k log n kf T k

n n

As an application of Leb esgues theorem lets sp eak briey ab out Chebyshev se

ries a notion that ts neatly in b etween our discussions of approximation by algebraic

p olynomials and by trig p olynomials

Theorem Supp ose that f C is twice continously dierentiable Then f may

b e written as a uniformly and absolutely convergent Chebyshev series that is f x

P P

a T x where ja j

k k k

k k

Proof As usual consider f cos C Since is even and twice dieren

tiable its Fourier series is an absolutely and uniformly convergent cosine series

X X

f cos a cos k a T cos

k k k

k k

P

where ja j k kk Thus f x a T x

k k k

k

P

n

If wewrite S f x a T x we get an interesting consequence of this

n k k

k

Theorem First notice that

S f cos s

n n

Thus from Leb esgues theorem

E f kf S f k k s k

n n n

C C

T

log n E f log n E

n

n

For n this reads

E f k f S f k E f

n n n

Fourier Series

P

n

That is for numerical purp oses the error incurred byusing a T x to approximate

k k

k

f is within one decimal place accuracy of the b est approximationNotice to o that E f

n

would b e very easy to estimate in this case since

X X X

E f kf S f k a T ja j k k

n n k k k

k

kn kn kn

Leb esgues theorem should remind you of our fancy version of Bernsteins theorem

T

if we knew that E f log n asn then wed knowthat s f converged uni

n

n

T

formly to f Our goal then is to improve our estimates on E f and the idea b ehind

n

these improvements is to replace D by a b etter kernel with regard to uniform approx

n

T

imation Before we pursue anything quite so delicate as an estimate on E f though

n

lets investigate a simple and useful replacementfor D

n

Since the sequence of partial sums s need not converge to f wemight try lo oking

n

at their arithmetic means or Cesaro sums

s s s

n

n

n

These averages typically have b etter convergence prop erties than the partial sums them

n

selves Consider in the scalar case s for example Sp ecicallyweset

n n

h i

s f x s f x f x

n n

n

Z Z

n

X

f x t f x t K t dt D t dt

n k

n

k

where K D D D n is called Fejers kernel The same techniques we

n n

used earlier can b e applied to nd a closed form for f which of course reduces to

n

simplifying D D D n As b efore we b egin with a trig identity

n

n n

X X

sin sin k cos k cos k

k k

cos n sin n

Thus

n

X

sin k t sin nt

K t

n

n sint

n sin t

k

Fourier Series

R

Please note that K is even nonnegative and K t dt Thus f is

n n n

a p ositive linear map from C onto T but its not a pro jectionwhy satisfying

n

k f k kf k why

n

Now the arithmetic mean op erator f is still a go o d approximation f in L norm

n

Indeed

n n

X X

f s f kf f k kf s f k

k n k

n n

k k

as n since kf s f k But more to the p oint f is actually a go o d

k n

uniform approximation to f a fact that well call Fejers theorem

Theorem If f C then f converges uniformly to f as n

n

Note that since f T Fejers theorem implies Weierstrasss second theorem

n n

CuriouslyFejer was only years old when he proved this result ab out while

Weierstrass was at the time he proved his appro ximation theorems

Well givetwoproofsofFejers theorem one with details one without But b oth

follow from quite general considerations First

Theorem Supp ose that k C satises

n

a k

n

Z

k t dt and b

n

Z

c k t dt for every

n

jtj

Z

Then f x t k t dt f x for each f C

n

Proof Let Since f is uniformly continuous wemaycho ose sothat

jf x f x tj for any x whenever jtj Next we use the fact that k is

n

nonnegative and integrates to to write

Z Z

f x t k t dt f x f x t k t dt f x

n n

Z

f x f x t k t dt

n

Fourier Series

Z Z

kf k

k t dt k t dt

n n

jtj jtj

for n suciently large

To see that Fejers kernel satises the conditions of the Theorem is easy In particular

c follows from the fact that K t on the set jtj Indeed since sint

n

increases on t wehave

sin nt

K t

n

n sin t n sin

Our second pro of or sketch really is based on a variant of the BohmanKorovkin

theorem for C due to Korovkin In this setting the three test cases are

f x f x cos x and f xsinx

Theorem Let L b e a sequence of p ositive linear maps on C If L f f for

n n

each of the three functions f x f x cos x and f xsinxthenL f f

n

for every f C

ewont prove this theorem rather well check that f f in each of the three W

n

test cases Since s is a pro jection this is painfully simple

n

f f f f f

n

n

n

f f f f f

n

n n

n

f f f f f

n

n n

Kernel op erators ab ound in analysis for example Landaus pro of of the Weierstrass

n

theorem uses the kernel L xc x And in the next section well encounter

n n

Jacksons kernel J tc sin ntn sin t whichisessentially the square of Fejers kernel

n n

While wewillhave no need for a general theory of such op erators please note that the key

to their utility is the fact that theyre nonnegative

Fourier Series

Lastlyaword or two ab out Fourier series involving complex co ecients Most mo dern

textb o oks consider the case of a p erio dic integrable function f R C and dene the

Fourier series of f by

X

ik t

c e

k

k

where nowwehave only one formula for the c s

k

Z

ik t

c f t e dt

k

but of course the c s maywell b e complex This somewhat simpler approach has other

k

ik t

advantages for one the exp onentials e are now an orthonormal set relative to the

normalizing constant And if we remain consistent with this choice and dene the

L norm by

Z

jf tj dt kf k

then wehave the simpler estimate kf k kf k for f C

The Dirichlet and Fejer kernels are essentially the same in this case to o except that

P

n

ik x

wewould now write s f x c e Given this the Dirichlet and Fejer kernels

n k

k n

can b e written

n n

X X

ik x ik x ik x

D x e e e

n

k n k

n

X

cos kx

k

x sin n

sin x

and

n m n

X X X

jk j

ik x ik x

K x e e

n

n n

m

k m k n

n

X

x sin m

n

x sin

m

sin nt

n sin t

Fourier Series

In other words eachistwice its real co ecient counterpart Since the choice of normalizing

p

p

constant versus and sometimes even or has a small eect on

these formulas you may nd some variation in other textb o oks

Math Problem Set Fourier Series

Dene f x x for x and extend f to a p erio dic continuous

function on R in the obvious way Check that the Fourier series for f is

P

cos nxn Since this series is uniformly convergent it actually converges to f

n

P

In particular note that setting x yields the familiar formula n

n

a Given n and show that there is a continuous function f C

R

jf t sgn D tj dt n satisfying kf k and

n

b Show that s f and hence that ks f k

n n n n

R

a If f k C prove that g x f x t k t dt is also in C

b If we only assume that f is p erio dic and Riemann integrable on but

still k C is g still continuous

c If we simply assume that f and k are p erio dic and Riemann integrable on

is g still continuous

Supp ose that k C satises

n

Z Z

k t dt and k t dt n k

n n n

jtj

R

for every If f is Riemann integrable show that f x t k t dt f x

n

p ointwise as n at each p oint of continuityof f In particular f x f x

n

at each p oint of continuityoff

Given f g C we dene the convolution of f and g written f g by

Z

f g x f t g x t dt

Compare this integral with that used in problem

a Show that f g g f and that f g C

b If one of f or g is a trig p olynomial show that f g is again a trig p olynomial

of the same degree

c If one of f or g is continuously dierentiable showthat f g is likewise continu

ously dierentiable and nd an integral formula for f g x

Math Jacksons Theorems

We continue our investigations of the middle ground b etween algebraic and trigonometric

approximation by presenting several results due to the great American mathematician

Dunham Jackson from roughly The rst of these results will giveusthebest

p ossible estimate of E f in terms of and n

n f

T

Jacksons Theorem If f C then E f

f

n

n

Theorem should b e viewed as an improvementover Bernsteins Theorem which

p

forf C As well see the pro of of Theorem stated that E f

f n

n

not only mimics the pro of of Bernsteins result but also uses some of the ideas we talked

ab out in the last section In particular the pro of well giveinvolves integration against an

improved Dirichlet kernel

Before we diveinto the pro of lets list several immediate and imp ortant Corollaries

Corollary Weierstrasss second theorem since for any f C

f

n

Corollary The DiniLipschitz theorem If log n as n then the Fourier

f

n

series for f converges uniformly to f

Proof From Leb esgues theorem

T

kf s f k log n E f log n

n f

n

n

Jacksons Theorem If f C then E f

n f

n

Proof Let f cos Then as weve seen

T

E f E

n f

n

n n

where the last inequality follows from the fact that

j j jf cos f cos j j cos cos j j j

f f

Jacksons Theorems

Corollary If f lip on then E f Kn Recall that Bernsteins

n

K

theorem gives only n

Corollary If f C has a b ounded derivative then E f kf k

n

n

E f Corollary If f C has a continuous derivative then E f

n n

n

Proof Let p P b e the b est uniform approximation to f and consider px

n

R

x

p t dt P From the previous Corollary

n

E f E f p Why

n n

kf p k E f

n

n n

Iterating this last inequalitywillgive the following result

Corollary If f C is k times continuously dierentiable then

k

E f

n k

nn n k n k

k

where is the mo dulus of continuityof f

k

Well enough corollaries Its time weproved Jacksons Theorem Now Jacksons

approachwas to showthat

Z

sin nt

f x t c dt f x

n

sin t

where J tc sin nt sin t is the improved kernel we alluded to earlier its essen

n n

tially the square of Fejers kernel The approachwell take due to Korovkin proves the

existence of a suitable kernel without giving a tidy formula for it On the other hand

its relatively easy to outline the idea The key here is that J t should b e an even

n

R

nonnegative trig p olynomial of degree n with J t dt In other words

n

n

X

cos kt J t

kn n

k

why is the rst term where must b e chosen so that J t As

n nn n

suming we can nd such s heres what weget kn

Jacksons Theorems

Lemma If f C then

r

Z

n

n f x t J t dt f x

n f

n

Proof We already knowhow the rst several lines of the pro of will go

Z Z

f x f x t J t dt f x f x t J t dt

n n

Z

jf x f x tj J t dt

n

Z

jtj J t dt

f n

Next we b orrow a trick from Bernstein We replace jtj by

f

jtj njtj njtj

f f f

n n

and so the last integral on the righthand side ab ove is dominated by

Z Z

n

jtj J t dt njtj J t dt

n n f f

n n

R

All that remains is to estimate jtj J t dt and for this well app eal to the Cauchy

n

Schwarz inequality again compare this to the pro of of Bernsteins theorem

Z Z

jtj J t dt jtj J t J t dt

n n n

Z Z

jtj J t dt J t dt

n n

Z

jtj J t dt

n

But

t

jtj sin cos t

So

r

Z Z

n

jtj J t dt cos t J t dt

n n

Jacksons Theorems

Nowwe still have to prove that we can actually nd a suitable choice of scalars

We already know that we need to cho ose the s so that J twillbe

n nn kn n

nonnegative but now its clear that wealsowant to b e very close to To get us

n

started lets rst see why its easy to generate nonnegative cosine p olynomials Given real

numbers c c note that

n

n n n

X X X X

ik j x ik x ik x ij x

A

c c e c e c e c e

k j k k j

j

kj k k

n

X X

ik j x ij k x

c c c e e

k j

k

k kj

n

X X

c c c cosk j x

k j

k

k kj

n n

X X

c c c cos x c c cos nx

k k n

k

k k

In particular we need to nd c c with

n

n n

X X

and c c c

n k k

k

k k

P P

n n

What well do is nd c s with c c c and then normalize But in fact

k k k

k

k k

wewont actually nd anythingwell simply write down a choice of c s that happ ens to

k

workConsider

n n

X X

k k k k

sin sin sin sin

n n n n

k k

n

X

k k k

sin sin sin

n n n

k

By changing the index of summation its easy to see that rst two sums are equal and

hence each is equal to the average of the two Next we rewrite this last sum using the

AB AB

trig identity sin A sin B cos sin to get

n n

X X

k k k

sin sin cos sin

n n n n

k k

Jacksons Theorems

k

Since cos where for large nweve done itIf w e dene c c sin

k

n n

P

n

and if we dene J x using then J x and c c is chosen so that

n n

k k

cos why The estimate needed in our Lemma b ecomes

n

n

v

u

r

u

cos

t

n

n

sin

n n

and so wehave

T

E f

f f

n

n n

Jacksons theorems are what wemight call direct theorems If weknow something

ab out f thenwecansay something ab out E f There is also the notion of an inverse

n

theorem meaning that if we know something ab out E f we should b e able to say

n

something ab out f In other words wewouldexpectaninverse theorem to b e more or

less the converse of some direct theorem Nowinverse theorems are typically much harder

to prove than direct theorems but in order to have some idea of what such theorems might

tell us and to see some of the techniques used in their pro ofs we present one of the easier

inverse theorems due to Bernstein This result gives the converse to one of our corollaries

to Jacksons theorem see the top of page

T

f An for some constants A and Theorem If f C satises E

n

then f lip for some constant K

K

Proof For each ncho ose U T so that kf U kAn Then in particular

n n n

n

n

U converges uniformly to f NowifwesetV U and V U U for n

n n

P

n

then V T and f V Indeed

n n

n

n n n

n

n

A B kV kkU f k kU f kA

n

P

which is summable thus the telescoping series V converges uniformly to f

n

n

Why

Next we estimate jf x f y j using nitely many of the V s the precise number to

n

b e sp ecied later Using the mean value theorem and Bernsteins inequalitywe get

X

jf x f y j jV x V y j

n n

n

Jacksons Theorems

m

X X

jV x V y j kV k

n n n

nm

n

m

X X

kV k jV jjx y j

n n

n

nm

n

m

X X

n

jx y j kV k kV k

n n

n nm

m

X X

n n

B B jx y j

nm

n

h i

m m

C jx y j

n

where weve used in the fourth line the fact that V T and in the last line standard

n

estimates for geometric series Nowwewant the righthand side to b e dominated bya

constant times jx y j In other words if weset jx y j then wewant

m m

D

or equivalently

m m

D

m

Thus we should cho ose m so that is b oth b ounded ab ove and b ounded awayfrom

m

zero For example if we could cho ose m so that

In order to b etter explain the phrase more or less the converse of some direct theo

rem lets see how the previous result falls apart when Although we might hop e

T

that E f An would imply that f lip it happ ens not to b e true The b est result

n K

in this regard is due to Zygmund who gave necessary and sucient conditions on f so

T

that E f An and these conditions do not characterize lip functions Instead of

n K

pursuing Zygmunds result well settle for simple surgery on our previous result keeping

an eye out for what go es wrong This result is again due to Bernstein

T

Theorem If f C satises E f An then Kj log j for some constant

f

n

K and all suciently small

Proof If we rep eat the previous pro of setting only a few lines change In

particular the conclusion of that long string of inequalites would nowread

m m

jf C m x f y jC jx y jm

Jacksons Theorems

Clearly the righthand side cannot b e dominated by a constanttimes aswe mighthave

hop ed for this would force m to b e b ounded indep endentof which in turn b ounds

m

away from zero But if we again think of as the variable in this inequality then the

term m suggests that the correct order of magnitude of the righthand side is j log j

Thus wewould try to nd a constant D so that

m

m D j log j

or

m m

m D j log j

Nowifwe take then log log j log j Hence if w e again cho ose

m

m so that well get

log log

m log log log m j log j

log log

and nally

m m

j log j j log j m m m

log log

Math Orthogonal Polynomials

Given a p ositive except p ossibly at nitely many p oints Riemann integrable weight

function w xona b the expression

Z

b

hf gi f x g x w x dx

a

denes an inner pro duct on C a b and

Z

b

p

hf f i f x w x dx kf k

a

denes a strictly convex norm on C a b Thus given a nite dimensional subspace E of

C a b and an element f C a b there is a unique g E such that

kf g k minkf hk

hE

Wesay that g is the leastsquares approximation to f out of E relativeto w

Nowifwe apply the GramSchmidt pro cedure to the sequence xx we will

arrive at a sequence Q oforthogonal p olynomials relative to the ab ove inner pro duct

n

In this sp ecial case however the GramSchmidt pro cedure simplies substantially

Theorem The following pro cedure denes a sequence Q of orthogonal p olynomials

n

relativeto w Set

Q x Q xx a and Q xx a Q x b Q x

n n n n n

for n where

a h xQ Q i h Q Q i and b h xQ Q i h Q Q i

n n n n n n n n n n

and where xQ is shorthand for the p olynomial xQ x

n n

Proof Its easy to see from these formulas that Q is a monic p olynomial of degree

n

exactly n In particular the Q s are linearly indep endent and nonzero n

Orthogonal Polynomials

Nowwechecked in class that Q Q andQ are mutually orthogonal so lets use

induction and check that Q is orthogonal to each Q k nFirst

n k

h Q Q i h xQ Q ia h Q Q ib h Q Q i

n n n n n n n n n n

and

h Q Q i h xQ Q ia h Q Q ib h Q Q i

n n n n n n n n n n

since h Q Q i Next we take kn and use the recurrence formula twice

n n

h Q Q i h xQ Q ia h Q Q ib h Q Q i

n k n k n n k n n k

h xQ Q i h Q xQ i Why

n k n k

h Q Q a Q b Q i

n k k k k k

since k n

Observations

Using the same trickasabove wehave

b h xQ Q i h Q Q i h Q Q i h Q Q i

n n n n n n n n n

P

n

Q where h p Q i h Q Q i Each p P can b e uniquely written p

i i i i i i n

i

P

n

If Q is any monic p olynomial of degree exactly n then Q Q Q why

n i i

i

and hence

n

X

kQk kQ k kQ k kQ k

n i n

i

i

unless Q Q That is Q has the least k k norm of all monic p olynomials of

n n

degree n

The Q s are unique in the following sense If P is another sequence of orthogonal

n n

p olynomials suchthat P has degree exactly nthenP Q for some

n n n n n

Why Consequently theres no harm in referring to the Q s as the sequence of

n

orthogonal p olynomials relativeto w

R

b

For n note that Q t w t dt hQ Q i

n n a

Orthogonal Polynomials

Examples

On the Chebyshev p olynomials of the rst kind T are orthogonal relative

n

p

x to the weight w x

Z Z

m n

dx

p

m n

T x T x cos m cos n d

m n

x

m n

Since T has degree exactly n this must b e the rightchoice Notice to o that

n

p

p

x T T T are orthonormal relative to the weight

In terms of the inductive pro cedure given ab ove wemust have Q T

n

and Q T for n Why From this it follows that a b

n n n

and b forn Why That is the recurrence formula given in our rst

n

Theorem reduces to the familar relationship T xxT x T x Curiously

n n n

n

Q T minimizes b oth

n n

Z

dx

p

max jpxj and px

x

x

over all monic p olynomials of degree exactly n

The Chebyshev p olynomials also satisfy x T x xT xn T x

n

n n

Since this is a p olynomial identity it suces to check it for all x cos In this case

n sin n

T x

n

sin

and

n cos n sin n sin n cos

T x

n

sin sin

Hence

xn T x x xT x T

n

n n

n cos n n sin n cot n sin n cot n cos

On the Chebyshev p olynomials of the second kind U are orthogonal relative

n

p

to the weight w x x

Z

dx

p

U x U x x

m n

x

Z

sin m sin n

m n

sin d

m n

sin sin

Orthogonal Polynomials

While were at it notice that

n sin n

nU x T x

n

n

sin

As a rule the derivatives of a sequence of orthogonal p olynomials are again orthogonal

p olynomials but relative to a dierentweight

On with weight w x the sequence P of Legendre p olynomials are

n

orthogonal and are typically normalized by P The rst few Legendre p oly

n

nomials are P x P xx P x x and P x x x Check

this After weve seen a few more examples well come back and give an explicit

formula for P

n

All of the examples weve seen so far are sp ecial cases of the following On

consider the weight w x x x where The corresp onding

are called the Jacobi p olynomials and are typically orthogonal p olynomials P

n

normalized by requiring that

n n

P

n

n

It follows that P P

n

n

n

T P

n

n

n

n

and

n

P U

n

n

n

n

The p olynomials P are called ultraspherical p olynomials

n

There are also several classical examples of orthogonal p olynomials on unb ounded

intervals In particular

x

w xe Laguerre p olynomials

x

w xx e generalized Laguerre p olynomials

x

w xe Hermite p olynomials

Since Q is orthogonal to every elementofP a fuller understanding of Q will

n n n

follow from a characterization of the orthogonal complement of P We b egin with an

n

easy fact ab out leastsquares approximations in inner pro duct spaces

Orthogonal Polynomials

Lemma Let E b e a nite dimensional subspace of an inner pro duct space X and let

x X n E Then y E is the leastsquares approximation to x out of E aka the

nearest p ointto x in E if and only if hx y yi for every y E that is if and only

if x y E

Proof Wevetaken E to b e nite dimensional so that nearest p oints will exist since

X is an inner pro duct space nearest p oints must also b e unique see the exercises for a

pro of that every inner pro duct norm is strictly convex

First supp ose that x y E Then given any y E wehave

kx y k kx y y y k kx y k ky y k

b ecause y y E and hence x y y y Thus kx y k kx y k unless

y y that is y is the unique nearest p ointto x in E

Supp ose that x y is not orthogonal to E Then there is some y E

with ky k such that hx y yi Now I claim that y y E is a b etter

approximation to x than y and y y y of course that is y is not the leastsquares

approximation to xTo see this we again compute

kx y y k kx y y k hx y y x y y i

hx y yi k kx y

kx y k kx y k

Thus wemust have hx y yi forevery y E

Lemma Integration byparts

Z Z

n

b b

i

X

b

n n n k nk k

uv u v u v

a

a a

k

n

Nowif v is a p olynomial of degree n then v and weget

Z

b

f x px w x dx for all p olynomials p P if Lemma f C a b satises

n

a

n

and only if there is an ntimes dierentiable function u on a b satisfying fw u and

k k

u au b for all k n

R

b

Proof One direction is clear from Lemma Given u as ab ove wewould have fpw

a

R R

b b

n n n

u p up

a a

Orthogonal Polynomials

R

b

fpw for all p P Byintegrating fw rep eatedly So supp ose wehave that

n

a

n

cho osing constants appropriatelywemay dene a function u satisfying fw u and

k

u a for all k n Wewanttoshow that the hyp otheses on f force

k

u b for all k n

Now Lemma tells us that

Z

n

b

X

k nk k

fpw u b p b

a

k

n

for all p P But the numbers pbp b p b are completely arbitrary that

n

is again byintegrating rep eatedlycho osing our constants as we please we can nd

j k

bforj k In b and p p olynomials p of degree knsuch that p

k

k k

k k

fact p xx b works just ne In any case wemust have u b for all

k

k n

Rolles theorem tells us a bit more ab out the functions orthogonal to P

n

Z

b

Lemma If w x in a b and if f C a b satises f x px w x dx for

a

all p olynomials p P then f has at least n distinct zeros in the op en interval a b

n

n k k

Proof Write fw u where u au b for all k n In

particular since uau b Rolles theorem tells us that u would have at least one

zero in a b But then u au cu b and so u must have at least two zeros

n

in a b Continuing we nd that fw u must have at least n zeros in a b Since

w the result follows

Corollary Let Q b e the sequence of orthogonal p olynomials asso ciated to a given

n

weight w with w in a b Then the ro ots of Q are real simple and lie in a b

n

Lemma If p is the leastsquares approximation to f C a b out of P and if

n

w in a b then f p has at least n distinct zeros in a b

Proof The leastsquares approximation satises h f p pi for all p P

n

The sheer volume of literature on orthogonal p olynomials and other sp ecial func

tions is truly staggering Well content ourselves with the Legendre and the Chebyshev

Orthogonal Polynomials

p olynomials In particular lets return to the problem of nding an explicit formula for

the Legendre p olynomials We could as Rivlin do es use induction and a few observations

that simplify the basic recurrence formula youre encouraged to read this see pp

Instead well give a simple but at rst sightintimidating formula that is of use in more

general settings than ours

Lemma with w and a b says that if wewant to nd a p olynomial

f of degree n that is orthogonal to P then well need to takeap olynomial for u

n

n n

and this u will have to b e divisible byx x Why That is wemust have

n n

P xc D x where D denotes dierentiation and where we nd c by

n n n

evaluating the righthand side at x

n

X

n

n k nk

Lemma Leibnizs formula D fg D f D g

k

k

n n n

Proof Induction and the fact that

k k k

P

n

n

n n n k n nk n

Consequently QxD x x D x D x and

k

k

n n n

it follows that Q nand Q n This nally gives us the formula

discovered by Ro drigues in

n n

D x P x

n

n

n

The Ro drigues formula is quite useful and easily generalizes to the Jacobi p olynomials

Observations

By Lemma the ro ots of P are real distinct and lie in

n

P

n

n

n k nk n

x x Ifwe apply D and simplifywe get another

n

k

k n

formula for the Legendre p olynomials

n

X

n n k

k nk

x P x

n

n

k n

k

In particular if n is even o dd then P is even o dd Notice to o that if welet

n

e

P denote the p olynomial given by the standard construction then wemust have

n

n

n

e

P P

n n n

Orthogonal Polynomials

In terms of our standard recurrence formula it follows that a b ecause xP x

n n

is always o dd It remains to compute b Firstintegrating by parts

n

Z Z

i

x P x P x dx P x dx xP x

n n n

n

or h P P i h P xP i But xP nP lower degree terms hence

n n n n

n n

h P xP i nh P P i Thus h P P i n Using this and the fact

n n n n n

n

n

n

e

that P P wed nd that b n n Thus

n n n

n

n n n

n n

e e e

P P x P P

n n n n

n n n

n n

xP P

n n

n n

That is the Legendre p olynomials satisfy the recurrence formula

n P x n xP x nP x

n n n

q

n

b

It follows from that the sequence P P is orthonormal on

n n

The Legendre p olynomials satisfy x P x xP xn n P x If

n

n n

n n n

we set u x that is if u nP note that u x nxuNowwe

n

n

apply D to b oth sides of this last equation using Leibnizs formula and simplify

n n

n n n n n

u x n u x u n u x n u

n n n

x u xu n n u

Through a series of exercises similar in spirit to Rivlin shows that jP xjon

n

See pp of Rivlin for details

Given an orthogonal sequence it makes sense to consider generalized Fourier series

relative to the sequence and to nd analogues of the Dirichlet kernel Leb esgues theorem

and so on In case of the Legendre p olynomials wehave the following

P

b b

Example The FourierLegendre series for f C is given by h f P i P

k k

k

where

r

Z

k

b b b

P P and h f P i f x P x dx

k k k k

Orthogonal Polynomials

P

n

b b

The partial sum op erator S f h f P i P is a linear pro jection onto P and may

n k k n

k

b e written as

Z

S f x f t K t x dt

n n

P

n

b b

where K t x P t P x Why

n k k

k

b

Since the P s are orthonormal wehave

k

n

X X

b b

jh f P ij jh f P ij kS f k kf k

k k n

k k

b

and so the generalized Fourier co ecients h f P i are square summable in particular

k

b

h f P iask As in the case of Fourier series the fact that the p olynomials

k

b

ie the span of the P s are dense in C a b implies that S f actually converges to f

k n

in the kk norm These same observations remain valid for any sequence of orthogonal

p olynomials The real question remains just as with Fourier series whether S f isa

n

go o d uniform or even p ointwise approximation to f

If youre willing to swallow the fact that jP xj then

n

r r

n n

X X

k k n

jK t xj k

n

k k

Hence kS f kn kf k That is the Leb esgue numb ers for this pro cess are at

n

most n The analogue of Leb esgues theorem in this case would then read

kf S f kCn E f

n n

Thus S f f whenever n E f and Jacksons theorem tells us when this will

n n

happ en If f is twice continuously dierentiable then the FourierLegendre series for f

converges uniformly to f on

The ChristoelDarb oux Identity

It would also b e of interest to have a closed form for K t x That this is indeed always

n

y sequence of orthogonal p olynomials is a very imp ortant fact p ossible for an

Using our original notation let Q b e the sequence of monic orthogonal p olynomials

n

b

corresp onding to a given weight w and let Q b e the orthonormal counterpart of Q

n n

Orthogonal Polynomials

p

b

in other words Q Q where h Q Q i It will help things here if you recall

n n n n n n

from Observation on page that b

n

n n

As with the Legendre p olynomials each f C a b is represented by the generalized

P

b b

Fourier series h f Q i Q with partial sum op erator

k k

k

Z

b

f t K t x w t dt S f x

n n

a

P

n

b b

where K t x Q t Q x As b efore S is a pro jection onto P in particular

n k k n n

k

for every n S

n

Theorem ChristoelDarb oux The kernel K t x can b e written

n

n

X

b b b b

Q t Q x Q t Q x

n n n n

b b

Q t Q x

k k n

n

t x

k

Proof We b egin with the standard recurrence formulas

Q tt a Q t b Q t

n n n n n

Q xx a Q x b Q x

n n n n n

where b Multiplying the rst by Q x the second by Q t and subtracting

n n

Q t Q x Q t Q x

n n n n

t x Q t Q xb Q t Q x Q x Q t

n n n n n n n

and again b Ifwe divide b oth sides of this equation by weget

n

Q t Q x Q t Q x

n n n n

n

b b

t x Q t Q x Q t Q x Q x Q t

n n n n n n

n

Thus wemay rep eat the pro cess arriving nally at

n

X

b b

Q t Q x Q t Q x t x Q t Q x

n n n n n n

n

k

b

etc The ChristoelDarb oux identitynow follows by writing Q Q

n n n

And wenowhaveaversion of the DiniLipschitz theorem

Orthogonal Polynomials

Theorem Let f C a b and supp ose that at some p oint x in a b wehave

i f is Lipschitz at x that is jf x f xjK jx xj for some constant K and all

x in a b and

b

ii the sequence Q x is b ounded

n

P

b b

Then the series h f Q i Q x converges to f x

k k

k

Proof First note that the sequence is b ounded Indeed by CauchySchwarz

n

n

h Q Q i h Q xQ i

n n n n

n

kQ k kx kkQ k maxfjaj jbjg

n n n n

Thus c maxfjaj jbjgNow using the ChristoelDarb oux identity

n

n

Z

b

f t f x K t x w t dt S f x f x

n n

a

Z

b

f t f x

b b b b

Q t Q x Q t Q x w t dt

n n n n n

n

t x

a

b b b b

h h Q i Q x hh Q i Q x

n n n n n

n

where htf t f x t x But h is b ounded and continuous everywhere

b

is b ounded and Q x is b ounded by except p ossiblyatx byhyp othesis i

n n

n

b

hyp othesis ii All that remains is to notice that the numbers h h Q i are the generalized

n

Fourier co ecients of the b ounded Riemann integrable function h and so must tend to

zero since in fact theyre even square summable

We end this section with a negative result due to Nikolaev

Theorem There is no weight w such that every f C a b has a uniformly convergent

expansion in terms of orthogonal p olynomials In fact given any w there is always some

f for which kf S f k is unbounded n

Math Problem Set Orthogonal Polynomials

Throughout w denotes a xed p ositive except p ossibly at nitely many p oints Riemann

integrable weight function on a b and we consider the inner pro duct on C a b dened

by

Z

b

hf gi f x g x w x dx

a

and the asso ciated strictly convex norm

Z

q

b

kf k hf f i jf xj w x dx

a

Prove that every inner pro duct norm is strictly convex Sp ecicallyleth i be an

p

hx xi b e the asso ciated norm inner pro duct on a vector space X and let kxk

Showthat

a kx y k kx y k kxk ky k for all x y X the parallelogram identity

xy

b If kxk r ky k and if kx y k then r In particular

xy

r whenever x y

We dene a sequence of p olynomials Q whicharemutually orthogonal relativeto w

n

by setting Q xQ xx a and

Q xx a Q x b Q x for n where

n n n n n

a h xQ Q i h Q Q i and b h xQ Q i h Q Q i

n n n n n n n n n n

and where xQ is shorthand for the p olynomial xQ x

n n

Check that Q is a monic p olynomial of degree exactly n

n

If P is another sequence of orthogonal p olynomials such that P has degree exactly

n n

n for each n show that P Q for some In particular if P is a monic

n n n n n

p olynomial then P Q Hint Cho ose so that P Q P and note

n n n n n n n

that P Q P Conclude that P Q

n n n n n n n

Check that h xQ Q i h Q Q i and conclude that b for each n

n n n n n

Orthogonal Polynomials

Given f C a b and n prove that q P is the leastsquares approximation

n

n

to f out of P with resp ect to w if and only if

n

Z

b

hf q pi f x q x px w x dx

n n

a

for every p P that is if and only if f q P

n n

n

n or more p oints in changes sign at If f C a b but fP show that f q

n

n

a b Hint If not show that there is a p olynomial p P such that f q p

n

n

but f q p ina b Now app eal to the result in problem to arriveata

n

contradiction

n

Show that the leastsquares approximation to f xx out of P relativetow

n

n

xx Q x is q

n

n

Showthat Q has n distinct simple zeros in a b Hint Combine and

n

Given f C a b let p denote the b est uniform approximation to f out of P and

n

n

let q denote the leastsquares approximation to f out of P Show that kf q k

n

n n

kf p k and conclude that kf q k asn

n n

Show that the Chebyshev p olynomials of the rst kind T and of the second kind

n

U satisfy the identities

n

T xU x xU x and x U xxT x T x

n n n n n n

Show that the Chebyshev p olynomials of the second kind U satisfy the recurrence

n

relation

U xxU x U x n

n n n

where U x and U xx Please compare this with the recurrence relation

satised by the T s n

Math Gaussian Quadrature

Numerical integration or quadrature is the pro cess of approximating the value of a denite

R

b

f x w x dx based only on a nite number of values or samples of f much integral

a

like a Riemann sum A linear quadrature formula takes the form

Z

n

b

X

f x w x dx A f x

k k

a

k

where the no des x and the weights A are at our disp osal Note that b oth sides of

k k

the formula are linear in f

Example Consider the quadrature formula

Z

n

X

k

I f f x dx f I f

n

n n

k n

R

If f is continuous then we clearly have I f f as n Why But in the

n

particular case f xx wehave after some simplication

n n

X X

k

I f k

n

n n n n

k k n

That is j I f I f j n In particular wewould need to take n to get

n

n for example and this would require that we p erform over evaluations

of f Wed like a metho d that converges a bit fasterIn other w ords theres no shortage

of quadrature formulaswejustwant faster ones

One reasonable requirement for our prop osed quadrature formula is that it b e exact

for p olynomials of low degree As it happ ens this is easy to come by

Lemma Given w x on a b and no des a x x b there exist unique

n

weights A A such that

n

Z

n

b

X

px w x dx A px

i i

a

i

Gaussian Quadrature

for all p olynomials p P

n

Proof Let b e the Lagrange interp olating p olynomials of degree n associ

n

P

n

px for all p P ated to the no des x x and recall that wehave p

i i n n

i

Hence

Z Z

n

b b

X

px w x dx px x w x dx

i i

a a

i

R

b

x w x That is A dx works To see that this is the only choice supp ose that

i i

a

Z

n

b

X

B px px w x dx

i i

a

i

is exact for all p P and set p

n j

Z

n

b

X

A x w x dx B x B

j j i j i j

a

i

The p oint here is that form a basis for P and integration is linear thus

n n

integration is completely determined by its action on the basisthat is by the n values

A I i n

i i

T

n

Said another way the n pointevaluations ppx satisfy P ker

i i n i

i

fg and it follows that every linear realvalued function on P must b e a linear com

n

bination of the s Heres why Since the x s are distinct P may b e identied with

i i n

n

p px px A linear realvalued function on R bywayoftheisomorphism

n

n

P must then corresp ond to some linear realvalued function on R In other words

n

its given by inner pro duct against some xed vector A A in particular wemust

n

P

n

A px have I p

i i

i

In any case wenowhave our quadrature formula For f C a b we dene I f

n

R

P

b

n

A f x where A x w x dx But notice that the pro of of our last result

i i i i

i

a

suggests an alternate way of writing our quadrature formula Indeed if L f x

n

P

n

f x x is the Lagrange interp olating p olynomial for f of degree n based on

i i

i

the no des x x then

n

Z Z

n n

b b

X X

L f x w x dx f x x w x dx A f x

n i i i i

a a

i i

Gaussian Quadrature

In summary I f I L f I f that is

n n

Z Z

n

b b

X

f x w x dx I f L f x w x dx I f A f x

n n i i

a a

i

where L is the Lagrange interp olating p olynomial of degree n based on the no des

n

x x This formula is obviously exact for f P

n n

Its easy to give a b ound on jI f j in terms of kf k indeed

n

n n

X X

jI f j jA jjf x jkf k jA j

n i i i

i i

By considering a norm one continuous function f satisfying f x sgnA for each i

i i

P

n

n its easy to see that jA j is the smallest constant that works in this inequality

i

i

P

n

In other words jA j n are the Leb esgue numb ers for this pro cess

n i

i

As with all previous settings wewant these numb ers to b e uniformly b ounded

If w x and if f is ntimes continuously dierentiable weeven have an error

estimate for our quadrature formula

Z Z Z Z

n

b b b b

Y

n

jf L f j f L f kf k jx x j dx

n n i

n

a a a a

i

recall the Theorem on page of A Brief Intro duction to Interp olation As it happ ens

the integral on the right is minimized when the x s are taken to b e the zeros of the

i

Chebyshev p olynomial U see Rivlin page

n

The fact that a quadrature formula is exact for p olynomials of low degree do es not by

P

n

itself guarantee that the formula is highly accurate The problem is that A f x may

i i

i

b e estimating a very small quantity through the cancellation of very large quantities So

for example a p ositive function may yield a negative result in this approximate integral

This wouldnt happ en if the A s were all p ositiveand weve already seen how useful

i

p ositivity can b e Our goal here is to further improve our quadrature formula to have this

prop ertyButwehaveyet to takeadvantage of the fact that the x s are at our disp osal

i

Well let Gauss show us the w ay

Theorem Gauss Fixaweight w x on a b and let Q b e the canonical sequence of

n

orthogonal p olynomials relativeto w Given nlet x x b e the zeros of Q these all

n n

Gaussian Quadrature

R

P

b

n

lie in a b and cho ose A A so that the formula f x w x dx A f x

n i i

i

a

is exact for p olynomials of degree less than n Then in fact the formula is exact for all

p olynomials of degree less than n

Proof Given a p olynomial P of degree less than nwemay divide P Q R S

n

where R and S are p olynomials of degree less than n Then

Z Z Z

b b b

P x w x dx Q x Rx w x dx S x w x dx

n

a a a

Z

b

S x w x dx since deg Rn

a

n

X

A S x since deg Sn

i i

i

R

b

But P x Q x Rx S x S x since Q x Hence P x w x dx

i n i i i i n i

a

P

n

A P x for all p olynomials P of degree less than n

i i

i

AmazingBut w ell not really P is of dimension n and we had n numb ers

n

x x and A A to cho ose as wesaw t Said another way the division algorithm

n n

tells us that P Q P P Since Q P kerI the action of I on P

n n n n n n n n n

is the same as its action on a copy of P

n

In still other words since any p olynomial that vanishes at all the x s must b e divisible

i

T

n

by Q and conversely wehave Q P P ker kerI j Thus

n n n n i n P

n

i

I factors through the quotient space P Q P P

n n n n n

Also not surprising is that this particular choice of x s is unique

i

Lemma Supp ose that a x x b and A A are given so that the

n n

R

P

b

n

equation P x w x dx A P x is satised for all p olynomials P of degree less

i i

i

a

than n Then x x are the zeros of Q

n n

Q

n

Proof Let Qx x x Then for kn the p olynomial Q Q has degree

i k

i

n kn Hence

Z

n

b

X

Qx Q x w x dx A Qx Q x

k i i k i

a

i

Gaussian Quadrature

Since Q is a monic p olynomial of degree n which is orthogonal to each Q knwemust

k

have Q Q Thus the x s are actually the zeros of Q

n i n

According to Rivlin the phrase Gaussian quadrature is usually reserved for the sp ecic

R R

L f x dx where f x dx is approximated by quadrature formula whereby

n

L f is the Lagrange interp olating p olynomial to f using the zeros of the nth Legendre

n

p olynomial as no des What a mouthful What is actually b eing describ ed in our version

of Gausss theorem is Gaussiantyp e quadrature

Before computers Gaussian quadrature was little more than a curiosity the ro ots

of Q are typically irrational and certainly not easy to come by By now though its

n

considered a standard quadrature technique In any case we still cant judge the quality

of Gausss metho d without a bit more information

Gaussiantyp e Quadrature

First lets summarize our rather cumb ersome notation

orthogonal approximate

p olynomial zeros weights integral

Q x A I

Q x x A A I

Q x x x A A A I

P

n n

n

where Hidden here is the Lagrange interp olation formula L f f x

n

i i

i

n n n

x The nth denote the Lagrange p olynomials of degree n based on x

n

i

quadrature formula is then

Z Z

n

b b

X

n n

f x w x dx f x L f x w x dx A I f

n n

i i

a a

i

which is exact for p olynomials of degree less than n

n

By way of one example Hermite showed that A n for the Chebyshev weight

k

n

w x x on Remarkably A do esnt dep end on k The quadrature k

Gaussian Quadrature

formula in this case reads

Z

n

X

k f x dx

p

f cos

n n

x

k

Or if you prefer

Z

n

X

k k

f x dx f cos sin

n n n

k

Why You can nd full details in Natansons Constructive Function TheoryVol I I I

The key result due to Stieltjes is that I is p ositive

n

R

P

b

n n n

n

Lemma A A and A w x dx

n

i

i

a

Proof The second assertion is obvious since I I For the rst x j n

n

n

and notice that is of degree n nThus

j

Z

n

b h i h i

X

n n n n n n n

x w x dx A x A h i

j i j i j j j

a

i

n n

b ecause x

ij

j i

Now our last calculation is quite curious what weveshown is that

Z Z

b b h i

n n n

A x w x dx x w x dx

j j j

a a

The same calculation as ab ovealsoproves

n n

i for i j Corollary h

j i

n n

whenever A it follows that I f ispositive that is I f Since A

n

n n

f The second assertion in Lemma tells us that the I s are uniformly b ounded

n

Z

n

b

X

n

w x dx kf k jI f jkf k A

n

i

a

i

R

b

f x w x dx itself Given all of this and this is the same b ound that holds for I f

a

proving that I f I f is a piece of cake The following result is again due to Stieltjes

n

a la Leb esgue

Gaussian Quadrature

R

b

w x dx E f In particular Theorem In the ab ove notation jI f I f j

n n

a

I f I f for evey f C a b

n

Proof Let p b e the b est uniform approximation to f out of P Then since

n

I p I p wehave

n

f p j jI f p j jI f I f jjI

n n

Z

n

b

X

n

w x dx kf p k kf p k A

i

a

i

Z Z

b b

kf p k w x dx E f w x dx

n

a a

Computational Considerations

Youve probably b een asking yourself How do I nd the A s without integrating Well

i

rst lets recall the denition In the case of Gaussiantyp e quadrature wehave

Z Z

b b

Q x

n

n n

A x w x dx w x dx

i i

n n

a a

x x Q x

n

i i

b ecause W is the same as Q herethe x s are the zeros of Q Next consider the

n i n

function

Z

b

Q t Q x

n n

x w t dt

n

t x

a

Since t x divides Q t Q x note that is actually a p olynomial of degree at most

n n n

n and that

Z

b

Q t

n

n n n

Q x x w t dt A

n

n

i i i

n

a

t x

i

n n

Now Q x is readily available we just need to compute x

n

n

i i

Claim The s satisfy the same recurrence formula as the Q s

n n

xx a x b x n

n n n n n

but with dierent starting values

Z

b

x and x w x dx

a

Gaussian Quadrature

Proof The formulas for and are obviously correct since Q x andQ x

x a We only need to check the recurrence formula itself

Z

b

Q t Q x

n n

x w t dt

n

t x

a

Z

b

t a Q t b Q t x a Q xb Q x

n n n n n n n n

w t dt

t x

a

Z Z

b b

Q t Q x Q t Q x

n n n n

x a w t dt b w t dt

n n

t x t x

a a

x a x b x

n n n n

R

b

since Q t w t dt

n

a

Of course the derivatives Q satisfy a recurrence relation of sorts to o

n

Q x Q xx a Q x b Q x

n n n

n n n

Q

n n

n

But Q x can b e computed without knowing Q x Indeed Q x x x

n

n n

i i

i

Q

n n n

x so wehave Q x x

n

i j i

j i

n

The weights A orChristoel numbers together with the zeros of Q are tabulated

n

i

inavariety of standard cases See for example Handbook of Mathematical Functions with

Formulas Graphs and Tablesby Abramowitz and Stegun eds In practice of course its

enough to tabulate data for the case a b

Applications to Interp olation

Although L f isnt typically a go o d uniform approximation to f ifweinterp olate at the

n

zeros of an orthogonal p olynomial Q then L f will b e a go o d approximation in the

n n

kk or kk norm generated by the corresp onding weight w Sp ecicallyby rewording

R

b

and our earlier results its easy to get estimates for each of the errors jf L f j w

n

a

R

b

jf L f j w We use essentially the same notation as b efore except nowwe take

n

a

n

X

n n

L f f x

n

i i

i

n n n

is of degree n This leads to a where x x are the ro ots of Q and

n

n

i

quadrature formula thats exact on p olynomials of degree less than n

Gaussian Quadrature

n n

As weve already seen are orthogonal and so kL f k may b e computed

n

n

exactly

R

b

w x dx Lemma kL f k kf k

n

a

Proof Since L f is a p olynomial of degree nn wehave

n

Z

b

kL f k L f w x dx

n n

a

n n

X X

n n n n

A f x x

j i i j

j i

n

h i

X

n n

A f x

j j

j

Z

n

b

X

n

kf k A kf k w x dx

j

a

j

R

b

Please note that we also have kf k kf k w x dx that is this same estimate

a

holds for kf k itself

As usual once wehave an estimate for the norm of an op erator we also havean

analogue of Leb esgues theorem

R

b

Theorem kf L f k w x dx E f

n n

a

Proof Here we go againLet p b e the b est uniform approximation to f out of P and

n

use the fact that L p p to see that

n

k kf p k kL f p k kf L f

n n

Z Z

b b

kf p k w x dx kf p k w x dx

a a

Z

b

E f w x dx

n

a

Hence if weinterp olate f C a b at the zeros of Q then L f f in kk

n n

norm The analogous result for the kk norm is noweasy

Gaussian Quadrature

R R

b b

w x dx E f jf x L f xj w x dx Corollary

n n

a a

Proof We apply the CauchySchwarz inequality

Z Z

b b

p p

jf x L f xj w x dx jf x L f xj w x w x dx

n n

a a

Z Z

b b

jf x L f xj w x dx w x dx

n

a a

Z

b

w x dx E f

n

a

R R

b b

Essentially the same device allows an estimate of f x dx in terms of f x w x dx

a a

whichmay b e easier to compute

R

b

Corollary If w x dx is nite then

a

Z Z

b b

p

p

jf x L f xj dx jf x L f xj w x dx

n n

w x

a a

Z Z

b b

dx jf x L f xj w x dx

n

w x

a a

Z Z

b b

E f w x dx dx

n

w x

a a

In particular the Chebyshev weight satises

Z Z

p

dx

p

and x dx

x

Thus interp olation at the zeros of the Chebyshev p olynomials of the rst kind would

provide go o d simultaneous approximation in each of the norms kk kk andkk

The Moment Problem

Given a p ositive continuous weight function w xona b the number

Z

b

k

x w x dx

k

a

is called the k th moment of w Inphysical terms if we think of w x as the densityofa

thin ro d placed on the interval a b then is the mass of the ro d is its center of

Gaussian Quadrature

mass is its moment of inertia ab out and so on In probabilistic terms if

then w is the probability density function for some random variable is the exp ected

value or mean of this random variable and is its variance The moment problem

or problems really concern the inverse pro cedure What can b e measured in real life are

the momentscan the moments b e used to nd the density function

Questions Do the moments determine w Do dierentweights have dierent mo

ment sequences If we knew the sequence could we nd w Howdowe tell if a

k

given sequence is the moment sequence for some p ositiveweight Do sp ecial

k

weights give rise to sp ecial moment sequences

Nowweve already answered one of these questions The Weierstrass theorem tells us

that dierentweights have dierent moment sequences Said another wayif

Z

b

k

x w x dx for all k

a

R

b

then w Indeed by linearitythissays that px w x dx for all p olynomials p

a

R

b

w x dx Why The remaining questions are harder which in turn tells us that

a

to answer Well settle for simply stating a few p ertinent results

n

Given a sequence of numbers we dene the nth dierence sequence by

k k

k k

k k k

n n n

n

k k k

For example More generally induction will show that

k k k k

n

X

n

n i

k i k

i

i

In the case of a weight w on the interval this sum is easy to recognize as an integral

Indeed

Z Z

n n

X X

n n

i k n k i i

x x w x dx x w x dx

k i

i i

i i

n

In particular if w is nonnegative then wemust have for every n and k This

k

observation serves as motivation for

Gaussian Quadrature

Theorem The following are equivalent

a is the moment sequence of some nonnegativeweight function w on

k

n

b for every n and k

k

n

c a a a whenever a a x a x for all x

n n n

The equivalence of a and b is due to Hausdor A real sequence satisfying b or

c is sometimes said to b e p ositive denite

Now dozens of mathematicians worked on various asp ects of the moment problem

Chebyshev Markov Stieltjes CauchyRieszFrec het and on and on And several of

R

b

w t

them in particular Cauchy and Stieltjes noticed the imp ortance of the integral dt

xt

a

in attacking the problem Compare this expression to Cauchys integral formula It was

Stieltjes however who gave the rst complete solution to such a problemdevelopinghis

R

b

dW t

own integral by considering his own variety of continued fractions and planting

a xt

the seeds for the study of orthogonal p olynomials while he was at itW e will attempt to

at least sketch a few of these connections

To b egin lets x our notation To simpliy things we supp ose that were given a

nonnegativeweight w xonasymmetric interval a a and that all of the moments of

w are nite We will otherwise stick to our usual notations for Q the Gaussiantyp e

n

quadrature formulas and so on Next we consider the momentgenerating function

Z

a

X

w t

k

dt Lemma If x a a then

k

x t x

a

k

k

X

t

and the sum converges uniformly b ecause Proof

k

x t x tx x

k

jtxjajxj Now just multiply by w t and integrate

By way of an example consider the Chebyshev weight w x x on

For xwehave

Z

dt

p p

set t u u

x t t x

x x

Gaussian Quadrature

x x x

using the binomial formula Thus weve found all the moments

Z

dt

p

t

Z

n

t dt

p

n

t

Z

n

t dt n

p

n

n

n

t

R

a

w t

Stieltjes proved much more The integral dt is actually an analytic function of

xt

a

x in C n a a In any case since x a a weknow that is continuous on a a

xt

In particular we can apply our quadrature formulas and Stieltjes theorem p to

write

Z

n

n

a

X

A w t

i

dt lim

n

n

x t

a

x x

i

i

and these sums are recognizable

n

n

X

x A

n

i

Lemma

n

Q x

n

x x

i

i

n

forany iwemay app eal to partial Proof Since has degree n and x

n n

i

fractions to write

n

X

x x c

n n i

n n n

Q x

n

x x x x x x

n

i

i

where c is given by

i

n

x x

n n

n n

i

c x x A

i

i i

n

n

Q x

n

Q x

xx

n

i

i

Now heres where the continued fractions come in Stieltjes recognized the fact that

b x

n

b

Q x

n

x a

x a

b

n

x a n

Gaussian Quadrature

R

b

w t dt More generally induction will which can b e proved by induction where b

a

show that the nth convergent of a continued fraction can b e written as

A p

n

p

B

n q

q

p

n

q

n

by means of the recurrence formulas

A B

A p B q

A q A p A B q B p B

n n n n n n n n n n

where n Please note that A and B satisfy the same recurrence formula

n n

but with dierent starting values as is the case with and Q

n n

Again using the Chebyshev weightasanexampleforxwehave

Z

dt

p p

x x t t

x

x

x

since a for all n b and b forn In other words weve just found

n n

a continued fraction expansion for x

App endix

Finally here is a brief review of some of the fancier bits of linear algebra used in this

chapter To b egin we discuss sums and quotients of vector spaces

Each subspace M of a nitedimensional X induces an equivalence relation on X by

x y x y M

Standard arguments show that the equivalence classes under this relation are the cosets

translates x M x X That is

x M y M x y M x y

Gaussian Quadrature

Equally standard is the induced vector arithmetic

x M y M x y M and x M xM

where x y X and R The collection of cosets or equivalence classes is a vector

space under these op erations its denoted XM and called the quotient of X by M Please

note the the zero vector in XM is simply M itself

Asso ciated to the quotient space XM is the quotientmap q xx M Its easy

to check that q X XM isavector space homomorphism with kernel M Why

Next we recall the isomorphism theorem

Theorem Let T X Y b e a linear map b etween nitedimensional vector spaces and

let q X X ker T b e the quotient map Then there exists a unique into isomorphism

S X ker T Y satisfying S q x T x for every x X

Proof Since q maps onto X ker T its legal to dene a map S X ker T Y by

setting S q x T xforx X Please note that S is welldened since

T xT y T x y x y ker T

q x y q xq y

Its easy to see that S is linear and so precisely the same argumentasabove shows that S

is onetoone

Corollary Let T X Y b e a linear map b etween nitedimensional vector spaces

Then the range of T is isomorphic to X ker T

Math The Mun tz Theorems

For several weeks nowwevetaken advantage of the fact that the monomials xx

have dense linear span in C What if anything is so sp ecial ab out these particular

P

n

k

are they dense powers How ab out if we consider p olynomials of the form a x

k

k

n

to o More generally what can b e said ab out the span of a sequence of monomials x

where Of course well have to assume that but its not hard

P

n

k

a x to see that we will actually need for otherwise each of the p olynomials

k

k

vanishes at x and so has distance at least from the constant function for example

If the s are integers its also clear that well havetohave as n But

n n

what else is needed The answer comes to us from Mun tz in You sometimes see

the name Otto Szasz asso ciated with Mun tzs theorem b ecause Szasz proved a similar

theorem at nearly the same time

n

Theorem Let Then the functions x have dense linear span

P

in C if and only if and

n

n

What Mun tz is trying to tell us here is that the s cant get big to o quickly In

n

P

n

k

particular the p olynomials of the form a x are evidently not dense in C On

k

k

the other hand the s dont havetobeunb ounded indeed Mun tzs theorem implies an

n

earlier result of Bernstein from If K some constant then

x x have dense linear span in C

Before we give the pro of of Mun tzs theorem lets invent a bit of notation We write

n

X

k

a x a a R X

k n n

k

and given f C we write distf X to denote the distance from f to the space

n

S

n

spanned byx x Lets also write X X That is X is the linear span of

n

n

n

the entire sequence x The question here is whether X is dense and well address

n

the problem by determining whether distf X as n for every f C

n

m

If we can show that each xed p ower x can b e uniformly approximated by a linear

n

combination of x s then the Weierstrass theorem will tell us that X is dense in C

Muntz Theorems

m

How Surprisingly the numb ers distx X can b e estimated Our pro of wont give

n

P

the b est estimate but it will showhow the condition comes into the

n

n

picture

n

Y

m

m

Lemma Let m Then distx X

n

k

k

Proof Wemay certainly assume that m for any n Given this we inductively

n

m

dene a sequence of functions by setting P xx and

Z

n n

P x m x t P t dt

n n n

x

for n For example

Z

m m m

P x m x t t dt x t x x

x

x

P

n

m

k

By induction each P is of the form x a x for some scalars a

n k k

k

Z

n n

P t dt t P x m x

n n n

x

Z

n

X

m

k n n

dt t a t t m x

k n

x

k

n

X

a

k

m

n k n

x m x x x

n

n k

k

m

Finally kP k and kP kj jkP k b ecause

n n

n

Z

m j mj

n

n n n

x dt t j mj x

n

n n

x

Thus

n

Y

m

m

distx X kP k

n n

k

k

The preceding result is due to v Golitschek A slightly b etter estimate also due to

Q

n

jm j

k

m

v Golitschek is distx X

n

k

m

k

Nowa wellknown fact ab out innite pro ducts is that for p ositive a s the pro duct

k

P Q

diverges to if and only if the series a diverges to if and only a

k k

k k

Muntz Theorems

Q Q

n

m

if the pro duct a diverges to In particular if and only

k

k k

k

P P

n

m

if That is distx X if and only if This proves the

n

k k

k k

backward direction of Mun tzs theorem

Well prove the forward direction of Mun tzs theorem byproving a version of Mun tzs

theorem for the space L For our purp oses L denotes the space C

endowed with the norm

Z

kf k jf xj dx

although our results are equally valid in the real space L consisting of square

integrable Leb egue measurable functions In the latter case we no longer need to assume

n

that but we do need to assume that each in order that x be

n

integrable on

 n

f to the span of x x x can b e computed Remarkably the distance from

exactly in the L norm For this well need some more notation Given linearly indep endent

vectors f f in an inner pro duct space we call

n

h f f i h f f i

n

det h f f i Gf f

i j n

ij

h f f i h f f i

n n n

the Gram determinant of the f s

k

Lemma Gram Let F b e a nite dimensional subspace of an inner pro duct space V

and let g V n F Then the distance d from g to F is given by

Gg f f

n

d

Gf f

n

where f f is any basis for F

n

P

n

Proof Let f a f b e the b est approximation to g out of F Then since g f

i i

i

is orthogonal to F wehave in particular h f g i h f g i for all j that is

j j

n

X

a h f f i h f g i j n

i j i j

i

Since this system of equations always has a unique solution a a wemust have

n

and so the formula in our Lemma at least makes sense Gf f

n

Muntz Theorems

Next notice that

d h g f g f i h g f g i h g g ihg f i

in other words

n

X

d a h g f i h g g i

i i

i

Now consider and as a system of n equations in the n unknowns a a

n

and d in matrix form wehave

h g f i h g f i h g g i

d

n

h f f i h f f i h f g i

a

n

a

h f f i h f f i h f g i

n

n n n n

Solving for d using Cramers rule gives the desired result expanding along the rst

column shows that the matrix of co ecients has determinant Gf f while the

n

matrix obtained by replacing the d column by the righthand side has determinant

Gg f f

n

Note By our last Lemma and induction every Gram determinant is p ositive

 n

In what follows we will still use X to denote the span of x x but nowwell

n

write dist f X to denote the distance from f to X in the L norm

n n

Theorem Let m for k Then

k

n

Y

jm j

k

m

p

dist x X

n

m

m

k

k

Proof The pro of is based on a determinant formula due to Cauchy

a b a b

n

Y Y

a b a a b b

i j i j i j

ij ij

a b a b

n n n

If we consider each of the a s and b s as variables then each side of the equation is a

i j

p olynomial in a a b b Why Nowtherighthand side clearly vanishes if

n n

Muntz Theorems

a a or b b for some i j but the lefthand side also vanishes in any of these cases

i j i j

Thus the righthand side divides the lefthand side But b oth p olynomials have degree

n ineachofthea s and b s Why Thus the lefthand side is a constantmultiple

i j

of the righthand side Toshow that the constantmust b e write the lefthand side as

a b a b

a b a b

n

a b a b

Y

a b a b

n

a b

i j

i j

a b a b

n n n n

a b a b

n n n

and now take the limit as b a b a etc The expression ab ove tends to

Q

a a as do es the righthand side of Cauchys formula

i j

i j

R

p q pq

Now h x x i x dx for p q so

pq

Q

i j

ij

n 

Q

det x Gx

i j i j

ij

ij

m

 n

with a similar formula holding for Gx x x Substituting these expressions into

our distance formula and taking square ro ots nishes the pro of

Nowwe can determine exactly when X is dense in L For easier comparison to

the C casewe supp ose that the s are nonnegative

n

n

Theorem Let Then the functions x have dense linear span

P

in L if and only if

n

n

Q Q P

n n

m

m

then each of the pro ducts and Proof If

k k n

n k k

m

converges to some nonzero limit for any m not equal to any Thus dist x X

k n

n

as n for any m k In particular the functions x cannot have

k

dense linear span in L

Q Q P

n n

m

m

then diverges to while Converselyif

k k n

n k k

m

diverges to Thus dist x X as n for every m Since the

n

p olynomials are dense in L this nishes the pro of

Finallywe can nish the pro of of Mun tzs theorem in the case of C Supp ose

n

that the functions x have dense linear span in C Then since kf k kf kit

Muntz Theorems

n

follows that the functions x mustalsohave dense linear span in L Why

P

Hence

n

n

Just for go o d measure heres a second pro of of the backward direction for C

P

based on the L version Supp ose that and let m Then

n

n

Z Z

n n

x x

X X

a

k

m m

k k

t dt t dt x a x

k

m

k

k k

Z

n

X

a

k

m

k

t t dt

m

k

k

Z

n

X

a

k

m

k

A

t dt t

m

k

k

P

k

Now the functions x have dense linear span in L b ecause

n

n

Thus we can nd a s so that the righthand side of this inequality is less than some

k

Since this estimate is indep endentof xweveshown that

n

X

m

k

x a x max

k

x

k

P

Application Let with and let f be a continuous

n

n

function on for which c lim f t exists Then f can b e uniformly approximated

t

t

n

by nite linear combinations of the exp onentials e

n

Proof The function g xf log x for x and g c is continuous on

t

Thus given we can nd In other words g e f t for each t

n and a a such that

n

n n

X X

t

k k

a e f t max a x g x max

k k

t x

k k

Math The StoneWeierstrass Theorem

To b egin an algebra is a vector space A on which there is a multiplication f g fg

from A A into A satisfying

i fgh f gh for all f g h A

ii f g hfg fh and f g h fg gh for all f g h A

iii fgf g f g for all scalars and all f g A

In other words an algebra is a ring under vector addition and multiplication together

with a compatible scalar multiplication The algebra is commutative if

iv fg gf for all f g A

And wesay that A has an identity element if there is a vector e A suchthat

v fe ef f for all f A

In case A is a normed vector space we also require that the norm satisfy

vi kfgkkf kkg k

this simplies things a bit and in this case we refer to A as a normed algebra If a

normed algebra is complete we refer to it as a Banach algebra Finally a subset B of an

algebra A is called a subalgebra of AifB is itself an algebra under the same op erations

that is if B is a vector subspace of A which is closed under multiplication

If A is a normed algebra then all of the various op erations on A or A A are

continuous For example since

k fg hk k kfg fk fk hk k kf kkg k k kk kkf hk

it follows that multiplication is continuous How In particular if B is a subspace or

B the closure of B is also a subspace or subalgebra of A subalgebra of A then

Examples

n

If we dene multiplication of vectors co ordinatewise then R is a commutative

Banach algebra with identity the vector when equipp ed with the norm

kxk max jx j

i

in

StoneWeierstrass

n

Its not hard to identify the subalgebras of R among its subspaces For example the

subalgebras of R are fx x Rg fyy Rgand fx xx Rg along

with f g and R

Given a set X we write B X for the space of all b ounded realvalued functions on

X Ifwe endow B X with the sup norm and if we dene arithmetic with functions

p ointwise then B X is a commutative Banach algebra with identity the constant

function The constant functions in B X form a subalgebra isomorphic in every

sense of the word to R

If X is a metric or top ological space then wemay consider C X the space of all

continuous realvalued functions on X Ifwe again dene arithmetic with functions

p ointwise then C X is a commutative algebra with identity the constant function

The b ounded continuous functions on X written C X C X B X form a

b

closed subalgebra of B X If X is compact then C X C X In other words

b

if X is compact then C X is itself a closed subalgebra of B X and in particular

C X is a Banach algebra with identity

The p olynomials form a dense subalgebra of C a b The trig p olynomials form a

dense subalgebra of C Thesetwo sentences summarize Weierstrasss two classical

theorems in mo dern parlance and form the basis for Stones version of the theorem

Using this new language wemay restate the classical W eierstrass theorem to read

If a subalgebra A of C a b contains the functions exand f xx then A is

dense in C a b Any subalgebra of C a b containing and x actually contains all the

p olynomials thus our restatementofWeierstrasss theorem amounts to the observation

that any subalgebra containing a dense set is itself dense in C a b

Our goal in this section is to prove an analogue of this new version of the Weierstrass

theorem for subalgebras of C X where X is a compact metric space In particular we

will want to extract the essence of the functions and x from this statement That is we

seek conditions on a subalgebra A of C X that will force A to b e dense in C X The

key role played by and x in the case of C a b is that a subalgebra containing these

tain a much larger set of functions But since we cant two functions must actually con

b e assured of anything remotely like p olynomials living in the more general C X spaces

StoneWeierstrass

we mightwanttochange our p oint of view What we really need is some requirementon

a subalgebra A of C X that will allowusto construct a wide variety of functions in A

And if A contains a suciently richvariety of functions it might just b e p ossible to show

that A is dense

Since the two replacement conditions wehave in mind make sense in any collection of

realvalued functions we state them in some generality

Let A b e a collection of realvalued functions on some set X Wesay that A separates

p oints in X if given x y X there is some f A suchthat f x f y Wesaythat

A vanishes at no p ointof X if given x X there is some f A such that f x

Examples

The single function f xx clearly separates p oints in a b and the function ex

obviously vanishes at no p ointina b Any subalgebra A of C a b containing

these two functions will likewise separate p oints and vanish at no p ointina b

The set E of even functions in C fails to separate p oints in indeed

f xf x for anyeven function However since the constant functions are even

E vanishes at no p ointof Its not hard to see that E is a prop er closed

f xx subalgebra of C The set of o dd functions will separate p oints since

is o dd but the o dd functions all vanish at The set of o dd functions is a prop er

closed subspace of C although not a subalgebra

The set of all functions f C for which f is a prop er closed subalgebra

of C In fact this set is a maximal in the sense of containment prop er closed

subalgebra of C Note however that this set of functions do es separate p oints

in again b ecause it contains f xx

Its easy to construct examples of nontrivial closed subalgebras of C X Indeed

given any closed subset X of X the set AX ff C X f vanishes on X g is

a nonempty prop er subalgebra of C X Its closed in any reasonable top ology on

C X b ecause its closed under p ointwise limits Subalgebras of the typ e AX are

of interest b ecause theyre actually ideals in the ring C X That is if f C X

and if g AX then fg AX

StoneWeierstrass

As these few examples illustrate neither of our new conditions taken separatelyis

enough to force a subalgebra of C X to b e dense But b oth conditions together turn

out to b e sucient In order to b etter appreciate the utility of these new conditions lets

isolate the key computational to ol that they p ermit within an algebra of functions

Lemma Let A b e an algebra of realvalued functions on some set X and supp ose that

A separates p oints in X and vanishes at no p ointofX Then given x y X and a

b Rwe can nd an f A with f xa and f y b

e

Proof Given any pair of distinct p oints x y X the set A f f xfy f Ag

e

is a subalgebra of R IfA separates p oints in X then A is evidently neither f g nor

fx xx RgIfA vanishes at no p oint then fx x Rg and fyy Rg are

e

b oth excluded Thus A R That is for any a b R there is some f A for which

f xf y a b

Nowwe can state Stones version of the Weierstrass theorem for compact metric

spaces It should b e p ointed out that the theorem as stated also holds in C X when

X is a compact Hausdor top ological space with the same pro of but do es not hold for

algebras of complexvalued functions over C More on this later

StoneWeierstrass Theorem real scalars Let X b e a compact metric space and let

A b e a subalgebra of C X IfA separates p oints in X and vanishes at no p ointof X

then A is dense in C X

What Cheney calls an embryonic version of this theorem app eared in as a small

part of a massive page pap erLater v ersions app earing in and b enetted

from the work of the great Japanese mathematician Kakutani and were somewhat more

palatable to the general mathematical public But no matter whichversion you consult

youll nd them dicult to read For more details I would recommend you rst consult

Follands Real Analysis or Simmonss Topology and Modern Analysis

As a rst step in attacking the pro of of Stones theorem notice that if A satises the

AWhy Thus wemay assume that conditions of the theorem then so do es its closure

A is actually a closed subalgebra of C X and prove instead that A C X Now the

closed subalgebras of C X inherit more structure than you might rst imagine

StoneWeierstrass

Theorem If A is a subalgebra of C X and if f A then jf j A Consequently A is

a sublattice of C X

Proof Let and consider the function jtj on the interval kf k kf k By the

P

n

k

Weierstrass theorem there is a p olynomial pt a t such that jtj pt for

k

k

all jtjkf k In particular notice that jpj ja j

for all pf x jf xj Now since jf xjkf k for all x X it follows that

n

x X Butpf x pf x where pf a a f a f and the function

n

n

ja j jf xjg x g a f a f A since A is an algebra Thus

n

for all x X In other words for each we can supply an element g A suchthat

kjf jg k That is jf j A

The statement that A is a sublattice of C X means that if were given f g A then

maxff gg A and minff gg A to o But this is actually just a statement ab out real

numb ers Indeed since

maxfa bg a b ja bj and minfa bg a b ja bj

it follows that a subspace of C X is a sublattice precisely when it contains the absolute

values of all its elements

The p oint to our last result is that if were given a closed subalgebra A of C X then

A is closed in every sense of the word Sums pro ducts absolute values maxs and

mins of elements from A and even limits of sequences of these are all backinAThisis

precisely the sort of freedom well need if we hop e to show that A C X

Please notice that we could haveavoided our app eal to the Weierstrass theorem in this

last result Indeed we really only need to supply p olynomial approximations for the single

function jxj on and this can b e done directlyFor example we could app eal instead

p

to the binomial theorem using jxj x The resulting series can b e shown

to converge uniformly on By sidestepping the classical Weierstrass theorem it

b ecomes a corollary to Stones version rather than the other way around

Nowwere ready for the pro of of the StoneWeierstrass theorem As weve already

p ointed out wemay assume that were given a closed subalgebra subspace and sublattice

StoneWeierstrass

A of C X and wewant to show that A C X Well break the remainder of the pro of

into two steps

Given f C X x X and there is an element g A with g xf x Step

x x

and g y fy for all y X

x

From our computational Lemma we know that for each y X y xwe can nd

an h A so that h xf xandh y f y Since h f is continuous and vanishes

y y y y

at b oth x and y the set U ft X h t ft g is op en and contains b oth x and y

y y

Thus the sets U form an op en cover for X Since X is compact nitely many U s

y y x y

suce say X U U Now set g maxfh h g Because A is a lattice

y y x y y

n n

wehave g A Note that g xf x since each h agrees with f at xAndg f

x x y x

i

since given y xwehave y U for some i and hence g y h y fy

y x y

i i

Step Given f C X and there is an h A with kf hk

From Step for each x X we can nd some g A such that g xf xand

x x

g y fy for all y X And nowwereverse the pro cess used in Step For each x

x

the set V fy X g y fy g is op en and contains x Again since X is compact

x x

X V V for some x x This time set h minfg g gAAs

x x m x x

m m

b efore hy fy for all y since each g do es so and hy fy for all y since

x

i

at least one g do es so

x

i

The conclusion of Step is that A is dense in C X but since A is closed this means

that A C X

Corollary If X and Y are compact metric spaces then the subspace of C X Y spanned

by the functions of the form f x y g x hy g C X h C Y is dense in C X Y

n

Corollary If K is a compact subset of R then the p olynomials in nvariables are

dense in C K

Applications to C

In many texts the StoneWeierstrass theorem is used to show that the trig p olynomials are

dense in C One approach here mightbetoidentify C with the closed subalgebra of

f Probably easier though C consisting of those functions f satisfying f

StoneWeierstrass

i

is to identify C with the continuous functions on the unit circle T fe Rg

fz C jz j g in the complex plane using the identication

it

f C g C T where g e f t

Under this corresp ondence the trig p olynomials in C match up with certain p olyno

it it

mials in z e and z e But as weve seen even if we start with realvalued trig

p olynomials well end up with p olynomials in z and z having complex co ecients

Given this it mightmake more sense to consider the complexvalued continuous func

tions on T Well write C T to denote the complexvalued continuous functions on

C

T and C T to denote the realvalued continuous functions on T Similarly C is

R

C

the space of complexvalued p erio dic functions on Rwhile C stands for the real

R

valued p erio dic functions on RNow under the identication we made earlier wehave

C T C and C T C The complexvalued trig p olynomials in C now match

C R

C R C

it it

up with the full set of p olynomials with complex co ecients in z e and z e Well

use the StoneWeierstrass theorem to show that these p olynomials are dense in C T

C

Now the p olynomials in z obviously separate p oints in T and vanish at no p ointof T

Nevertheless the p olynomials in z alone are not dense in C T To see this heres a pro of

C

z cannot b e uniformly approximated by p olynomials in z First supp ose that that f z

P

n

k

were given some p olynomial pz c z Then

k

k

Z Z Z

n

X

it it it ik t

it

f e pe dt e pe dt c e dt

k

k

and so

Z Z

it it it

it it

f e f e dt f e f e pe dt

b ecause f z f z jf z j Now taking absolute values weget

Z

it it

f e pe dt kf pk

That is kf pk for any p olynomial p

We mightaswell pro ceed in some generality Given a compact metric space X well

write C X for the set of all continuous complexvalued functions f X C and we C

StoneWeierstrass

norm C X by kf k max jf xj where jf xj is the mo dulus of the complex number

C

xX

f x of course C X isaBanach algebra over C In order to make it clear which eld

C

of scalars is involved well write C X for the realvalued memb ers of C X Notice

R C

though that C X is nothing other than C X with a new name

R

More generallywell write A to denote an algebra over C of complexvalued func

C

tions and A to denote the realvalued members of A Its not hard to see that A is

R C R

then an algebra ov er Rofrealvalued functions

f xf x the complexconjugate of Nowiff is in C X then so is the function

C

f x This puts

f f Ref f and Imf f

i

the real and imaginary parts of f inC X too Converselyifg h C X then

R R

g ih C X

C

This simple observation gives us a hintastohowwemight apply the StoneWeierstrass

theorem to subalgebras of C X Given a subalgebra A of C X supp ose that we could

C C C

prove that A is dense in C X Then given any f C X we could approximate Ref

R R C

and Imf by elements g h A But since A A this means that g ih A and

R R C C

g ih approximates f That is A is dense in C X GreatAnd what did w e really use

C C

here Well weneed A to contain the real and imaginary parts of most functions in

R

C X If we insist that A separate p oints and vanish at no p oint then A will contain

C C R

most of C X And to b e sure that we get b oth the real and imaginary parts of each

R

elementof A well insist that A contain the conjugates of each of its memb ers f A

C C C

whenever f A That is well require that A be selfconjugate or as some authors

C C

say selfadjoint

StoneWeierstrass Theorem complex scalars Let X b e a compact metric space and

let A b e a subalgebra over C ofC X If A separates p oints in X vanishes at no

C C C

p ointof X and is selfconjugate then A is dense in C X

C C

Proof Again write A for the set of realvalued members of A Since A is self

R C C

conjugate A contains the real and imaginary parts of every f A

R C

f A and Imf f A f f Ref

R R

i

StoneWeierstrass

Moreover A is a subalgebra over Rof C X In addition A separates p oints in X

R R R

and vanishes at no p ointof X Indeed given x y X and f A with f x f y

C

wemust have at least one of Ref x Ref y orImf x Imf y Similarly f x

means that at least one of Ref x orImf x holds That is A satises the

R

hyp otheses of the realscalar version of the StoneWeierstrass theorem Consequently A

R

is dense in C X

R

Now given f C X and take g h A with kg Ref k and

C R

kh Imf k Then g ih A and kf g ihk Thus A is dense in

C C

C X

C

Corollary The p olynomials with complex co ecients in z and z are dense in C TIn

C

other words the complex trig p olynomials are dense in C

C

Note that it follows from the complexscalar pro of that the real parts of the p olyno

mials in z and z that is the real trig p olynomials are dense in C T C

R

R

Corollary The real trig p olynomials are dense in C

R

Application Lipschitz Functions

In most Real Analysis courses the classical Weierstrass theorem is used to provethat

C a b is separable Likewise the StoneWeierstrass theorem can b e used to showthat

C X is separable where X is a compact metric space While wewont haveanything quite

so convenient as p olynomials at our disp osal we do at least have a familiar collection of

functions to work with

Given a metric space X d and Kwell write lip X to denote the

K

collection of all realvalued Lipschitz functions on X with constant at most K that is

f X R is in lip X ifjf x f y jKdx y for all x y X Andwell write

K

lipX to denote the set of functions that are in lip X for some K in other words

K

S

lipX lip X Its easy to see that lipX is a subspace of C X in fact if X

K

K

is compact then lipX iseven a subalgebra of C X Indeed given f lip X and

K

g lip X wehave

M

jf xg x f y g y j jf xg x f y g x j jf y g x f y g y j

K kg kjx y j M kf kjx y j

StoneWeierstrass

Lemma If X is a compact metric space then lipX is dense in C X

Proof Clearly lipX contains the constant functions and so vanishes at no p ointof

X To see that lipX separates p ointinX we use the fact that the metric d is Lipschitz

Given x y X the function f xdx y satises f x f y moreover

f lip X since

jf x f y j jdx y dy y jdx y

Thus by the StoneWeierstrass Theorem lipX is dense in C X

Theorem If X is a compact metric space then C X is separable

Proof It suces to show that lipX is separable Why To see this rst notice that

S

lipX E where

K

K

E ff C X kf kK and f lip X g

K

K

Why The sets E are uniformly b ounded and equicontinuous Hence by the Arzela

K

Ascoli theorem each E is compact in C X Since compact sets are separable as are

K

countable unions of compact sets it follows that lipX is separable

As it happ ens the converse is also true whichiswhy this is interesting see Follands

Real Analysis for more details

Theorem If C X is separable where X is a compact Hausdor top ological space then

X is metrizable

A Short List of References

Bo oks

Abramowitz M and Stegun I eds Handbook of Mathematical Functions with For

mulas Graphs and Mathematical Tables Dover

Birkhoff G A SourceBook in Classical Analysis Harvard

Buck R C ed Studies in Modern Analysis MAA

Carothers N L Real Analysis Cambridge

Cheney E W Introduction to Approximation Theory Chelsea

Davis P J Interpolation and Approximation Dover

DeVore R A and Lorentz G G Constructive Approximation SpringerVerlag

Dudley R M Real Analysis and Probability Wadsworth Bro oksCole

Folland G B Real Analysis modern techniques and their applications Wiley

Fox L and Parker I B Chebyshev Polynomials Oxford University Press

Hoffman K Analysis in Euclidean Space PrenticeHall

Jackson D TheoryofApproximation AMS

Jackson D Fourier Series and Orthogonal Polynomials MAA

Korner T W Fourier Analysis Cambridge

Korovkin P P Linear Operators and Approximation Theory Hindustan Publishing

La Vallee Poussin ChJ de Lecons sur lApproximation des Fonctions dune Vari

able Reele GauthierVillars

Lorentz G G Bernstein Polynomials Chelsea

Lorentz G G Approximation of Functions Chelsea

Natanson I Constructive Function Theory vols Ungar

Powell M J D Approximation Theory and Methods Cambridge

Rivlin T J An Introduction to the Approximation of Functions Dover

Rivlin T J The Chebyshev Polynomials Wiley

Rudin W Principles of Mathematical Analysis rd ed McGrawHill

Simmons G F Introduction to Topology and Modern Analysis McGra wHill

reprinted by Rob ert E Krieger Publishing

Articles

BoasRPInequalities for the derivatives of p olynomials Mathematics Magazine

Fisher S Quantitative approximation theory The American Mathematical Monthly

References

Hedrick E R The signicance of Weierstrasss theorem The American Mathemat

ical Monthly

Jackson D The general theory of approximation by p olynomials and trigonometric

sums Bul letin of the American Mathematical Society

Lebesgue H Sur lapproximation des fonctions Bul letin des Sciences Mathematique

Shields A Polynomial approximation The Mathematical Intel ligencer

No

Shohat J A On the development of functions in series of orthogonal p olynomials

Bul letin of the American Mathematical Society

Stone M H Applications of the theory of Bo olean rings to general top ology Trans

actions of the American Mathematical Society

Stone M H A generalized Weierstrass theorem in Studies in Modern AnalysisR

C Buck ed MAA

Van Vleck E B The inuence of Fouriers series up on the development of mathe

matics Science

Weierstrass K Ub er die analytische Darstellbarkeit sogenannter willkurlic her Func

tionen einer reellen Ver anderlichen Sitzungsberichte der Koniglich Preussischen

Akademie der Wissenshcaften zu Berlin

Weierstrass K Sur la p ossibilite dune representation analytique des fonctions

dites arbitraires dune variable reele Journal de Mathematiques Pures et Appliquees