Baire Categories, Continuous Functions, and Approximations
Total Page:16
File Type:pdf, Size:1020Kb
BAIRE CATEGORIES, CONTINUOUS FUNCTIONS, AND APPROXIMATIONS ANDREW CHEN Abstract. This paper is an exploration into continuous functions and how they can be approximated. We first gain insight into the nature of continuous functions and demonstrate why most continuous functions are nowhere differentiable using the Baire Category Theorem. We then discuss two methods of approximating continuous functions: the first of which is using polynomials via the Weierstrass Approximation Theorem, and the second of which is using neural networks via the Universal Approximation Theorem. Contents 1. Introduction 1 2. Continuous Functions and Differentiability 1 2.1. Introducing the Baire Category Theorem 2 2.2. Applying the Baire Category Theorem to the Set of Continuous Functions 4 3. Weierstrass Approximation Theorem 6 4. The Universal Approximation Theorem 9 Acknowledgments 14 References 15 1. Introduction For many years, it was widely thought by mathematicians that most (if not all) continuous functions are differentiable, due to intuitive notions of continuity and differentiability. While calculus was invented in the late 1600s, the first known example of a non-differentiable continuous function was not discovered until around year 1830, by Bernard Bolzano, and the first published example was not presented until 1872, by Karl Weierstrass [9]. Due to later advances in the field of topology, we now know that most continuous functions are in fact nowhere differentiable. Section 2 will explore a proof of this, which utilizes an important theorem in topology called the Baire Category Theorem. We will then explore methods of approximating continuous functions, even those that are nowhere differentiable. Section 3 will explore a probabilistic proof of the Weierstrass Approximation Theorem, which will show that all continuous functions f : [0; 1] ! R can be approximated arbitrarily well using polynomials. The fourth and final section will then explore the ability of machine learning to approximate continuous functions. The main result is the Universal Approximation Theorem, which states that a feedforward neural network used in machine learning can approximate continuous functions arbitrarily well. 2. Continuous Functions and Differentiability The goal of this section is to use analysis to explore the nature of continuous functions. In particular, we will apply the Baire Category Theorem to the space of real-valued continuous functions C(K) on a compact 1 set K to show that most continuous functions are nowhere differentiable. The definitions in this section can be found in most real analysis textbooks such as [8], and the specific proof presented here was inspired by [5]. 2.1. Introducing the Baire Category Theorem. This section will assume that the reader has a general understanding of metric spaces and their usual topology. As a reminder, a metric space is a set X with some notion of distance between any two elements of the set, given by a distance function d : X × X ! [0; 1) that is symmetric, satisfies the triangle inequality, and has the property that d(x; y) = 0 () x = y. Definition 2.1. For an element p in a metric space X, and a real number r > 0, we define the open ball B(p; r) as B(p; r) = fq 2 X j d(p; q) < rg: Similarly, we define the closed ball B(p; r) as B(p; r) = fq 2 X j d(p; q) ≤ rg: Definition 2.2. A set D is dense in a metric space X if and only if for every element p 2 X and > 0, D \ B(p; ) 6= ?. Alternatively, D is dense in X iff its closure D = X. Example 2.3. The rational numbers Q are dense in the real numbers R. Proof. Between any two real numbers a and b with a < b, there exists a rational number q such that a < q < b. Thus the intersection of the rationals with any open ball in the reals will always be nonempty. ◦ Definition 2.4. A set E is nowhere dense in a metric space X if E = ?. Example 2.5. The integers Z are nowhere dense in the real numbers R. Proof. Z has no limit points, so we know that Z = Z. Then assume for contradiction that the interior of Z is not empty; in other words, assume there exists some nonempty open ball U ⊂ R such that U ⊂ Z. Since Z is countable, U must also be countable. However, this is a contradiction because all open subsets of R are uncountable. Therefore U does not exist, so the interior of Z is empty. Remark 2.6. A set can be not dense, but also not nowhere dense. For example, consider the interval (0; 1) ⊂ R. (0; 1) = [0; 1] 6= R, so (0; 1) is not dense in R. However, [0; 1]◦ is also nonempty in R since it contains (0; 1), so the interval is also not nowhere dense in R. We can see that nowhere dense is a much stronger property than simply being not dense; in a sense, it means a set is not dense everywhere. Lemma 2.7. The complement of a closed nowhere dense set is an open dense set. Proof. Let A be a closed nowhere dense set. X n (A◦) = X n A, but A has empty interior, so X = X n A. Therefore X n A is open and dense. We are ready to state the major result of this section. 1 Theorem 2.8 (Baire Category Theorem). If X is a nonempty complete metric space, and (Gn)n=1 is a T1 countable collection of dense open subsets of X, then n=1 Gn is dense in X. Proof. Let U be an arbitrary open ball in X. Since G1 is dense in X, we know it has nonempty intersection with U. Pick x1 2 G1 \ U and 0 < r1 < 1 such that B(x1; r1) ⊆ G1 \ U. This is possible since both 2 G1 and U are open. Since G2 is dense in X, we know that G2 \ B(x1; r1) is nonempty. We repeat the 1 same process, picking x2 2 G2 \ B(x1; r1) and 0 < r2 < 2 such that B(x2; r2) ⊆ G2 \ B(x1; r1). Since every Gn is dense, we continue repeating the process to get a nested sequence of closed balls, such that 1 B(xn; rn) ⊆ Gn \ B(xn−1; rn−1), and 0 < rn < n . Note that xn 2 B(xm; rn) for all n > m, or in other 1 1 words, jxm − xnj < rn < n . Thus, for any > 0, there exists N 2 N such that N < , and for n; m ≥ N, jxm − xnj < rn < . Since X is a complete metric space, xn ! x 2 X as n ! 1, for some x. By definition of closure, we know that x 2 B(xn; rn). Since our sequence of balls is nested, and B(xn; rn) ⊆ Gn, we know that x 2 Gn for all n. Additionally, since B(x1; r1) ⊆ U, we know that x 2 U. Therefore, we have shown T T that ( Gn) \ U contains x and is thus nonempty. Since U is arbitrary, we have shown that Gn is dense in X. To understand why this is called the Baire Category Theorem, we introduce the following definition. Definition 2.9. In a metric space X, a set E ⊆ X is called meager if E can be written as a countable union of nowhere dense sets: 1 [ E = En; n=1 where each En is nowhere dense. Meager sets are also known as first category. E is called non-meager, or second category, if it is not first category. Two important properties of meager sets are that a subset of a meager set is also meager, and the union of meager sets is also meager. Remark 2.10. The usage of the word \category" in this context is unrelated to category theory. Corollary 2.11. If X is a nonempty complete metric space, then X is non-meager in itself. Proof. Let fEig be an arbitrary countable collection of closed nowhere dense subsets of X. Then by Lemma T 2.7, Vi = X nEi is an open dense subset of X for every i. By Theorem 2.8, i Vi is dense in X and nonempty. T S S T Since i Vi is disjoint with i Ei, we know that X 6= i Ei since there are elements in i Vi ⊂ X that are S not in i Ei. If fEig is instead an arbitrary countable collection of open nowhere dense subsets of X, then S each Ei must be the empty set, and thus i Ei = ? 6= X. Therefore, X cannot be written as a countable union of nowhere dense subsets. Example 2.12. Z and Q are meager in R, but the set of irrationals R n Q is non-meager in R. Proof. Z and Q are both countable, so they can be expressed as the countable union of singleton sets. Singleton sets are nowhere dense in R, so Z and Q are meager in R. Assume for contradiction R n Q is also meager in R. Then Q [ (R n Q) = R is meager in R, but R is non-meager by Corollary 2.11, so our assumption is false and R n Q is non-meager. Remark 2.13. This example helps show why we can think of meager sets as being small relative to non- meager sets in the same topological space. However, it is important to note that not all non-meager sets are of the same \size", and that meagerness and non-meagerness have nothing to do with the \absolute" or \objective" size of a set.