Empirical Processes: Lecture 06 Spring, 2010 Introduction to Empirical Processes and Semiparametric Inference Lecture 06: Metric Spaces Michael R. Kosorok, Ph.D. Professor and Chair of Biostatistics Professor of Statistics and Operations Research University of North Carolina-Chapel Hill 1 Empirical Processes: Lecture 06 Spring, 2010 §Introduction to Part II ¤ ¦ ¥ The goal of Part II is to provide an in depth coverage of the basics of empirical process techniques which are useful in statistics: Chapter 6: mathematical background, metric spaces, outer • expectation, linear operators and functional differentiation. Chapter 7: stochastic convergence, weak convergence, other modes • of convergence. Chapter 8: empirical process techniques, maximal inequalities, • symmetrization, Glivenk-Canteli results, Donsker results. Chapter 9: entropy calculations, VC classes, Glivenk-Canteli and • Donsker preservation. Chapter 10: empirical process bootstrap. • 2 Empirical Processes: Lecture 06 Spring, 2010 Chapter 11: additional empirical process results. • Chapter 12: the functional delta method. • Chapter 13: Z-estimators. • Chapter 14: M-estimators. • Chapter 15: Case-studies II. • 3 Empirical Processes: Lecture 06 Spring, 2010 §Topological Spaces ¤ ¦ ¥ A collection of subsets of a set X is a topology in X if: O (i) and X , where is the empty set; ; 2 O 2 O ; (ii) If U for j = 1; : : : ; m, then U ; j 2 O j=1;:::;m j 2 O (iii) If U is an arbitrary collection of Tmembers of (finite, countable or f αg O uncountable), then U . α α 2 O S When is a topology in X, then X (or the pair (X; )) is a topological O O space, and the members of are called the open sets in X. O 4 Empirical Processes: Lecture 06 Spring, 2010 For a subset A X, the relative topology on A consists of the sets ⊂ A B : B : check that this is a topology. f \ 2 Og A map f : X Y between topological spaces is continuous if f 1(U) 7! − is open in X whenever U is open in Y . A set B in X is closed if and only if its complement in X, denoted X B, is open. − 5 Empirical Processes: Lecture 06 Spring, 2010 The closure of an arbitrary set E X, denoted E, is the smallest closed 2 set containing E. The interior of an arbitrary set E X, denoted E , is the largest open 2 ◦ set contained in E. A subset A of a topological space X is dense if A = X. A topological space X is separable if it has a countable dense subset. 6 Empirical Processes: Lecture 06 Spring, 2010 A neighborhood of a point x X is any open set that contains x. 2 A topological space is Hausdorff if distinct points in X have disjoint neighborhoods. A sequence of points x in a topological space X converges to a point f ng x X, denoted x x, if every neighborhood of x contains all but 2 n ! finitely many of the xn. 7 Empirical Processes: Lecture 06 Spring, 2010 Suppose x x and x y. n ! n ! Then x and y share all neighborhoods, and x = y when X is Hausdorff. If a map f : X Y between topological spaces is continuous, then 7! f(x ) f(x) whenever x x in X; to see this, n ! n ! Let x X be a sequence with x x X. • f ng ⊂ n ! 2 Then for any open U Y containing f(x), all but finitely many x • ⊂ f ng are in f 1(U), and thus all but finitely many f(x ) are in U . − f n g Since U was arbitrary, we have f(x ) f(x). • n ! 8 Empirical Processes: Lecture 06 Spring, 2010 We now review the important concept of compactness: A subset K of a topological space is compact if for every set A K, • ⊃ where A is the union of a collection of open sets , K is also S contained in some finite union of sets in . S When the topological space involved is also Hausdorff, then • compactness of K is equivalent to the assertion that every sequence in K has a convergent subsequence (converging to a point in K). This result implies that compact subsets of Hausdorff topological • spaces are necessarily closed. Note that a compact set is sometimes called a compact for short. • A σ-compact set is a countable union of compacts. • 9 Empirical Processes: Lecture 06 Spring, 2010 A collection of subsets of a set X is a σ-field in X (sometimes called a A σ-algebra) if: (i) X ; 2 A (ii) If U , then X U ; 2 A − 2 A (iii) The countable union 1 U whenever U for all j 1. j=1 j 2 A j 2 A ≥ S Note that (iii) clearly includes finite unions. When (iii) is only required to hold for finite unions, then is called a field. A 10 Empirical Processes: Lecture 06 Spring, 2010 When is a σ-field in X, then X (or the pair (X; )) is a measurable A A space, and the members of are called the measurable sets in X. A If X is a measurable space and Y is a topological space, then a map f : X Y is measurable if f 1(U) is measurable in X whenever U is 7! − open in Y . If is a collection of subsets of X (not necessary open), then there exists O a smallest σ-field in X so that . A∗ O ⊂ A∗ This is called the σ-field generated by . A∗ O 11 Empirical Processes: Lecture 06 Spring, 2010 To see that such an exists: A∗ Let be the collection of all σ-fields in X which contain . • S O Since the collection of all subsets of X is one such σ-field, is not • S empty. Define to be the intersection of all . • A∗ A 2 S Clearly, and is in every σ-field containing . • O 2 A∗ A∗ O All that remains is to show that is itself a σ-field. • A∗ 12 Empirical Processes: Lecture 06 Spring, 2010 To show this, Assume that A for all integers j 1. • j 2 A∗ ≥ If , then j 1 Aj . • A 2 S ≥ 2 A Since j 1 AjS for every , we have j 1 Aj ∗. • ≥ 2 A A 2 S ≥ 2 A Also XS since X for all ; and forS any A , both • 2 A∗ 2 A A 2 S 2 A∗ A and X A are in every . − A 2 S Thus is indeed a σ-field. A∗ 13 Empirical Processes: Lecture 06 Spring, 2010 A σ-field is separable if it is generated by a countable collection of subsets. Note: we have already defined “separable” as a characteristic of certain topological spaces. There is a connection between the two definitions which we will point out shortly in our discussion on metric spaces. 14 Empirical Processes: Lecture 06 Spring, 2010 When X is a topological space, the smallest σ-field generated by the B open sets is called the Borel σ-field of X. Elements of are called Borel sets. B A function f : X Y between topological spaces is Borel-measurable if 7! it is measurable with respect to the Borel σ-field of X. Clearly, a continuous function between topological spaces is also Borel-measurable. 15 Empirical Processes: Lecture 06 Spring, 2010 For a σ-field in a set X, a map µ : R¯ is a measure if: A A 7! (i) µ(A) [0; ] for all A ; 2 1 2 A (ii) µ( ) = 0; ; (iii) For any disjoint sequence A , f jg 2 A µ j1=1 Aj = j1=1 µ(Aj) (countable additivity). S P The triple (X; ; µ) is called a measure space. A 16 Empirical Processes: Lecture 06 Spring, 2010 If X = A A for some finite or countable sequence of sets in 1 [ 2 [ · · · A with µ(A ) < for all indices j, then µ is σ-finite. j 1 If µ(X) = 1, then µ is a probability measure. For a probability measure P on a set Ω with σ-field , the triple A (Ω; ; P ) is called a probability space. A 17 Empirical Processes: Lecture 06 Spring, 2010 If the set [0; ] in Part (i) is extended to ( ; ] or replaced by 1 −∞ 1 [ ; ) (but not both), then µ is a signed measure. −∞ 1 For a measure space (X; ; µ), let be the collection of all E X A A∗ ⊂ for which there exists A; B with A E B and µ(B A) = 0, 2 A ⊂ ⊂ − and define µ(E) = µ(A) in this setting. Then is a σ-field, µ is still a measure, and is called the A∗ A∗ µ-completion of . A 18 Empirical Processes: Lecture 06 Spring, 2010 §Metric Spaces ¤ ¦ ¥ A metric space is a set D together with a metric. A metric or distance function is a map d : D D [0; ) where: × 7! 1 (i) d(x; y) = d(y; x); (ii) d(x; z) d(x; y) + d(y; z) (the triangle inequality); ≤ (iii) d(x; y) = 0 if and only if x = y. A semimetric or pseudometric satisfies (i) and (ii) but not necessarily (iii). 19 Empirical Processes: Lecture 06 Spring, 2010 Technically, a metric space consists of the pair (D; d), but usually only D is given and the underlying metric d is implied by the context. A semimetric space is also a topological space with the open sets generated by applying arbitrary unions to the open r-balls Br(x) y : d(x; y) < r for r 0 and x D (where B0(x) ). ≡ f g ≥ 2 ≡ ; A metric space is also Hausdorff, and, in this case, a sequence xn D f g 2 converges to x D if d(xn; x) 0.