AppendixA Auxiliary Results
A.1 Equivalence Relations; Groups
A relation: x ∼ y among the points of a space X is an equivalence relation if it is reflexive, symmetric, and transitive, that is, if (i) x ∼ x for all x ∈X; (ii) x ∼ y implies y ∼ x; (iii) x ∼ y, y ∼ z implies x ∼ z.
Example A.1.1 Consider a class of statistical decision procedures as a space, of which the individual procedures are the points. Then the relation defined by δ ∼ δ if the procedures δ and δ have the same risk function is an equivalence relation. As another example consider all real-valued functions defined over the real line as points of a space. Then f ∼ g if f(x)=g(x) a.e. is an equivalence relation.
Given an equivalence relation, let Dx denote the set of points of the space that are equivalent to x. Then Dx = Dy if x ∼ y, and Dx ∩Dy = 0 otherwise. Since by (i) each point of the space lies in at least one of the sets Dx, it follows that these sets, the equivalence classes defined by the relation ∼, constitute a partition of the space. AsetG of elements is called a group if it satisfies the following conditions. (i) There is defined an operation, group multiplication, which with any two elements a, b ∈ G associates an element c of G. The element c is called the product of a and b and is denoted by ab. A.2. Convergence of Functions; Metric Spaces 693
(ii) Group multiplication obeys the associative law
(ab)c = a(bc).
(iii) There exists an element e ∈ G, called the identity, such that
ae = ea = a for all a ∈ G.
(iv) For each element a ∈ G, there exists an element a−1 ∈ G,itsinverse, such that
aa−1 = a−1a = e.
Both the identity element and the inverse a−1 of any element a can be shown to be unique.
Example A.1.2 The set of all n × n orthogonal matrices constitutes a group if matrix multiplication and inverse are taken as group multiplication and inverse respectively, and if the identity matrix is taken as the identity element of the group. With the same specification of the group operations, the class of all non- singular n × n matrices also forms a group. On the other hand, the class of all n × n matrices fails to satisfy condition (iv).
If the elements of G are transformations of some space onto itself, with the group product ba defined as the result of applying first transformation a and following it by b, then G is called a transformation group. Assumption (ii) is then satisfied automatically. For any transformation group defined over a space X the relation between points of X given by
x ∼ y if there exists a ∈ G such that y = ax is an equivalence relation. That it satisfies conditions (i), (ii), and (iii) required of an equivalence follows respectively from the defining properties (iii), (iv), and (i) of a group. Let C be any class of 1 : 1 transformations of a space, and let G be the class ±1 ±1 ±1 of all finite products a1 a2 ...am ,witha1,...,am ∈ C, m = 1, 2, . . . , where each of the exponents can be +1 or −1 and where the elements a1, a2, . . . need not be distinct. Then it is easily checked that G is a group, and is in fact the smallest group containing C.
A.2 Convergence of Functions; Metric Spaces
When studying convergence properties of functions it is frequently convenient to consider a class of functions as a realization of an abstract space F of points f in which convergence of a sequence fn to a limit f, denoted by fn → f, has been defined.
Example A.2.1 Let µ be a measure over a measurable space (X , A). 694 AppendixA. Auxiliary Results
(i) Let F be the class of integrable functions. Then fn converges to f in the 1 mean if
|fn − f| dµ → 0. (A.1)
(ii) Let F be a uniformly bounded class of measurable functions. The sequence is said to converge to f weakly if
fnpdµ→ fpdµ (A.2)
for all functions p that are integrable µ.
(iii) Let F be the class of measurable functions. Then fn converges to f pointwise if
fn(x) → f(x) a.e. µ. (A.3)
A subset of F0 is dense in F if, given any f ∈F, there exists a sequence in F0 having f as its limit point. A space F is separable if there exists a countable dense subset of F. A space F such that every sequence has a convergent subsequence whose limit point is in F is compact.2 AspaceF is a metric space if for every pair of points f, g in F there is defined a metric (or distance) d(f,g) ≥ 0 such that (i) d(f,g) = 0 if and only if f = g; (ii) d(f,g)=d(g, f); (iii) d(f,g)+d(g, h) ≥ d(f,h) for all f, g, h. The space is a pseudometric space if (i) is replaced by (i ) d(f,f) = 0 for all f ∈F. A pseudometric space can be converted into a metric space by introducing the equivalence relation f ∼ g if d(f,g) = 0. The equivalence classes F , G, . . . then constitute a metric space with respect to the metric D(F, G)=d(f,g) where f ∈ F , g ∈ G. In any pseudometric space a natural convergence definition is obtained by putting fn → f if d(fn,f) → 0.
Example A.2.2 The space of integrable functions of Example A.2.1(i) becomes a pseudometric space if we put d(f,g)= |f − g| dµ and the induced convergence definition is that given by (1).
1Here and in the examples that follow, the limit f is not unique. More specifically, if fn → f,thenfn → g if and only if f = g (a.e. µ). Putting f ∼ g when f = g (a.e. µ), uniqueness can be obtained by working with the resulting equivalence classes of functions rather than with the functions themselves. 2The term compactness is more commonly used for an alternative concept. which coincides with the one given here in metric spares. The distinguishing term sequential compactness is then sometimes given to the notion defined here. A.2. Convergence of Functions; Metric Spaces 695
Example A.2.3 Let P be a family of probability distributions over (X , A). Then P is a metric space with respect to the metric d(P, Q)= sup|P (A) − Q(A)|. (A.4) A∈A
Lemma A.2.1 If F is a separable pseudometric space, then every subset of F is also separable.
Proof. By assumption there exists a dense countable subset {fn} of F.Let 1 S = f : d(f,f ) < , m,n n m and let A be any subset of F. Select one element from each of the intersections A ∩ Sm,n that is nonempty, and denote this countable collection of elements by A0.Ifa is any element of A and m any positive integer, there exists an element fnm such that d(a, fnm ) < 1/m. Therefore a belongs to Sm,nm , the intersection
A∩Sm,nm is nonempty, and there exists therefore an element of A0 whose distance to a is < 2/m. This shows that A0 is dense in A, and hence that A is separable.
Lemma A.2.2 A sequence fn of integrable functions converges to f in the mean if and only if
fn dµ → fdµ uniformly for A ∈A. (A.5) A A Proof. That (1) implies (5) is obvious, since for all A ∈A % % % % % % % fn dµ − fdµ% ≤ |fn − f| dµ. A A Conversely, suppose that (5) holds, and denote by An and An the set of points x for which fn(x) >f(x) and fn(x)