John Nachbar Washington University March 12, 2016 Separation in General Normed Vector Spaces1 1 Introduction N Recall the Basic Separation Theorem for convex sets in R . N Theorem 1. Let A ⊆ R be non-empty, closed, and convex. If 0 2= A then there is N a v 2 R , v 6= 0, such that v · a ≥ v · v for all a 2 A. The goal here is to extend this result to general normed vector spaces over the reals. (See the notes on Vector Spaces and Norms.) N I have stated the Basic Separation Theorem for R using inner products. The next section discusses the fact that this approach does not generalize. In general vector spaces, it is necessary instead to work with real-valued linear functions. In N R , such functions can always be represented as inner products. In general vector spaces, this is no longer true. 2 Linear Functions Given a vector space X, a function F : X ! R is linear iff for any x; x^ 2 X and any θ 2 R, 1. F (θx) = θF (x), 2. F (x +x ^) = F (x) + F (^x). Taking θ = 0, the first condition implies that the graph of a linear function always goes through the origin: F (0) = 0. It is common to use the word \functional" to describe a real-valued function defined on a general vector space. To avoid excess jargon, I call a functional simply a \real-valued function." N In R , but not more generally, a real-valued function is linear iff it can be written N as an inner product. Thus, for R , writing the Basic Separation Theorem in terms of the inner product v · x is equivalent to writing the theorem in terms of the linear function F (x) = v · x. N N N Theorem 2. F : R ! R is linear iff there is a v 2 R such that for all x 2 R , F (x) = x · v. 1cbna. This work is licensed under the Creative Commons Attribution-NonCommercial- ShareAlike 4.0 License. 1 Proof 1. ). Let en be the n coordinate vector: en = (0;:::; 0; 1; 0;:::; 0), with a 1 in n P n the n place. Define vn = F (e ). Then, for any x = (x1; : : : ; xn) = n xne , P n P n P the linearity of F impies that F (x) = F ( xne ) = n xnF (e ) = n xnvn = x · v. 2. (. Almost immediate. For a general vector space X, let X∗ denote the set of real-valued linear functions ∗ N defined on X. X is called the dual space of X. The above theorem says that R is N \self-dual:" every real-valued linear function F on R is identified with an element N N ∗ N v 2 R , and conversely. Abusing notation, (R ) = R . One can also show that, ∗ similarly, (L2) = L2. But not all vector spaces are self-dual. Consider `1. For many v; x 2 `1, v · x is P 2 not even well defined. For example, if v = (1; 1;::: ) then v · v should be n(vn) , but the latter is infinite. One can partially fix this problem by restricting v to `1, the normed vector space 1 consisting of the points in R that are absolutely summable, X kxk1 = jxnj < 1: n 1 1 P For any v 2 ` , the function F : ` ! R given by F (x) = v · x = n vnxn is well defined. Therefore `1 ⊆ (`1)∗. Unfortunately, `1 is a proper subset of (`1)∗: there exist elements of (`1)∗ that cannot be represented as inner products with elements of `1, and in fact cannot be represented as inner products in any standard sense. The canonical examples are the Banach Limits, real-valued linear functions on `1 whose existence is implied by the Hahn-Banach Extension Theorem, given below. Because they have no inner product representation, Banach Limits are hard to visualize, but they operate somewhat like limits of averages, N 1 X lim xn: n!1 N n=1 Unlike limits of averages, which are not defined if the limit does not exist, Banach Limits are defined for all x 2 `1. Banach Limits have occasionally been used in economic theory to formulate the utility functions of decision makers who are, loosely speaking, infinitely patient. In any vector space, a plane is, by definition, the level set of a real-valued linear function. The fact that there are elements of (`1)∗ that have no inner-product interpretation means that there are planes in `1 that cannot be represented via 2 inner products. Since such planes may be needed for separation, the implication is that the Basic Separation Theorem for general normed vector spaces cannot rely on inner products. 3 The Basic Separation Theorem. Theorem 3. Let X be a normed vector space and let A be a non-empty, closed, convex subset of X. If 0 2= A then there is a linear function F : X ! R and an r > 0 such that for all a 2 A, F (a) > r. The Basic Separation Theorem is a consequence of the Hahn-Banach Extension Theorem, which I state and prove in the next section. The Basic Separation Theorem can be used in turn to establish the Separating Hyperplane Theorem for closed convex sets, at least one of which is compact, exactly N as in R . Generalizing the Supporting Hyperplane Theorem is less straightforward. N In R , the Supporting Hyperplane Theorem says that if A is non-empty, closed, and ∗ convex and x is not an interior point of A then there is a linear function F : X ! R such that for all a 2 A, F (a) ≥ F (x∗). For general normed vector spaces, a condition stronger than \x∗ is not be an interior point of A" is needed. 4 The Hahn-Banach Extension Theorem The Hahn-Banach Theorem is a relative of the Supporting Hyperplane Theorem. It is stated for special types of convex sets, namely convex sets generated by sublinear functions. A function p : X ! R is sublinear iff for any x; x^ 2 X and any θ 2 R, θ ≥ 0, 1. p(θx) = θp(x), 2. p(x +x ^) ≤ p(x) + p(^x). By the first property, if p is sublinear then p(0) = 0. Any linear function is sublinear. An example of a sublinear function that is not linear is p : R ! R, p(x) = jxj. More generally, in any normed vector space, the norm is sublinear. Any sublinear function is convex: p(θ(x) + (1 − θ)^x) ≤ p(θx) + p((1 − θ)^x) = θp(x) + (1 − θ)p(^x). Recall that a function is convex iff its epigraph, which is the set of points lying on or above the graph, is convex. Thus, the epigraph of a sublinear function is convex. In fact, one can readily verify that the epigraph of a sublinear function is a closed, convex, cone. The epigraph of p : R ! R, p(x) = jxj provides illustration. The Hahn-Banach theorem is a statement about supporting this cone at the origin, which is the cone's vertex. Theorem 4 (Hahn-Banach Extension Theorem). Let W be a vector subspace of a vector space X. Let f : W ! R be linear and p : X ! R be sublinear. If for all x 2 W , f(x) ≤ p(x), then there is a linear function F : X ! R such that, 3 1. For all x 2 W , f(x) = F (x), 2. For all x 2 X, F (x) ≤ p(x). To interpret the statement of the Hahn-Banach Theorem, focus on the finite N dimensional case: X = R . The epigraph of p is then a closed, convex cone in N+1 N N+1 R . If F (x) ≤ p(x) for all x 2 R then the graph of F , which is a plane in R , lies on or below the cone determined by the epigraph of p: the graph of F supports this cone at the origin. Thus if W = f0g, in which case f is trivially defined by f(0) = 0, then Hahn-Banach says that the closed convex cone generated by the epigraph of p can be supported at the origin, something we already knew from the Supporting Hyperplane Theorem. Hahn-Banach generalizes this in two ways. First, and most importantly, it allows N X to be a general vector space rather than just X = R . Second, it accommodates some restrictions on the supporting plane. If one already has a real-valued linear function f that supports the epigraph of p on a subspace W , then the Hahn-Banach Theorem says that one can find a real-valued linear function F , agreeing with f on W , that supports the epigraph of p on all of X. That is, the supporting plane through 0 defined by F must contain the lower dimensional plane through 0 defined by f. 5 Proof of the Hahn Banach Extension Theorem Proof of Theorem4 . The proof below is complete with one exception, which I flag. Let Wg;Wh be vector subspaces of X and let g : Wg ! R, h : Wh ! R be real-valued continuous functions such that for all x 2 Wg, g(x) ≤ p(x) and for all x 2 Wh, h(x) ≤ p(x). Say that h extends g iff 1. Wg ⊆ Wh, 2. For all x 2 Wg, h(x) = g(x). Say that h is a proper extension of g if h is an extension of g and Wg 6= Wh. Consider the set of all extensions of f. This set is not empty (it contains f).