Proximal Analysis and Boundaries of Closed Sets in Banach Space, Part I: Theory
Total Page:16
File Type:pdf, Size:1020Kb
Can. J. Math., Vol. XXXVIII, No. 2, 1986, pp. 431-452 PROXIMAL ANALYSIS AND BOUNDARIES OF CLOSED SETS IN BANACH SPACE, PART I: THEORY J. M. BORWEIN AND H. M. STROJWAS 0. Introduction. As various types of tangent cones, generalized deriva tives and subgradients prove to be a useful tool in nonsmooth optimization and nonsmooth analysis, we witness a considerable interest in analysis of their properties, relations and applications. Recently, Treiman [18] proved that the Clarke tangent cone at a point to a closed subset of a Banach space contains the limit inferior of the contingent cones to the set at neighbouring points. We provide a considerable strengthening of this result for reflexive spaces. Exploring the analogous inclusion in which the contingent cones are replaced by pseudocontingent cones we have observed that it does not hold any longer in a general Banach space, however it does in reflexive spaces. Among the several basic relations we have discovered is the following one: the Clarke tangent cone at a point to a closed subset of a reflexive Banach space is equal to the limit inferior of the weak (pseudo) contingent cones to the set at neighbouring points. This generalizes the results of Penot [14] and Cornet [7] for finite dimensional spaces and of Penot [14] for reflexive spaces. What is more we show that this equality characterizes reflex ive spaces among Banach spaces, similarly as the analogous equality with the contingent cones instead of the weak contingent cones, characterizes finite dimensional spaces among Banach spaces. We also present several other related characterizations of reflexive spaces and variants of the considered inclusions for boundedly relatively weakly compact sets. For several years it has been an open question as to how to extend Clarke's proximal normal formula [6] from finite dimensional spaces to infinite dimension. We give the answer to this question for reflexive Banach spaces. We present a characterization of the Clarke normal cone at a point to a closed subset of a reflexive Banach space by means of the proximal normal functionals and the other one, by means of the Frechet normals. The importance of our result is underlined by the fact that we use exact normals in contrast to the c-normals used for example in [18]. Our main results are presented in the first part of the paper, the second part is devoted to their applications. For example, we obtain generaliza- Received November 8, 1984. The research of the first author was partially supported on NSERC Grant A5116 while that of the second was partially supported on National Science Foundation Grant ECS 8300214. 431 Downloaded from https://www.cambridge.org/core. 29 Sep 2021 at 23:55:26, subject to the Cambridge Core terms of use. 432 J. M. BORWEIN AND H. M. STROJWAS tions of results given in [12] and of the primal Bishop-Phelps theorem proven in [1]. We complete the results of Borwein and O'Brien [4] and of Borwein [2] on pseudoconvexity. We mention implications of our main results for vector fields and give detailed analysis of consequences in the theory of differentiability and subdifferentiability. We show how the "lim inf" inclusions under consideration relate to the subdifferential properties of spaces and functions. In particular we show that every lower semicontinuous function in a reflexive Banach space is densely sub- Frechet and that every locally Lipschitian function in a weakly compactly generated Banach space is densely sub-Hadamard. We prove the proximal subgradient formula in a reflexive Banach space which extends one for a finite dimensional space given in [17]. We also provide many examples which are carefully analyzed. 1. Preliminaries. Let E be a normed space with continuous linear dual E*. For the subset C c E and a point x G C we will consider the following tangent cones to C at x: 1) the contingent cone Kc(x) and 2) the Clarke tangent cone Tc(x), as defined for example in [6], where also the corresponding references may be found; 3) the weak (sequential) contingent cone WKc(x), which is the set of weak limits of sequences t~\cn — x), with tn > 0, tn —> 0, and cn G C; 4) the weak (sequential) tangent cone WTc(x\ which we define to be the set of those y G E, that for any sequences xn and tn such that xn G C, tn > 0, xn —> x, tn —> 0 there exists a sequence cn9 cn G C for which the sequence t~ (cn — xn) tends weakly to 7; 5) the pseudocontingent cone Pc(x) := ~cdKc(x)\ 6) the weak pseudocontingent cone WPc(x) := ~côWKc(x)\ 1) the Bony tangent cone, defined as the set of vectors y G E such that whenever n is a proximal normal vector to C at x, then the inequality ll« - ty\\ iï NI holds for every / > 0. We recall the definition of a proximal normal vector. Definition 1.1. Vector n G E is said to be a proximal normal vector to C at x G C if there are u <£ C and X > 0 such that n = X(u — x) and \\u — x\\ = dc(u), where dc(u) : = inf \\u — c||. Downloaded from https://www.cambridge.org/core. 29 Sep 2021 at 23:55:26, subject to the Cambridge Core terms of use. PROXIMAL ANALYSIS 433 As a straightforward consequence of the introduced definitions we obtain the following inclusions: WTc(x) c WKc{x) c WPc(x) (1.1) U U U Tc(x) c Kc{x) c pc(x). PROPOSITION 1.1. Bc(x) is a closed cone which satisfies the inclusion (1.2) Kc{x) c Bc(x). Furthermore it is convex, whenever E is a smooth Banach space in which case also (1.3) Pc(x) c Bc(x). If the norm of E is Frechet differentiable (that is Frechet differentiable away from zero), then (1.4) WPc(x) c Bc(x). Proof of the above proposition follows from [14]. Inclusion (1.4) is implied by the one obtained in [14] for another type of the weak contingent cone: its definition uses bounded nets instead of sequences. It may be observed that in a reflexive space these two definitions are equivalent. In all the sequel, whenever we refer to a general approximating cone which is among the defined above, we will denote it Rc(x). Definition 1.2. We say that C is R-pseudoconvex at x e C if (1.5) C - x c Rc(x). If (1.5) holds for all x G C, C is R-pseudoconvex. Definition 1.3. We say x e C is an R-proper point of C if Rc(x) ¥= E. Inclusions (1.1)-(1.4) imply obvious relations for .R-pseudoconvexity and .R-properness. We will consider also the polars of the approximating cones: R°(x) : = {y* G E*\ (y*9 y) ^ 0 for all y G RC(X) }. Nc(x) := Tç(x) is called the Clarke normal cone. We point out that the pseudocontingent cone and the weak pseudocon tingent cone are essential for dual and differentiability results. The following theorem was first proven in [18]. Alternative proofs may be also found in [15], [16] and in [5]. Downloaded from https://www.cambridge.org/core. 29 Sep 2021 at 23:55:26, subject to the Cambridge Core terms of use. 434 J. M. BORWEIN AND H. M. STROJWAS THEOREM 1.1 (Treiman): If C is a closed subset of a Banach space, x G C, then lim inf Kc(x') c Tc(x). x'—*x C THEOREM 1.2 (Penot). 7/*2s is reflexive C ci E, x Œ C then Tc(x) c lim inf WKc(x'). x'^x C The proof of Theorem 1.2 can be found in [14]. 2. Tangent cone separation principle and stars of closed sets in Banach spaces. Definition 2.1. star C := {x e C| [je, j] c C for all 7 G C}, where [x, y] := co{x, j}. We say C is starshaped when star C is non-empty. PROPOSITION 2.1. For any subset C of a {locally convex vector) space E the following inclusion holds (2.1) star C c n Tc(x) + x. /Vex?/. Let x e star C and JC G C. Let V be any neighbourhood of 0. Then for any x' G (JC + V) Pi C and any /, 0 < / ^ 1, we have t_1[(f(jt - jcr) + jcr) - xr] H- F c r\c - y) + v, which shows that x — x e TC(JC), hence (2.1) follows. In particular Proposition 2.1 implies that convex sets are 7"-pseudocon- vex in any locally convex vector space. For closed sets in Banach spaces the inclusion in (2.1) may be replaced with equality. This is a consequence of the following theorem. THEOREM 2.1. Let C be a closed subset of a Banach space E. Suppose that x in C and y in E are such that (2.2) [JC, y] t C. Then for any r > 0 one can find xr in C with (i) y £ Kc(xr) + xr and (ii) \\xr - x\\ ^ 117 - x\\ + r. Downloaded from https://www.cambridge.org/core. 29 Sep 2021 at 23:55:26, subject to the Cambridge Core terms of use. PROXIMAL ANALYSIS 435 Proof. (2.2) implies that there is some z in [x, y] and some s > 0 such that (z + sB) n C = 0, where 5 is a closed unit ball in E. Take r e (0, s). Applying Ekeland's variational principle as given in [9], to the function z •-> Ik - 711 on the set D := C n U _ (z 4 r£), ze[jc,z] yields an jcr in Z) such that for all x in D (2.3) ||x - y|| ^ IK - yll - q\\x - xr\\, for some q e (0, 1).