TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY Volume 350, Number 2, February 1998, Pages 539–558 S 0002-9947(98)01865-0
EXTREMAL VECTORS AND INVARIANT SUBSPACES
SHAMIM ANSARI AND PER ENFLO
Abstract. For a bounded linear operator on Hilbert space we define a se- quence of so-called minimal vectors in connection with invariant subspaces and show that this presents a new approach to invariant subspaces. In par- ticular, we show that for any compact operator K some weak limit of the sequence of minimal vectors is noncyclic for all operators commuting with K and that for any normal operator N, the norm limit of the sequence of minimal vectors is noncyclic for all operators commuting with N. Thus, we give a new and more constructive proof of existence of invariant subspaces. The sequence of minimal vectors does not seem to converge in norm for an arbitrary bounded linear operator. We will prove that if T belongs to a certain class of operators, then the sequence of such vectors converges in norm, and C that if T belongs to a subclass of , then the norm limit is cyclic. C
Introduction In this paper we will study different types of extremal vectors for operators on Hilbert space and their connection to invariant subspaces. We present a new method to find invariant subspaces, which we feel is more constructive then the known ones and which gives hyperinvariant subspaces for all compact and all normal operators in a unified way. We feel that the method may give invariant subspaces for several other classes of operators, but so far we have only succeeded in proving this under extra assumptions. The second author started this study by considering the best approximate so- n lutions of the equations T y = x0 for the case when T has dense range and x0 is not in the range of T . More precisely, he considered the yn of smallest norm such that T ny x . Using these backward minimal points, T ny ’s, he proved k n − 0k≤ n Theorem 4. The first author introduced forward minimal points vn’s such that n vn x0 and T vn is minimal. This gives even a simpler proof of Theorem k4, and− therek≤ is a dualityk betweenk the forward and the backward minimal points as given in Proposition 2. In this paper we also study geometric properties of vn and n T yn involving scalar products and limit points. These geometric properties have n a connection to invariant subspaces. We show that (T yn) converges in norm for normal operators. In relation to the norm convergence of the extremal vectors, we introduce a new class of operators, called operators of class . This class general- R izes C1-contractions, and we feel that it is of interest in its own right too. Sections 1-2 are due to the second author, 3-4 are due to the first and the appendix is due to both.
Received by the editors October 16, 1995. 1991 Mathematics Subject Classification. Primary 47A15. Partially supported by NSF grant number 441003.
c 1998 American Mathematical Society
539
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 540 SHAMIM ANSARI AND PER ENFLO
1. Extremal vectors and their properties In this paper H will denote a Hilbert space over the field of complex numbers, X a reflexive strictly convex Banach space over the field of complex numbers. For any operator T : X X, r = r(T ) will represent the spectral radius of T and (T ) the range of T .→ R Backward Minimal Vectors. Let T : H H be a bounded operator with dense range. Let x H and >0 with < x→. There is a unique vector y such 0 0 n,x0 n ∈ k k that T y x0 and k n,x0 − k≤ n y =inf y : T y x0 . k n,x0 k {k k k − k≤ } The points y are called backward minimal points. n,x0 Forward Minimal Vectors. Let T : H H be an injective bounded operator. Let x H and >0 with < x . There→ is a unique vector v such that 0 0 n,x0 ∈ k k v x0 and k n,x0 − k≤ n n T v =inf T v : v x0 . k n,x0 k {k k k − k≤ } The points T nv are called forward minimal points. n,x0 When there is no ambiguity, we will drop some of the superscripts and subscripts from y and v . Some times the vectors T ny and v will also be referred as n,x0 n,x0 n n backward, respectively, forward minimal vectors. It is clear that T ny x = n,x0 0 and v x = . k − k k n − 0k Theorem 1 (Orthogonality Equations). There exist constants δ < 0 and δ0 < 0 such that
y = δ T ∗(Ty x ), − 0 (v x )=δ0T∗Tv . − 0 Proof. We will prove only the first equation, since the proof of the second equation is similar. Note that for any vectors u, w,ifRe(u, w) < 0, then the function r(t)= u+tw is decreasing over [0,t0]forsomet0 >0, whereas if Re(u, w) 0, then thek functionk r(t)= u+tz is increasing over [0, ). Thus, if we have≥ k k ∞ Re(T ∗(Ty x ),z)<0, then r(t)= (Ty x )+tT z is decreasing over [0,t ]for − 0 k − 0 k 0 some t0.So, Ty x0 (Ty x0)+tT z for all t [0,t0]. By the definition of y it followsk that− y k≥k+ tz y− for t [0k,t ]. Hence,∈ Re(y ,z) 0. Since z is k k≥k k ∈ 0 ≥ arbitrary it follows that for some δ < 0wehavey =δ T∗(Ty x ). · − 0 Remark 1. From the orthogonality equations we obtain n n n 1 n n T y = δ (I δ T T ∗ )− T T ∗ x , n − n − n 0 n n 1 v =(I δ T∗ T )− x . n − n 0 n n 1 n n n n 1 Let An = δn(I δnT T ∗ )− T T ∗ and Bn =(I δnT∗ T )− .ThenAnand Bn − − − δn z are positive operators. An < 1and Bn =1foralln.Letfn(z)= − · .Then 1 δn z k k k kn n n− n· fn(z) is analytic in a neighborhood of σ(T T ∗ )andσ(An)=fn(σ(T T∗ )). The n 2 n 2 n n δn T δn T maximum point of f(σ(T T ∗ )) is − ·k nk 2 . It follows that An = − ·k nk 2 < 1 δn T 1 δn T 1 − ·k k k k −n ·kn k 1. Now let g (z)= . Then, g is analytic in a neighborhood of σ(T T ∗ ), and n 1 δn z n n −n · n n σ(Bn)=gn(σ(T T∗ ). Since 0 σ(T T ∗ ), it can be verified that the maximum n n ∈ point of g (σ(T T ∗ )) is 1. Hence, B =1. n k nk
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use EXTREMAL VECTORS AND INVARIANT SUBSPACES 541
Theorem 2. Let x0 H and x0 =0. The functions y and Tv are analytic on (0, x ). ∈ 6 7−→ 7−→ k 0k 1 Proof. From the orthogonality equation for y we have y = δ (I δ T ∗T )− T ∗x . − − 0 So, clearly the analyticity of the function δ implies the analyticity of y. To show that δ is analytic we will prove7−→ that its inverse function δ 7−→ is 7−→ 7−→ analytic. (Note that this inverse function is well defined by the uniqueness of y.) We have
1 = Ty x = δ T(I δ T∗T)− T∗x x , k − 0k k− − 0 − 0k
2 1 2 = δ T(I δ T∗T)− T∗x x . k− − 0 − 0k It follows that δ 2 is analytic and injective. Hence, δ is analytic. 7−→ 7−→ Proposition 1. If Ty x ,then y,y y 2. k − 0k≤ |h i| ≥ k k
Proof. Consider the plane spanned by x0 and Ty. Assume z = ay +r,wherea<1 and r y .WegetTz =aT y + Tr.PutTr = u+v,whereu span x ,Ty and ⊥ ∈ { 0 } v span x0,Ty . We know that u x0 Ty.Weget x0 Tz x0 aT y u . We⊥ have { x Ty} ,Ty > 0. Thus,⊥ −x Ty 2 =k x− k≥kTy ,x − Ty −
Theorem 3. A. If T has a dense range, then Ty x0,Ty =0. B. For any bounded operator T with dense range,h − if x i6(T ),then 0 6∈ R Ty x Ty n − 0 , n 0 Tyn x0 Tyn → Dk − k k kE for some sequence (n) converging to 0.
1 Proof. A. Ty x ,Ty = T∗(Ty x ),y = δ− y ,y =0. h − 0 i h − 0 i h i6 B. Let >0. Let θ be the angle between the vectors Ty x0 and Ty.Then there exists t>0 such that x (1 + t)Ty = sinθ . Let− us see that t0. We have (1 + t) Ty Ty +2.So,tTy 2and t 2 0. k k≤k k k k≤ ≤ x0 · Nowk k− let = sinθ . Then, y (1 + t) y < (1 + s) y .Forn 2 10 10 let = sinθ· . Applyingk thek≤ above argumentk k repeatedly,k wek get y ≥ n0 n0 1 n 1 n0 − · − π k k≤ (1+s0)(1+10 ) (1+3n0 ) y0 . We will prove that θnk 2 for some subsequence (n ) of natural··· numbers. Ifk not,k then θ > π + δ for some→δ>0andforalln n k n 2 ≥ 0 for some n0. So, there exists r such that 0 < sinθn
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 542 SHAMIM ANSARI AND PER ENFLO
2. Invariant subspaces
yn 1 k k− k Lemma 1. If T is quasinilpotent, then y 0 as k 0 for some subsequence k nk k → → (nk).
yn 1 Proof. Suppose not. Then there is a t>0 such that k − k >tfor all n>0. yn n 1 k k So, y1 ty2 ... t − yn for all n>0. By the minimality of y1 , nk1 k≥ k k≥ ≥ k kn1 n 1 n 1k k T − yn y1 .SowehaveT− yn T −yn y1 t− yn . k k≥kn 1k n 1 k kk k≥k k≥k k≥ k k Therefore, T − t − for all n>0. This contradicts that σ(T )= 0 . k k≥ { } Theorem 4. Every compact operator has a hyperinvariant subspace. Proof. Let K be a compact operator. If (K) = H,then (K) is a hyperinvariant subspace for K. So, we will assume thatR (K6 ) is dense.R Also if K has nonzero eigenvalues, then the eigenspace correspondingR to each eigenvalue is a hyperinvari- ant subspace for K. So, we can assume that K has no eigenvalues, i.e. σ(K)= 0 . { } Let x0 H with x0 =0and0<< x0 .Letynbe the backward min- ∈ k k6 n k k imal points with respect to (x0,,K ). Let T be any operator commuting with yn 1 k k− k K, with T =1.Let y 0. We may assume, by passing to a subse- k k nk → kn k n 1 quence if necessary, that K k yn y0 weakly and K k− yn 1 z weakly. Then k k− nk → → TK ynk 1 TKz in norm. Let αnk be scalars and γnk be vectors with γnk ynk such that− → ⊥
Tyn 1 = αn yn +γn , k− k k k nk nk nk K Tyn 1 = αn K yn +K γn , k− k k k nk nk nk nk TK yn 1,K yn x0 =αn K yn ,K yn x0 k− k − k k k − D E D E + Knkγ ,Knky x , nk nk − 0
nk nk D nk nk E TK yn 1,K yn x0 =αn K yn ,K yn x0 , k− k − k k k −
D nk nk E D E limk TK yn 1,K yn x0 =0. →∞ k− k − D E So,
TKz,y x =0 0 − 0 D E and
Tz,K∗(y x ) =0. 0 − 0 D E
n n π 2 Observe that the angle between K yn and K yn x0 is bigger than 2 .So, y0 n 2 2 2 − k k ≤ lim inf K k y x .Thatis,y x =0andK∗(y x)=0.Note k nk k ≤k 0k − 0− 06 0− 0 6 that Tz :TK = KT is a closed subspace invariant under any operator commuting with{K. This space is} not the whole space because it is orthogonal to the nonzero vector K∗(y x ). 0 − 0 Definition 1. A vector x is called hypernoncyclic for an operator T if the vector space Ax : AT = TA , which is invariant under all operators commuting with T , is not{ dense. }
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use EXTREMAL VECTORS AND INVARIANT SUBSPACES 543
Theorem 5. Let T be a quasinilpotent operator. Let x H. If for every δ > 0 0 ∈ 1 and each δ2 with 0 <δ2 <δ1 there exists a τ>0with T ny ,Tny x τ n n − 0 ≤− D E for all n 1 and all in [δ2,δ1],thenx0 is hypernoncyclic for T . In particular, T has hyperinvariant≥ subspaces.
Proof. We will first prove that for every δ1 and δ2 with 0 <δ2 <δ1 there is a
constant Kδ1,δ2 such that for all n yδ2 K yδ1 . k n k≤ δ1,δ2 ·k n k The hypothesis of the theorem implies that for every in [δ2,δ1] and every n,if n n π the angle between T yn and T yn x0 is θn,thenθn > 2 +θ0 for some θ0 > 0. − Consequently, sin θn δ2 δ1 Ayn 1 = αnk yn + γnk . k− · k Then 1 yδ2 yδ1 α = Ayδ2 ,yδ1 k nkk K k nk k 0. nk nk 1 nk δ δ δ1,δ2 δ | | | − |· y 1 2 ≤ y 1 ≤ · y 1 → nk nk 1 nk 1 k k k − k k − k It follows that nk δ2 nk δ1 AT yn 1,T yn x0 0. k− k − → D E nk δ2 Since, AT yn 1 = AT x0 + δ2 AT znk 1 for some znk 1 with znk 1 =1, k− · − − k − k nk δ1 AT x0 + δ2 AT znk 1,T yn x0 0. · − k − → D E License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 544 SHAMIM ANSARI AND PER ENFLO Consequently, nk δ1 limn 0 AT x0,T yn x0 δ2. → k − ≤ D E As the left-hand side of the above limit is independent of δ2, nk δ1 limn 0 AT x0,T yn x0 0 → k − → and D E AT x ,y x =0. 0 − 0 In other words, D E Ax ,T∗(y x ) =0. 0 − 0 D E Theorem 6. If σ(T )= 0 , then either “A” or “B” holds. { } A. T has hyperinvariant subspaces. B. For every x in H with x =1, for every >0, and for every γ>√2,there k k are x0 with x0 =1and 0 with 0 <0 <such that either 1 or 2 holds. k k 1. x x0 <. k − k 2. For some subsequence (n ) of integers we have T nk y0 = A x +S , k nk,x1 nk 1 nk where S x and each weak limit of S has norm less than γ 0. nk ⊥ 1 nk · Proof. Assume “B” does not hold for some unit vector x0,some>0andsome γ>√2.SinceTis quasinilpotent there is a subsequence (y )of(y )such nk,x0 n,x0 that y / y 0. Put T nk y = Ax + w + S ,whereS 0 nk,x0 nk,x0 nk,x0 0 0 nk nk k k k k→ γ2→ weakly. Since “B” does not hold, w0 >γ . It follows that Snk < (1 2 ) . We get k k · k k − · (1 (γ2/2)) y − y . nk,x0 ≤k nk,x0 k Let x1 = Ax0 + w0.Wehave x1 1 . So, after normalization we get (with x1 k k≥ − x10 = x ) k 1k 1 (γ2/2) −1 1 y − yn ,x . nk,x10 ≤ 1 ·k k 0 k − 1 (γ2/2) We now start the process over with x instead of x , e = − instead of . 10 0 10 1 − Either we have “B”orwegetx20 with 1 (γ2/2) 1 (γ2/2) − − 1 1 1 y − − 1 y . n ,x nk,x0 k 20 ≤ (1 )(1 1) ·k k − − 1 γ2 n We see that the sequence x0,x10 ,x20 , ... converges, since xn0 xn0 1 < 1− ) . 2 k − − k − 1 (γ /2) n Since n < ( −1 ) ,eitherwehave“B00 for some n, or with x = lim xn0 we − · will have x = T nk w with w License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use EXTREMAL VECTORS AND INVARIANT SUBSPACES 545 Proof. We can assume that 0 σ(T ). Represent T as multiplication by z on L2(µ), ∈ where µ is a regular Borel measure on σ(T ). Let x0 = f. We have the following lemma, whose proof is obvious. Lemma 2. Given >0, there are a δ>0and 0 <α<1such that if g(z)=0for z <δ,g(z)=αf(z) for z = δ and g(z)=f(z)for z >δ,then f g =.If |µ|z:z=δ =0,thenα|is| arbitrary. | | k − k { || } 2n n n n n 1 n δn z We have T y = δn T (I δn T ∗ T )− T ∗ x0 = | | 2n f(z). Let δ and α be n − 1+δn z · as in the above lemma. We will show that | | δ z 2n (1) n| | 0 z <δ. 1 δ z 2n → ∀| | − n| | 2n δn δ0 Assume not. Then, there are δ0 <δand γ>0 such that 1+δ| δ| 2n >γfor infinitely n| 0| 2nν many n, say for the sequence nν . Then for all z with z >δ0 we have δnν z 2nν | | | | →∞ δnν z as ν .So, ||2n 1 for all z with z >δ0. But by Lemma 2 and since 1 δνz ν →∞2nν − || → 2n| ν| δnν z δnν z 0 | | 2nν < 1 for all z,weget | | 2nν f f <for ν large enough. ≤ 1 δν z 1 δnν z · − This contradiction− | | gives (1), and in the− same| | way we get δ z 2n (2) n| | 1 z >δ. 1 δ z 2n → ∀| | − n| | 2n δn z Now if µ z : z = δ > 0, then (1), (2), and Lemma 2 give | | 2n α 1 δn z { | | } 2n − | | → δn z z = δ,asotherwise | | 2n f f =would not hold for n large enough. ∀| | 1 δn z · − Hence, T ny h in norm,− where| | h(z)=0if z <δ,h(z)=α f(z)if z =δ,and n → | | · | | h(z)=f(z)if z >δ.Let Sδ= z:z δ , and let Sδ be the characteristic | | 2 { ||≥2 } K2 function of Sδ.Thenh L(µ) Sδ L(µ), and Sδ L (µ) is a hyperinvariant subspace for T .So,his· a hypernoncyclic⊂K · vector forKT . · Remark 2. A. Note that in the proof of Theorem 7, as 0, δ 0. So, as 0, → → → KSδ 1σ(T ). It follows that if x0 =1σ(T),thenas 0, the hyperinvariant subspaces→ corresponding to grow to H. → B. The proof of Theorem 4 can be written in terms of (vn). C. From the proof of Theorem 4 it is clear that if T is quasinilpotent and either (v )or(Tny ) converges in norm for some x and some with 0 << x , n,x0 n,x0 0 0 then T has hyperinvariant subspaces. k k D. Suppose T is either a compact operator or a normal operator such that (T ) is dense and is not the whole space. Then, the set of hypernoncyclic vectors ofRT is dense. In particular, if an operator A commutes with T ,thenAhas a dense set of noncyclic vectors. In other words, any operator with an open set of cyclic vectors does not commute with T . 3. Operators of class R Suppose T : H H is a noninvertible bounded linear operator with dense range → and x0 (T ). In the previous section it was shown that if T is normal then (T ny 6∈) converges R in norm to a hypernoncyclic vector, that if T is quasinilpotent n,x0 and (T ny ) converges in norm, then the norm limit is hypernoncyclic, and that if n,x0 T is compact, then some weak limit of (T ny ) is hypernoncyclic. This shows that n,x0 the norm convergence of (T ny ) is an important issue, and raises the following n,x0 License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 546 SHAMIM ANSARI AND PER ENFLO question. Suppose T is a nonsurjective bounded linear operator with dense range. Suppose the set = x : x (T ) and T ny y in norm = .Isittrue 0 0 n,x0 x0 that for some x N {the norm6∈ limit R y of (T ny →) is noncyclic?}6In∅ this section 0 x0 n,x0 and the next we∈N work on the norm convergence of (T ny )and(v ). Remark n,x0 n,x0 7 concerns the question we just posed. Definition 2. Let T : X X be a continuous linear operator. A vector x in X is → T nx said to be quasimaximal for T if k T n k δx for some δx > 0. k k ≥ Lemma 3. Let T : H H be any bounded linear operator, x0 H, x0 =1 and 0 << x .Letδ→ be the negative constant appearing in the∈ orthogonalityk k 0 n,x0 k k n n equation v x = δ T ∗ T v . Then, the following hold true. n,x0 − 0 n,x0 n,x0 a. < δ T n 2 n 0. − n,x0 k k ∀ ≥ b. The sequence ( δ T n 2) is bounded from above if and only if there exists n,x0 − k Tnkx a δ such that 0 <δ< kTn k x B(x0,) n. k k ∀ ∈ ∀ c. If x0 is quasimaximal for T , then there exist >0and δ > 0 such that T nx k k δ > 0 n and x B(x ,). That is, the set of all quazimaximal T n ≥ ∀ ∀ ∈ 0 vectorsk k is open. (Here T can be assumed to be an operator from X to X.) d. If x is quasimaximal, then there exists an >0such that ( δ T n 2) is 0 n,x0 bounded from above. − k k Proof. (a) Let v = v . From the orthogonality equation we have n n,x0 n n n 2 n 2 = δ T ∗ T vn δ T vn δ T . − n,x0 k k≤− n,x0 k k k k≤− n,x0 k k Tnx (b) Suppose 0 <δ< kTn k n x B(x0,). Then, k k ∀ ∀ ∈ n 2 n n T v T ∗ T v δ2 k nk k nk = . ≤ T n 2 ≤ T n 2 δ T n 2 k k k k − n,x0 k k So, δ T n 2 . Conversely, suppose the sequence ( δ T n 2) is bounded. n,x0 δ2 n,x0 From− equationk k (1)≤ we have − k k n n n 2 n n T ∗ T δ T = δ T ∗ T = M. n,x0 n,x0 k n n k − k k − k k T ∗ T vn ≤ n n k k T ∗ T n n n n So, k k T∗ T v T∗ T v . Hence, M ≤k nk≤k kk nk T nv T nx k nk k k n 0, x B(x ,). M ≤ T n ≤ T n ∀ ≥ ∀ ∈ 0 n k k k k T x0 (c) Let 0 <ρ k Tn k n 0. Let 0 <<ρ. Then, ≤ k k ∀ ≥ Tnx Tny Tnx Tny x y < = k 0k k k 0 <. k 0 − k ⇒ Tn − Tn ≤ Tn − Tn k k k k k k k k T ny That is, ρ k k .Letδ=ρ . − ≤ T n − (d) is clear fromk (b)k and (c). Remark 3. a. From Lemma 3 it follows that if T is quasinilpotent injective and n 2 with dense range, then for any x0 with x0 =1, δ T . k k − n,x0 k k →∞ b. If x is nonquasimaximal for T , then the space Ax : AT = TA consists of nonquasimaximal vectors. Consequently, if T has a quasimaximal{ and} a nonzero nonquasimaximal vector, then T has hyperinvariant subspaces. License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use EXTREMAL VECTORS AND INVARIANT SUBSPACES 547 T nk+1 c. Suppose T is quasinilpotent. Then, k k 0 for some subsequence. So, T nk → for any vector x the vector Tx cannot bek quasimaximalk for T .ThusifThas a quasimaximal vector x,thenTx is hypernoncyclic. It is easy to see that for any dense range operator A, if a vector Ay is hypernoncyclic (cyclic) for A,thenyis itself hypernoncyclic (cyclic) for A.Thus,ifTis quasinilpotent with dense range and with a quasimaximal vector, then all vectors are hypernoncyclic for T . Definition 3. A bounded linear operator T : X X is said to be of class if each nonzero vector is quaisimaximal for T . → R Definition 4. A sequence (an) of real numbers is called admissible for a bounded linear operator T if there exist constants k>0andK< such that for all n 1 ∞ ≥ k Tn a K Tn . ·k k≤ n ≤ ·k k Remark 4. a. Suppose (an) is a sequence such that for each nonzero vector x there exist δx and τx with Tnx ( ) 0 <δx k k τx < . ∗ ≤ an ≤ ∞ Then there exist constants k and K such that ( ) k Tn a K Tn . ∗∗ ·k k≤ n ≤ ·k k Consequently, if ( ) holds for each nonzero x,thenTis of class . Conversely, if ∗ R T is of class ,and(an) is an admissible sequence, then ( ) holds for each nonzero vector x. R ∗ b. If T is of class ,thenr(T)>0. If (an) is an admissible sequence for T and an+1 R t,thent=r(T). Recall that a C1-contraction is an operator T : X X an → n → such that T =1forallnand for each x = 0 there exists δx > 0 such that n k k 6 δx < T x for all n. Clearly, a C1-contraction is of class . Recall that an operatork T kis said to be power bounded if there exist δ>0andRM< such that δ< Tn Let l∞ denote the Banach space of all bounded sequences of complex numbers. Recall (see [3]) that there are linear functionals : l∞ and : l∞ with the following properties. F →C G →C (1) ((ξ )) = limit of a subsequence of (ξ ). F n n (2) ((ξn)) ((ξn0 )) if ξn, ξn0 are real and ξn ξn0 n 1. F ≤F ξ1+...+≤ξn ∀ ≥ (10) ((ξ )) = limit of a subsequence of ( ). G n n (20) ((ξ )) ((ξ0 )) if ξ , ξ0 are real and ξ ξ0 n 1. G n ≤G n n n n ≤ n ∀ ≥ (30) ((ξn)) = ((ξn+1)). That is, is invariant under the shift. RecallG that a linearG functional orG as above is a Banach limit (see [3]). In F G this article we will consider Banach limits that satisfy either 1 and 2 or 10 30. − License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 548 SHAMIM ANSARI AND PER ENFLO Remark 5. a. Suppose T : H H is of class ,(an) is an admissible sequence for T ,and is any Banach limit.→ Then, the functionR [ , ]:H H defined by F · · → T nx T ny [x, y]an = , F an an D E 2 is an inner product on H and it defines a Hilbert norm ,an on H by x ,a = F F n T nx 2 k·k k k k k n a2 . The norms ,an and , T are equivalent. F n k·kF k·kF k k b. Suppose T : X X is of class and (an) is an admissible sequence for T . → R T nx Let be any Banach limit. Define x ,a = k k for all x X. Then, F k kF n F an ∈ n ,an is a norm on X. The norms ,an and , T are equivalent. F F F k k k·kc. Suppose T is of class ,(a ) is ank·k admissible sequencek·k for T and is a Banach R n G limit invariant under the shift (i.e. (xn)= (xn+1)). Then, T extends uniquely to G G a a bounded linear operator T˜ from the completion X˜ to itself. If lim n+1 = r,then an T A= :(X, ,a ) (X, ,a ) is an isometry, which obviously extends to r k·kG n → k·kG n an isometry A˜ from the completion X˜ of X onto itself. In particular, if T is power bounded, (a ) is any admissible sequence for T with an+1 1(e.g. a =1 n), and n an n ˜ ˜→ ˜ ∀ is a Banach limit invariant under the shift, then T :(X, ,an ) (X, ,an) isF an isometry. The case when T is power bounded has alreadyk·kF been→ consideredk·kF by C. Foias and B. Sz.-Nagy [4]. Proof. (a) It is easy to see that ,a is a Hilbert norm. For the equivalence of k·kF n n ,an and , T note that there exist constants m and M with k·kF k·kF k k T nx 2 T nx 2 T nx 2 m2 k k k k M 2 k k . · T n 2 ≤ a2 ≤ · T n 2 k k n k k (b) This is similar to (a). (c) Suppose a is defined as in (a). k·kF n Tn+1x 2 T n+1x 2 a 2 Tx 2 = = n+1 . an k 2 k k 2k 2 k k F an F an+1 · an For any >0wehave Tn+1x 2 Tn+1x 2 a2 Tn+1x 2 (r )2 n+1 (r + )2 . k 2k k 2k 2 k 2k F an+1 · − ≤F an+1 · an ≤F an+1 · It follows that T n+1x 2 a2 Tn+1x 2 n+1 = r2 = r2 x 2 . k 2k 2 k 2k an F an+1 · an ·F an+1 ·k k The proof is similar if ,a is defined as in (b). k·kF n Remark 6. A result of C. Foias and B. Sz.-Nagy [4] states that if T : H H is n→ power bounded, and for each x =0there exists δx > 0 such that δx < T x and n 6 k k δx < T ∗ x n,thenT is quasisimilar to a unitary operator, and hence it has hyperinvariantk k∀ subspaces. This result extends (with a similar proof) to a Hilbert space operator of class as follows. If T and T ∗ both are of class and if there R an+1 T R is an admissible sequence (an) such that r = r(T ),then is quasisimilar an r to a unitary operator. Hence, T has hyperinvariant→ subspaces. Notation. Let T : X X be a bounded linear operator with dense range. Let n → X = x X : T − x exists n . Recall that X is dense (see [1]). 0 { ∈ ∀ } 0 License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use EXTREMAL VECTORS AND INVARIANT SUBSPACES 549 Lemma 4. Suppose T : X X is of class , (an) is a sequence such that for → n R n some positive constants k and K, k T an K T for all n 0 and an+1 r. Then, the following hold true.·k k≤ ≤ ·k k ≥ an → n n − − n a. F = g (X, )0 : T∗ g exists n, and supn T ∗ g r < = 0 . n { ∈ k·k ∀− n k k· ∞} 6 { } b. For any g F define g =supn T∗ g r ,then, is a norm on F . c. Let x X ∈. | | k k· |·| ∈ 0 1. If is any Banach limit for which ((ξn)) = the limit of some subse- m F m Fr− quence of (ξn),then, T− x ,an k x . T k kF ≤ ·k k 2. Suppose r is power bounded and is any Banach limit. Then, there Fm m exists a constant K0 such that T − x ,an K0 r− x . m k˜ kF ≤ · ·k k Consequently, T − :(X0, ) (X, ,an ) is continuous. k·k → k·kFm m d. Let be any Banach limit such that T − x ,an K0 r− x for all F k kF ≤ · m ·k k ˜ ˜ − ˜ x X0 and some K0. Then, for any f (X, ,an )0, T ∗ (f X ) exists for all∈m. ∈ k·kF | ˜ ˜ ˜ e. Let be as in d. Then, for any f (X, ,an )0 we have f = f X F and F ˜ ∈ k·kF | ∈ f K0 f ,an . | |≤ ·kT kF f. Suppose r is power bounded. Suppose is any Banach limit. Suppose F is F ˜ -dense in (X, )0. Then, the mapping J :(F, ) (X, ,a )0 defined k·k k·k |·| → k·kF n by J(f)=f˜is an isomorphism, where f˜ denotes the unique extension of f ˜ ˜ to all of X. That is, there exist K0 and K00 such that K00 f ,an f ˜ ·k kF ≤| |≤ K0 f ,an .If is invariant under the shift, then T extends to a unique ·k kF F ˜ ˜ bounded linear operator T from (X, ,an ) to itself. ˜ ˜ ˜k·kF 1. For any f in F , (T )∗f =(T∗f)=J(T∗f). 2. Suppose A is any subspace of F .Then,Ais invariant under T ∗ iff JA ˜ is invariant under (T )∗. Tnx Proof. a. Let x =0and0<δx < k k.Let(fn) be a sequence in (X, ) an 6 n n nk·k T x T ∗ fn T∗ fn such that fn =1and fn >δx. Then, (x) >δx.Thatis, k k an an an 6→ 0inthew∗-topology. Since this sequence is bounded, it has a w∗- convergent n1 n1 1 T ∗ fn1 T∗ − fn1 1 subsequence, say g1 in (X, )0.As k k is bounded (by ), an1 an1 1 k → k·kn2 1 − T ∗ − fn2 it has a convergent subsequence, say g2.Notethat an2 1 − → n2 1 n2 − T∗ fn2 an2 1 T∗ fn2 = − T∗ . an2 an2 · an2 1 − 1 It follows that g = T ∗g .Thatis,T∗g=rg. Applying this argument 1 r 2 2 ·1 repeatedly and passing to the diagonal subsequence, we get a sequence (nk)of nk m T ∗ − fnk natural numbers such that gm in the w∗-topology. Since an m k− → n n m k k− T ∗ fnk ank m m T ∗ fnk = − T ∗ , ank ank · ank m − m m m − m it follows that T ∗ gm = r g1.Thatis, T∗ g1 r = gm .Notethat 1 · k k· k k gm k. This means that g1 F . kb.k≤This is obvious. ∈ License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 550 SHAMIM ANSARI AND PER ENFLO c. 1. There is some subsequence (nk) of natural numbers such that nk m nk m m T − x T − x ank m T − x an, = limk k k = limk k k − k k F →∞ an →∞ an m · an k k− k 1 m r− x . ≤ k · ·k k T n 2. Since r is power bounded, there exist c>0andC< such that c T n n ∞ ·k k≤ T C T n. In view of (1) we can assume that ((ξn)) = the limit of a m k k≤ ·k k∀ξ1+ +ξn − F subsequence of ···n .Letx X0. T∗ x ,an = limit of a subsequence of i m ∈ k kF i m j m T − x j m T − x − k k = the limit of a subsequence of − k k .Eachterm i=1 j ai i=m j ai · · j m j m P − i m − i m P T− x T− x ai m 1 (j 2m) C m k k= k k − − r− x . j a j a · a ≤k· j ·c· ·k k i=m i i=m i m i X · X · − C Let K0 = k c . d. For any· x X , with x 1, ∈ 0 k k≤ m m ˜ m ˜ m ˜ r− f(T − x) = f(T − x) f a , T− x ,a f ,a . | | | |≤k k n F ·k kF n ≤k kF n · k m Thus, fT− is uniformly bounded on the -unit ball of X0. k·km Let gm be the unique extension of fT− to a bounded linear functional on m m − (X, ). It is easy to verify that T ∗ gm = f for all m.Thatis,T∗ fexists for all mk·k. e. We have m − m m m T ∗ f r =supx 1,x X f(T − x) r k k· kk≤ ∈ 0 k k· m ˆ m m ˆ r− m ˆ 1 f ,a T− x ,a r f ,a r f ,a . ≤k kF n ·k kF n · ≤k kF n · k · ≤k kF n · k 1 ˆ So, f k f . f.| Let|≤ c>·k0andk C< be such that c Tn rn C Tn n. In view ∞ ·k k≤ ≤ ·k k∀ of (e) we need only to verify that if f F ,thenfhas a unique extension f˜ and ˜ ∈ f (constant) f .Letx Xbe such that x ,a 1. k k≤ ·| | ∈ k kF n ≤ n n n n n − n − n T x f(x) = f(T − T x) = T ∗ (f)(T x) T∗ f r k k | | | | | |≤ k k· · rn n n − n T x K T∗ f r k k . ≤ k k· · an · c K ˜ ˜ K So, f(x) f x an c .SinceX0is dense in X, it follows that f ,an c f . | |≤| |·k k · ˜ ˜ ˜˜ ˜ k kF ≤ ·| | (1) For any x in X we have (T )∗f(x)=f(Tx)=f(Tx)=f(Tx)=T∗f(x). ˜ ˜ ˜ ˜ ˜ That is, (T )∗f X = T ∗f or (T )∗f =(T∗f). | ˜ ˜ (2) Suppose A is invariant under T ∗.Letf JA.ThenfX=f A.So, ∈ ˜ ˜ |˜ ˜ ∈ T∗f A,andJ(T∗f) JA.By(1) above, J(T ∗f)=(T)∗f.So,(T)∗f JA.Now ∈ ∈ ˜ ˜ ˜∈ ˜ suppose JA is invariant under (T )∗.Letf A.Thenf JA and (T )∗f JA. ∈ ∈ ∈ That is, J(T ∗f) JA.So,T∗f A. ∈ ∈ T Theorem 8. Suppose T : X X is a bounded linear operator such that r is power → n n − − n bounded. Let F = x (X, )0 : T∗ x exists n, and supn T ∗ x r < . Let a be an admissible{ ∈ sequencek·k for T and be any∀ Banach limitk invariantk under∞} n F License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use EXTREMAL VECTORS AND INVARIANT SUBSPACES 551 the shift. Let T˜ denote the unique extension of T to an operator from the completion ˜ (X, ,a ) of (X, ,a ) to itself. Then, the following hold true. k·kF n k·kF n a. Some nonzero element of X is noncyclic for T˜ iff some nonzero element of F is noncyclic for T ∗. ˜ 1 b. Some nonzero element of X is noncyclic for (T )− iff some nonzero element 1 of F is noncyclic for (T ∗)− .(Here,Tis assumed to be invertible.) Proof. a. Suppose some nonzero element of F is noncyclic for T ∗. Then there is a proper closed subspace M of X0 invariant under T ∗ such that M F = (0). Clearly, ∩ 6 M F is invariant under T ∗.LetJbe an isomorphism as in Lemma 4 (f). Then, ∩ ˜ J(M F ) is invariant under (T )∗ by Lemma 4 (f) (2). Consequently, the polar ∩ ˜ ˜ [J(M F )] X˜ of J(M F )inXis invariant under T . We make the following assertions.∩ ⊥ ∩ ˜ (1) [J(M F )] X˜ = X. ∩ ⊥ 6 (2) [J(M F )] ˜ X =(0). ∩ X ∩ 6 Clearly, (2) implies⊥ that a nonzero element of X is noncyclic for T˜. ˜ Proofof(1).If[J(M F)] X˜ = X,thenJ(M F) = (0). So, M F =(0). This is a contradiction. ∩ ⊥ ∩ ∩ ˜ Proofof(2).If[J(M F)] ˜ X = (0), then for each x in X, there exists an f ∩ X ∩ in J(M F ) such that f˜(x) =0.Thatis,foreach⊥ xin X there exists f in M F ∩ 6 ∩ such that f(x) = 0. Therefore, the closure of M F is X 0. This is a contradiction. 6 ∩ For the converse suppose that some nonzero element of X is noncyclic for T˜. ˜ Then there is a proper ,an -closed subspace M of X such that M X =(0). k·kF ∩ 16 Clearly, M ⊥ JF,andM⊥ is invariant under (T˜)∗. By Lemma 4 (f) (2), J − (M ⊥) ⊂ 1 is invariant under T ∗.So,the -closure of J− (M ⊥) is invariant under T ∗.We k·k 1 k·k 1 k·k claim that J (M ) = X0. Taking the claim for granted, J (M ) is a − ⊥ 6 − ⊥ proper closed subspace invariant under T ∗ and intersecting with F . We now prove 1 k·k 1 the claim. If J − (M ⊥) = X0,then[J− (M⊥)] X =(0).But(0)=M X 1 ⊥ 6 ∩ ⊂ [J− (M⊥)] X . b can be⊥ proved the same way. 4. Norm convergence of extremal vectors Proposition 2 (Duality between the forward and the backward minimal points). Let T : H H be a bounded linear operator. → , n n , (a) If y ∗ is the backward minimal point for T ∗ ,and0 = T∗ y ∗ ,then n,x0 k n,x0 k 0 n n v = x0 T ∗ y is the forward minimal point for T . Conversely, if n,x0 − n,x0 v is the forward minimal point for T n,and = v ,thenT ny0 = n,x0 0 n,x0 ∗ n,x0 x v is the backward minimal point for T n. k k 0 n,x0 ∗ − (b) Let x0 be a fixed vector in H with x0 =1.Letβnbe the angle between T ny and T ny x , α , be thek anglek between v , and (v , x ), n,x0 n,x0 0 n∗ n,x∗ 0 n,x∗ 0 0 and α be the angle between− v and (v x ).Then, − n n,x0 n,x0 0 , − (1) lim 1Cos βn = lim 0Cosαn∗ ,and → →n T x0 (2) lim 0Cos αn −k T n k . → ≤ k k (c) Suppose for each 0 << x0 and for each sequence (nk ) converging to nk k k , vn v in norm. Then, there exists a subsequence (n ) such that k → x0 kl License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 552 SHAMIM ANSARI AND PER ENFLO n n kl , kl , T ∗ y∗ y∗ in norm, where T ∗ y∗ is the backward minimal nkl ,x0 x0 nkl ,x0 n→k vector for T ∗ . n , k ∗ nk (d) Let 0 << x0 . Suppose for each sequence (nk ) converging to , T ∗ ynk,x0 k k yx∗ in norm. Then, there exists a subsequence (nkl ) such that vn vx0 → 0 kl → in norm. Proof. (a) Let y, = y . y satisfies the equation T n y = δ T n T n(T n y x ) n,x∗ 0 n∗ n∗ ∗ n∗ n ∗ ∗ n∗ 0 n n n n − for some δn < 0. Set vn = x0 T ∗ yn∗ . Then, T ∗ yn∗ = δnT ∗ T vn.So, n n − − n n (v x)=δ T∗ T v . Since the equation (X x )=aT ∗ T X has a unique n−0 n n − 0 solution of the form (a, X)fora<0, it follows that vn is a forward minimal point. n As the v x = T y = , v = v0 . n 0 ∗ n∗ 0 n n,x0 k − k k k , , n (b) Let y = y , v = v ,andv∗ = v∗ and 0 = T y .Wehave n n,x0 n n,x0 n n,x0 k nk (Tny x,Tny)=( v ,0 ,x v ,0 ). So, T ny Cosβ == v ,0 Cosα ,. n 0 n n∗ 0 n∗ n n 0 n,x∗ 0 n∗ − − , − ·k k· ·k k· v∗ 0 Cos α k n k· n , Therefore, Cosβn = .As 1,0 0,and vn∗ ø1. The first equation is now immediate. → → k k We have (vn x0 ,vn)= vn cosαn. On the other hand, from the orthogonality equation − ·k k· n n n 2 (v x ,v )=(δ T∗ T v ,v )=δ T v . n − 0 n n n n nk nk So, δ T nv 2 T nv 2 T nv 2 T nv cosα = nk nk = −k nk −k nk = −k nk . n v 2 v 2 T nTnv ≤ v 2 Tn Tnv v 2 Tn k nk k nk ·k ∗ nk k nk ·k k·k nk k nk ·k k Note that n n T vn T x0 lim 0 −k k = −k k. → v 2 Tn T n k nk ·k k k k n kl , (c) Let (nkl ) be a subsequence such that ( T ∗ yn∗ )=(nk ) converges in k kl k l norm to say 0. Then, 0 <0 < x0 . Now the result follows from (a). (d) This is similar to (c). k k Tnx Thus, if x0 is quasimaximal for T and 0 <δ< kTn k for all n,thenforsome δ >0, lim Cos α δ n. Example 2 (below)k k shows that x does not 0 0 n,x0 0 0 → ≤− ∀ have to be quasimaximal for lim 0Cos αn,x δ0 to hold true. → 0 ≤− Our definition of minimal points make sense if the space is a strictly convex reflexive Banach space. More precisely, if X is reflexive and T : X X is a bounded operator, then the set T ny : y x is weakly compact→ for any { k − 0k≤ } x0 X and 0 << x0 . So, the norm of X attains its minimum over this set at a∈ point of the set.k Ifk the norm of X is strictly convex, then the minimum is attained at a unique element of this set. That unique element can be defined to be the forward minimal point v . Similarly, the definition of backward minimal n,x0 points can be extended. Recall that a result of M. I. Kadets [2] states that any separable Banach space admits an equivalent new norm which is locally uniformly convex. Recall also that a locally uniformly convex norm is strictly convex and has the property that if (xn) is a sequence such that x x weakly and x x ,thenx xin norm. n → k nk→k k n → Proposition 3. Suppose X is a reflexive locally uniformly convex Banach space and T : X X is any bounded linear operator. Let v = v . → n n,x0 License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use EXTREMAL VECTORS AND INVARIANT SUBSPACES 553 nk T vnk a. If vn v0 weakly and if k n k 1,thenvn v0 in norm. k → T k v0 → k → b. Suppose T is of class , (ak ) is ank admissible sequence for T ,and is any R n F Banach limit. If vnk v0 in norm, then T nk v → 1. k nk k 1 and T nk v k 0k → 2. v0 ,a x ,a . k kF nk ≤k kF nk Proof. (a) Suppose not. Since X is locally uniformly convex, v0 x0 <.So, T nk v k − k tv x = for some 0 nk nk nk nk T v0 T vnk T v0 T vnk k k k k vnk v0 0. T nk − T nk ≤ T nk − T nk ≤k − k→ k k k k k k k k So, T nk v T nk v 1 k nk k k 0k 0. − T nk v T nk → k 0k k k n nk k T v0 T vnk Since, k n k δ>0, it follows that k n k 1. T k ≥ T k v0 → k Tknk v k k (2) Let k nk k = δ . Then, δ < 1 k and δ 1. For any x B(x ,) T nk v nk nk nk 0 k 0k ∀ → ∈ we have T nk x Tnkv =δ Tnkv .So, k k≥k nkk nkk 0k nk nk nk T x T vnk T v0 k k k k=δnkk k. ank ≥ ank ank It follows that x ,a v0 ,a . k kF nk ≥k kF nk A corollary of Theorem 11, which appears in the appendix, is given bellow. Corollary. Suppose X is a strictly convex smooth reflexive Banach space, T : X → X is of class , (an) and (bn) are any admissible sequences and is any Banach R F limit. If for each x0 =0and each 0 << x0 , the sequence v v in norm, 6 k k n,x0 → x0 then there exists a K such that ,a = K ,b . k·kF n ·k·kF n Theorem 9. Suppose (X, ) is a reflexive locally uniformly convex Banach space, k·k T : X X is an operator of class and (an) is an admissible sequence for T .Let → R n n x0 X,0<,n < x0 ,n ,andv =v . ∈ k k → n n,x0 a. Suppose for a weak limit point v of vn : n 1 there is a norm on 0 { n ≥ } k·kv0 X such that v0 v0 x v0 x B(x0,) .Thenv0is a norm limit point n k k ≤k k ∀ ∈ k·k of vn : n 1 . { ≥ } T nv k k b. Suppose for each k the sequence k k k is increasing. Then, for each an n=1 weak limit point v of vn : n 1 , there is a norm on X such that 0 { n ≥ } k·kv0 v0 v0 xv0 x B(x0,) . Consequently, each weak limit point of k nk ≤kk ∀ ∈ k·k vn : n 1 is a norm limit point of it. { ≥ } n k n k k k T vk T vk T vk c. Suppose limn k k exists and limn k k k k for each k. Then, for an an ≤ ak each weak limit point v of vk : n 1 , there is a norm on X such 0 { n ≥ } k·kv0 that v0 v0 x v0 x B(x0,) . Consequently, each weak limit point of k k ≤k k ∀ ∈ k·k vn : n 1 is a norm limit point of it. { n ≥ } License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 554 SHAMIM ANSARI AND PER ENFLO n n k k k T x T vk T vk d. Suppose limn k k exists for all x X and limn k k k k for an an ak n ∈ ≤ each k.Then,vn v0in norm for some v0. In particular, if T is a C1- contraction, then (v→) converges in norm for each x =0and 0 << x . n,x0 0 6 k 0k Proof. (a) Suppose not. Since, (X, ) is locally uniformly convex, it follows that k·k v0 x0 <. So, for some t<1, tv0 x0 = . By the hypothesis, it follows thatk − v k t v . This is a contradiction.k − k k 0kv0 ≤ k 0kv0 (b) Let be any Banach limit. Let v0 be any weak limit point of the set and nk F let vn v weakly. For any m we have k → 0 m m nk T v0 T vnk k k lim infk k k am ≤ am n nk k T vnk lim infk k k ≤ ank n n kl T kl vn = lim k kl k (for some subsequence (n )of(n )) l a kl k nkl n T kl x lim inf k k for each x B(x ,) ≤ l a ∈ 0 nkl n T kl x Define x = k k for all x X. It follows that v0 an k k F kl ∈ n n T kl v T klx k 0k k k . F ank ≤F ank l l That is, v x x B(x ,). Since (const) , it follows that k 0kv0 ≤k kv0 ∀ ∈ 0 k·kv0 ≤ k·k v0 v0 x v0 x B(x0,) . k k ≤k k ∀ ∈ k·k n (c) Let v be any weak limit of the set and let v kl v weakly. Define 0 nkl 0 T nk x → k k x v0 = a .Since xv0 (constant) x for all x X, it follows that k k F nk kk ≤ ·k k ∈ nk vn v weakly in (X, ). Thus, k → 0 k·kv0 n nk k nk T vnk v0 v0 lim infk vnk v0 lim infk k k k k ≤ k k ≤ ank T nk x lim infk k k x B(x0,) . ≤ ank ∀ ∈ k·k It follows that v0 v0 x v0 for all x B(x0,) . k k ≤k k ∈ k·k T nx (d) Let v0 be any weak limit of the sequence (vn). Define x a = limn k k k k n an nk x X.Letvn v weakly in (X, ). Then, ∀ ∈ k → 0 k·k T mv nk nk nk v0 an lim infk vnk an = lim infk limm k k k k ≤ k k am n nk k nk T vnk T x lim infk k k lim infk k k. ≤ ank ≤ ank nk It follows that v0 an x an.By(a) it follows that vnk v0 in norm. Now suppose if possiblek k that≤kz isk a norm limit point of the set v :→n 1 and v = z . 0 { n ≥ } 0 6 0 By strict convexity of , v0+z0 x <. So, for some 0 License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use EXTREMAL VECTORS AND INVARIANT SUBSPACES 555 v0+z0 v0+z0 t B(x0,) .Notethat v0 an = z0 an.So,tv0 an t 2 ∈ k·k k k k k k k ≥ 2 an ≥ v . This is a contradiction. 0 an k k Theorem 10. Let X be a reflexive locally uniformly convex Banach space. Let T : X X be such that T =1. → k k a. Suppose there are x X and 0 << x such that (y ) is bounded. 0 0 n,x0 ∈ n k k n Then, y0 F = x : T − x exists n, Supn T − x < for each weak limit y of∈(T ny { ). ∀ k k ∞} 0 n,x0 n n b. Suppose the vector space F = x : T − x exist n, Sup T − x < is { ∀ nk k ∞} dense. Let x0 X and >0be such that 0 << x0 . Then the following hold. ∈ k k 1. (y ) is a bounded sequence. n,x0 2. T ny y in norm for some y . n,x0 0 0 → n n 3. y0 F = x : T − x exist n, Supn T − x < . ∈ { ∀ k k ∞} n 4. y0 =inf x : x B(x0,) F ,where x = Supn T − x is a norm on| |F . {| | ∈ k·k ∩ } | | k k Proof. (a). Let y = y for all n. Suppose y M n. Then, T ky M n n,x0 n n n, k 1. Suppose z is a weak limit point ofk (k≤T ny ).∀ Since z k x k≤= < ∀ ≥ 0 n k 0 − 0k x0 ,z0 = 0. We can find a subsequence (nk) of natural numbers such that knkkm 6 m T − ynk zm weakly for some zm for all m 0. Clearly, zm = T − z0 and m → ≥ T − z0 = zm sup yn M. This implies z0 F . k (b) Letk yk =k≤y fork allk≤n. Since the space F∈is dense, the sequence (y )is n n,x0 n bounded. Suppose T nk y y for some y weakly. By (a), y F . nk → 0 0 0 ∈ Claim. y0 =inf x : x B(x0,) F . | | {| | ∈ k·k ∩ } nl m m Proof of the claim. Let (nl) be a subsequence of (nk) such that T − ynl T − y0 for each m 1. We have → ≥ m n m T − y lim inf T l− y k 0k≤ lk nl k lim inf y ≤ lk nl k nl lim infl T − x for each x B(x0,) F ≤ k k ∈ k·k ∩ n supn T − x for each x B(x0,) F ≤ k k ∈ k·k ∩ Therefore, y0 x for each x B(x0,) F . | |≤| | ∈ k·k ∩ Now an argument similar to the one in the proof of Theorem 9 (a) implies nk that T ynk y0 in norm. Further, as in the proof of Theorem 9 (d), the strict → n convexity of implies that T yn y0 in norm. The next remarkk·k follows from Theorem→ 10 (b). Remark 7. Let X be a Banach space, let X0 have a locally uniformly convex norm, and let T : X X be a C1-contraction. Then, by Lemma 4 (a) F = x X0 : n n − → − { ∈ T ∗ x exists n, Sup T ∗ x < = 0 .IfFis not dense in (X, )0,then ∀ nk k ∞} 6 { } k·k Fis an invariant subspace for T . Suppose F is dense in (X, )0.Letx0 X0and n k·k ∈ n 0 << x0 .LetT∗ yn∗denote the backward minimal points w.r.t. (x0,,T∗ ). Then, thek followingk hold. n a. T ∗ y∗ y in norm for some y . n → 0 0 License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 556 SHAMIM ANSARI AND PER ENFLO n n − − b. y F = x X0 : T ∗ x exists n, sup T ∗ x < . 0 ∈ { ∈ ∀ nk k ∞} Remark 8. Suppose T : H H is a C1-contraction. If H does not contain any nonzero noncyclic vector for→T˜, then by Theorem 8 and Remark 6 it follows that all the norm limits of the sequences (T n y, ) are cyclic vectors for T .B.Beauzamy ∗ n,x∗ 0 ∗ defined an invertible C1-contraction Mφ on Hilbert space H and proved that no ˜ ˜ nonzero vector of H is noncyclic for Mφ,whereMφ is the unique extension of Mφ ˜ n to an operator on (H, )and x = limn T x (see [1]). (Note that ||| · ||| ||| ||| →∞k k n = , Tn for any Banach limit .) Thus, the norm limit of (Mφ∗ yn,x ) ||| · ||| k·kF k k F 0 is cyclic for M ∗ for each x =0and0<< x . It is worth mentioning φ 0 6 k0k that by a result of B. Beauzamy [1, Ch. XII], for T belonging to a large class of C1-contractions on a Banach space X (not necessarily locally uniformly convex), the space X contains a nonzero noncyclic vector for T˜. So, by Theorem 8 for T belonging to a large class of C1-contractions on X,thespaceF= x X: n n − − { ∈ T∗ xexists n, Sup T ∗ x < contains a nonzero noncyclic vector of T ∗. ∀ nk k ∞} Example 1. We define an operator T : l2 l2 such that T is of class , T n → T n T n R k k→ and for any real number t = 1, either t 0or t .Let ∞ 6 2→ →∞ en:n=0,1,2, ... be an orthonormal basis for l .Let(an) be a sequence of { } k positive real numbers such that a0 a1 a2 an =( n+1) . Define Te n =an en+1 and extend T to all of l2 to be a bounded· · ··· linear operator. It is easy to check· that n k T n T is of class , T = n , r(T )=1,for0 License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use EXTREMAL VECTORS AND INVARIANT SUBSPACES 557 n n bn T ∗ T vn = 4 e1 +4cne2 if n is odd and n n T ∗ T vn = bne1 + cne2 if n is even. From the orthogonality equation we get bn (bn 1)e1 +(cn 1)e2 = δn( 4 e1 +4cne2)ifnis odd and (b − 1)e +(c −1)e = δ (b e + c e )ifnis even. n − 1 n − 2 n n 1 n 2 Thus (bn,cn) satisfy the equations (1) (X 1)2 +(Y 1)2 =1and −1 1 −1 (2) 1 X = 16 16Y if n is odd and the− equations − (3) (X 1)2 +(Y 1)2 =1and (4) 1 −1 =1 1−if n is even. − X −Y Let u =(u1,u2)andw=(w1,w2) be the solutions to the equations (1), (2), and (3), (4), respectively. An operator T canbedefinedinasimilarwaysothatforsomex0 and < x0 , the sequence (T ny ) behaves like (v ) does in the above example. k k n,x0 n Appendix Definition. Let X be a normed linear space. Let f be a continuous linear func- tional and x be a nonzero vector in X. The hyperplane x +Kerf is said to be norming at X if x + h x for all h Ker f. For any δ>0 the hyperplane x+Kerfis said tok be (1+k≥kδ)-normingk at x∈ if x+h 1 x for all h Ker f. k k≥ 1+δ ·k k ∈ Remark. For any normed linear space X, each nonzero vector x has a norming hyperplane. Proof. Let x be any nonzero vector. Let r = x .LetUbe the open ball of radius r centered at the origin. Let C be a closedk convexk set disjoint from U such that x = C U. By the Hahn-Banach theorem, there is a continuous linear functional {f such} that∩ Re f(u) < x for all u U,and Re f(y) x for all y C. Clearly, f(x|) = x |andk Rek f = 1. It∈ can easily| be verified|≥k thatk x +Kerf∈is a normingk hyperplane| k k at x. k k Theorem 11. Let (X, ) be a strictly convex smooth reflexive Banach space. Let and be two equivalentk·k norms on X such that and . k·k1 k·k2 k·k1 ≤k·k k·k2 ≤k·k Clearly, for each x =0and for each 0 << x , the closed ball B(x ,) is 0 6 k 0k 0 weakly compact in (X, ) for i =1,2.So, and attain their minimumk·k k·ki k·k1 k·k2 over B(x0,) at some points of B(x0,) . Suppose, for each x0 =0and for k·k k·k 6 each 0 << x , and attain their minimum over B(x ,) at the k 0k k·k1 k·k2 0 same point. Then, =K for some constant K. k·k k·k1 ·k·k2 Proof. Let S = x X :forsomef X∗,the hyperplane x+Kerf is norming at x w.r.t. both {, and∈ . We will∈ show that S is dense. For any nonzero vector k·k1 k·k2} x and any 0 << x let v be the point in B(x ,)atwhichboth 0 k 0k x0, 0 k·k1 and attain their minimum over B(x ,). The set of all such v is dense. k·k2 0 x0, We will show that vx0, S for each x0 and with 0 << x0 .Letf X∗ be such that Re f = ∈v , Re f(x) < v for all x kBk(v , v ∈ ) k k k x0,k1 | | k x0,k1 ∈ x0, k x0,k1 and Re f(x) v for all x B(x ,). Then, v +Kerf is a norming | |≥kx0,k1 ∈ 0 x0, hyperplane at vx0, w.r.t 1. If this hyperplane is not norming at vx0, with respect to ,then vk·k+ h < v for some h Kerf. So, for some k·k2 kx0, k2 k x0,k2 ∈ License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 558 SHAMIM ANSARI AND PER ENFLO t>1, t(v + h) = v . The line segment joining t(v + h)andv k x0, k2 k x0,k2 x0, x0, is contained in the closed ball of radius vx0, 2 centered at the origin. Let us observe that for some λ>0 small enough,k the vectork λ (tv + h)+(1 λ) v · x0 − · x0 ∈ B(x0,) . (Suppose not. Then the closed line segment joining vx0, and t(vx0,+h) k·k is disjoint from the -open set B(x0,) .So,thereisa -closed hyperplane k·k k·k k·k containing this line segment and disjoint from B(x0,). Clearly, this hyperplane is different from vx0, +Kerf. But there is a unique supporting hyperplane at vx0,. This contradiction implies the assertion.) That is, B(0, vx0, ) intersects with k k k·k2 B(x0,) . This contradicts the hypothesis that 2 attains its minimum over k·k k·k B(x0,)atvx0,. Next, let x be any nonzero vector in X, and let δ>0 be arbitrary. Let >0 be such that δ > . Itiseasytocheckthatifsomey B(x, )admitsa 1+δ 1 ∈ norming hyperplane, then− that norming hyperplane is (1 + δ)-norming at x.Thus, each nonzero vector in X admits a (1 + δ)-norming hyperplane for each δ>0. Note that if x + H is (1 + δ)-norming at x, then for any closed subspace N with H N = 0 , x +(H N)isalso(1+δ)-norming at x in the span of x, H N . ∩If the6 conclusion{ } of∩ the theorem is not true, then there exist x and y{ in X∩such} that x 1 = K x 2 and y 1 = K(1 + ) y 2 Consider the two-dimensional spacekNkspanned·k byk x, y .k k ·k k { } If the conclusion of the theorem is not true, then there exist ri > 0fori=1,2 such that B(0,r1) N B(0,r2) N and B(0,r2) N B(0,r1) N. k·k1 ∩ 6⊂ k·k2 ∩ k·k2 ∩ 6⊂ k·k1 ∩ It can be seen that there exists some x ∂(B(0,r1) N) ∂(B(0,r2) N)which does not admit a hyperplane in the two-dimensional∈ space,∩ ∩ which is (1+∩δ)-norming w.r.t. both norms and . k·k1 k·k2 References 1. B. Beauzamy, Introduction to Operator Theory and Invariant Subspaces, North-Holland (1988). MR 90d:47001 2. M. I. Kadets, On spaces isomorphic to locally uniformly rotund spaces,Izv.Vysˇs. Uˇcebn. Zaved. Mathematika 1959, no. 6, 51-57. (Russian) MR 23:A3987 3. S. Mazur, On the generalized limit of bounded sequences, Colloquium Mathematicum, 2, (1949-1951), 173-175. MR 14:159k 4. H. Radjavi and P. Rosenthal, Invariant Subspaces, New York: Springer-Verlag, 1973. MR 51:3924 Department of Mathematics, Kent State University, Kent, Ohio 44242 Current address, Shamim Ansari: Department of Mathematics & Statistics, Drawer MA, Mis- sissippi State University, Mississippi State, Mississippi 39762 License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use