TEN PROBLEMS IN HILBERT SPACE1
BY P. R. HALMOS Dedicated to my teacher and friend Joseph Leo Doob with admiration and affection. TABLE OF CONTENTS PREFACE 887 1. CONVERGENT. Does the set of cyclic operators have a non-empty interior?... 889 2. WEIGHTED. Is every part of a weighted shift similar to a weighted shift?.... 892 3. INVARIANT. If an intransitive operator has an inverse, is its inverse also in transitive? 898 4. TRIANGULAR. Is every normal operator the sum of a diagonal operator and a compact one? 901 5. DILATED. Is every subnormal Toeplitz operator either analytic or normal?... 906 6. SIMILAR. Is every polynomially bounded operator similar to a contraction?.. 910 7. NILPOTENT. IS every quasinilpotent operator the norm limit of nilpotent ones? 915 8. REDUCIBLE. Is every operator the norm limit of reducible ones? 919 9. REFLEXIVE. Is every complete Boolean algebra reflexive? 922 10. TRANSITIVE. IS every non-trivial strongly closed transitive atomic lattice either medial or self-conjugate? 926 BIBLIOGRAPHY 931
PREFACE Machines can write theorems and proofs, and read them. The purpose of mathematical exposition for people is to communicate ideas, not theorems and proofs. Experience shows that almost always the best way to communicate a mathematical idea is to talk about concrete examples and unsolved problems. In what follows I try to communicate some of the basic ideas of Hilbert space theory by dis cussing a few of its problems. Nobody, except topologists, is interested in problems about Hil bert space; the people who work in Hilbert space are interested in problems about operators. The problems below are about operators. I do not know the solution of any of them. I guess, however, that A MS 1969 subject classifications. Primary 4702. Key words and phrases. Hilbert space, operators, cyclic vectors, weighted shifts, invariant subspaces, compact operators, dilations, contractions, quasinilpotence, reducibility, reflexive lattices, transitive lattices. 1 Lectures delivered at a Regional Conference in the Mathematical Sciences, sponsored by the National Science Foundation and arranged by the Conference Board of the Mathematical Sciences, at Texas Christian University, Fort Worth, Texas, May 25-29,1970. License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 887 888 P. R. HALMOS [September
they vary a lot in depth. Some have been around for many years and are known to be both difficult and important; others are untested and may turn out to be trivial. The problems are formulated as yes-or-no questions. That, I believe, is the only clear way to formulate any problem. What a mathematician usually wants to do is something vague, like study, determine, or characterize a class of objects, but until he can ask a clear-cut test question, the chances are he does not understand the problem, let alone the solution. The purpose of formulating the yes- or-no question, however, is not only to elicit the answer; its main purpose is to point to an interesting area of ignorance. A discussion of unsolved problems is more ephemeral than an expo sition of known facts. Deciding to make a virtue out of necessity, I chose to focus on a narrow segment of the present, rather than on a broad view of history. As often as possible the references are to recent publications, and the proofs presented in detail are more from cur rent folklore than from standard texts. Consequently some results (with appropriate credit lines to their discoverers) are published here for the first time. Most of the results and proofs with no assigned credit are "well known"; only a small number are mine. If a result is attributed to someone but is not accompanied by a reference to the bibliography, that means I learned it through personal communica tion, and, as far as I could determine, it is not available in print. The chief prerequisite for understanding the exposition is a knowledge of the standard theory of operators on Hubert spaces. In some cases that means that not even the spectral theorem is needed, and in other cases every available ounce of multiplicity theory would help to understand things clearly. In any case, the presentation is not cumulative; some readers who find §1 obscure may find §10 obvious. As an expository experiment I have put some exercises into each section. This is unusual in papers (as opposed to books). The reason I did it is that originally each section was a lecture, and lectures (as opposed to papers) can stand every bit of elasticity that anyone can think of to put in. The audience is trapped and cannot impatiently riffle the pages forward and back, and the lecturer can use something that is easily omitted or inserted without disturbing the continuity. As for terminology: Hubert space means a complete, complex, inner-product space; subspace means a closed linear manifold; operator means a bounded linear transformation. The Hubert spaces of principal interest are the ones that are neither too large nor too small, i.e., they are separable but infinite-dimensional. The problems are stated without any such explicit size restrictions, but in some
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 1970] TEN PROBLEMS IN HILBERT SPACE 889
cases they become trivial for either the large spaces, or the small ones, or both. I thank Peter Fillmore, Pratibha Gajendragadkar, and Eric Nordgren for reading the first written version of these lectures and for making many helpful suggestions.
1. CONVERGENT Problem L Does the set of cyclic operators have a non-empty interior? A vector ƒ in a Hilbert space H is cyclic for an operator A on H in case the smallest subspace of H that contains ƒ and is invariant under A is H itself. An operator is cyclic in case it has a cyclic vector. In the finite-dimensional case, A is cyclic if and only if its minimal polynomial is equal to its characteristic polynomial. (Equivalently: each eigenvalue occurs in only one Jordan block, or the space of eigenvectors corresponding to each eigenvalue has dimension 1 [24, p. 69 et seq]; cf. also [33].) It follows easily that in the finite-dimen sional case the set 6 of cyclic operators is open. If the dimension is n, then the operators with n distinct eigenvalues constitute a dense set; it follows that 6 is dense. (Consequence: C is not closed; for instance, diag(l, l + lA*>-*diag(l, 1).) There is only one reasonable topology for the operators on a finite-dimensional space; in the infinite-dimensional case there are several. Problem 1 refers to the norm (or uniform) topology, the one according to which An-*A in case \\An—A\\—»0. If ƒ is a cyclic vector for A, then the countable set {ƒ, Af> A2f, • • •} spans H ; it follows that in the non-separable case C is empty. In the separable infinite-dimensional case what little is known is negative. The set e is surely not open; the cyclic operator diag(l, i, i, • • • ) is the limit of the non-cyclic operators diag(l, J, • • • , l/n> 0, 0, 0, • • • ). It is, of course, possible that the interior of G is non-empty; that is what Problem 1 is about. A reason able candidate for an operator in the interior of 6 is the unilateral shift U. (According to the most easily accessible definition U is the 2 operator on I that sends (£0, £i, £2, • • • ) onto (0, £0, £1, £2, • • • ).) That candidate happens to fail (J. G. Stampfli). The set 6 is also not dense; in face if V is the direct sum of two copies of U and || F—^4|| <1, then A is not cyclic. Indeed:
||l - A*V\\ = ||7*7 - A*V\\ â ||F* - A*\\ < 1,
so that A* V is invertible; it follows that V*A is inverti ble and hence that ran A C\ ker F*=0. Since, thus, ran A is disjoint from the 2- dimensional space ker V*t the co-dimension of ran A must be at least
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 890 P. R. HALMOS [September
2. (The proof so far was needed to get this inequality only. An alter native way to get it is to note that it is a special case of a small part of the theory of Fredholm operators [35].) Now it is clear that A cannot be cyclic ; for each ƒ, the span of {ƒ, Af, A 2f, • • •} has co-dimension at least 1. The point of the preceding discussion is to call attention to the existence of topological problems of analytic importance in operator theory. There are many; Problem 1 is just a sample. Other examples are Problems 7 and 8, about the closures of certain sets of operators, or, in other words, about approximation theory. Here is a well known example of the same kind: is the set of all invertible operators dense? The answer is yes for finite-dimensional spaces, and furnishes an occasionally useful proof technique; to prove something for all matrices, prove it for the non-singular ones first and then approxi mate. For infinite-dimensional spaces the answer is no [20, Probl em 109]. It is slightly less well known that the answer remains no even for a somewhat larger set, the set of kernel-free operators. (If A is invertible, then A is kernel-free, i.e., ker 4=0, but the converse is not true.) In other words, there exists a non-empty open set every element of which has a non-trivial kernel. Indeed, the open ball {A :|| [/*— A\\ <1} is such a set (where U is, as before, the unilateral shift). For the proof, apply the (Fredholm) argument that disproved the density of the cyclic operators to infer that, for each A in that ball, ran A* has positive co-dimension, and then conclude that ker A ?*0. Here is a classically important question: is the set of invertible operators connected? The answer is yes [20, Problem HO]; the proof is an application of the polar decomposition of operators. A much less important question that takes most people a few seconds longer to answer than it should is this (Exercise 1) : is the set of non-scalar normal operators connected? All the questions so far concerned the norm topology; questions about the other operator topologies are likely to be (a) easier and (b) more pathological. The pathology of topology can be fun, however, and, although the ground has been pretty well worked over, new and even useful facts do turn up from time to time. The strong operator topology is the one according to which An—>A in case ||'-4„f—-4/||—»0 for each vector/. (The indices, here and else where, need not and should not be restricted to integers; any directed set will do.) A famous and sometimes bothersome misbehavior of the strong topology is the discontinuity of the adjoint. Indeed, if Ak = U* (the unilateral shift again), & = 1, 2, 3, • • -, then Ak—>0 strongly, but {At} is not strongly convergent to anything [20, Problem 90 ]. This
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use i97o] TEN PROBLEMS IN HILBERT SPACE 891
negative result is old; the following interesting positive observation of Kadison's is quite recent [26]: the restriction of the adjoint to the set of normal operators is strongly continuous. (That is: if An—*A strongly, where A and each An is normal, then A*—>A* strongly.) The proof is a trick, but a simple trick:
2 2 \\(At-A*)f\\ = \\Anf\\ -\\Af\\%(f,(A-An)A*f) + ((A-An)A*f,f)
^\\(A-An)f\\(\\Af\\+Unf\\)+2\\(A-An)A*fl\\f\\- Approximation techniques that rely on the strong topology do not seem to have been used much, although there are some interesting questions whose known and surprising answers might turn out to be useful. Example: what is the strong closure of the set of normal operators? Answer: the set of subnormal operators [4]. The proof is not trivial. Another example (Exercise 2) : what is the strong closure of the set of unitary operators? The weak operator topology is the one according to which An-*A in case (Anf, g)—>(Af, g) for each ƒ and g. A useful recent result is that rank (dimension of the closure of the range) is weakly lower semicon- tinuous; that is, if An-*A weakly, then lim inf « rank A „^ rank A [2l]. The proof is not immediately obvious, but it is certainly not deep. Warning: the possible values of rank in this context are the nonnegative integers, together with 00 ; no distinction must be made among different infinite cardinals. Were such a distinction to be made, the result would become false. Suppose indeed that the under lying Hubert space has an uncountable orthonormal basis {e/.jG/}. Let D be the set of all countable subsets of /, ordered by inclusion; for each n in P, write An for the projection onto the span of {ejljÇHn}. Since for each vector ƒ there exists an n0 in D such that f±.ej when ever,; is not in n§, it follows that An—*\ (not only weakly, but, in fact, strongly). Since rank^4w = K0 and rank 1>^0» the cardinal version of semicontinuity is false. What is the weak closure of the set of normal operators? Answer: in the finite-dimensional case it is the set of normal operators [21 ]; in the infinite-dimensional case it is the set of all operators [18]. What is the weak closure of the set of projections? Answer: in the finite- dimensional case it is the set of projections; in the infinite-dimen sional case it is the set of Hermitian operators A with 0 SA ^ 1 [18]. The proofs of the infinite-dimensional statements are not difficult, but they depend on a little dilation theory. The results show why weak approximation theory is not likely to be fruitful; it is too good to be any good. Exercise 3: what is the weak closure of the set of scalars?
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 892 P. R. HALMOS [September
A simple example that illustrates many of the curious properties of the weak operator topology can be described in terms of matrices. Let An be the infinite matrix (or, by a slight abuse of language, the operator) whose entries at the positions (1, 1), (1, n), {n, 1), and {n, n) are J, and whose remaining entries are all 0 (n = 2, 3, 4, • • • ). It is easy to verify that An is idempotent, and it is plausible (and true, and easy to verify) that if A is the matrix whose (1,1) entry is \ and whose remaining entries are all 0, then An-*A weakly. The example shows how projections can converge to a non-projection, it proves that multiplication (in fact squaring) is not weakly continuous, and, incidentally, it implies that the strong and the weak topologies are distinct. (Here is an unimportant but unsolved teaser: does squaring have any points of weak continuity?) ANSWERS TO THE EXERCISES. (1) The non-scalar Hermitian oper ators form a (norm) connected set. For a 1-dimensional space this is trivial. In any other case, the set of Hermitian operators is a real vector space of dimension 3 or more. If, therefore, A and B are non- scalar Hermitian operators, then there always exists a non-scalar Hermitian C such that tA +(1 — t)C and tB + (l —t) C axe non-scalar Hermitian operators whenever Q^t^l. The normal case can be joined to the Hermitian one. Indeed, if A +iB (with A and B Her mitian) is a non-scalar normal operator, then assume, with no loss of generality, that A is not a scalar, and consider A+itB, O^/^l. (This solution is due to J. J. Schâffer.) (2) The strong closure of the set of unitary operators is the set of isometries. Since the set of isometries is easily seen to be strongly closed, and since every isometry is a direct sum of a unitary operator and a number of copies of the unilateral shift [20, Problem 118], it is sufficient to prove that U, the unilateral shift, is a strong limit of unitary operators. That is easy: If
Un(%Q, £l, &,••') = (£n, £o, ?1, ' * ' • £n-l» £n+l» £n+2, * * * )
for n = 1, 2, 3, . . . , then each Un is unitary and Z7n—>!7 strongly. (3) The set of scalars is weakly closed. Indeed, if Xn(jf, g)—>(Sf, g) for each ƒ and g, then {Xn} is a Cauchy net of complex numbers, and therefore Xn—»X for some complex number X; it follows that Xn(f, g) —>X(f, g) and hence that Sf=Xf for each/.
2. WEIGHTED Problem 2. Is every part of a weighted shift similar to a weighted shift? If U is the unilateral shift and if en is the sequence (£o, £i, £2, • • • )
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use I970] TEN PROBLEMS IN HILBERT SPACE 893
with £n = l and £t=0 when^Vw, then {e0, ei, e2l • • • } is an orthonor 2 mal basis for I and Uen — en+i> n = 0, 1, 2, • • • . If («0, «i, «2, • • • ) is a bounded sequence of positive numbers, then the equations Aen =anen+i, tt = 0, 1, 2, • • • , unambiguously define an operator A. Such operators are called weighted shifts ; they are of interest because they can be used to construct examples of many different kinds of opera- torial behavior. (To eliminate easy but dull case distinctions, the number 0 is not allowed as a weight.) For any operator A on any Hubert space H, a part of A is, by definition, the restriction of A to a subspace of H invariant under A. One of the main reasons for the success of the Beurling treatment of the unilateral shift is that every non-trivial part of U is unitarily equivalent to U [20, Problem 123]. Here "non-trivial" excludes the most trivial case only, the restriction to the subspace 0. That should, of course, be excluded in the formulation of Problem 2 also, and in all related contexts; that exclusion is hereby made, retroactively and forward to eternity. What sense does it make to say that a certain operator "is" a weighted shift? If shifts are defined on the concrete Hubert space I2 (as above), then the best that can be hoped about a part of one is that it is "abstractly identical" with or "isomorphic" to another; the cor rect technical phrase is "unitarily equivalent". If shifts are defined with respect to arbitrary orthonormal bases {e0, ei, eif • • • } in arbitrary separable infinite-dimensional Hubert spaces (the best way), then "is" means "is". There is a third interpretation, which may seem artificial but has some merit; it is obtained by demanding iso morphism with respect to the linear and the topological structures, but ignoring the metric structure. The correct technical word is "similar"; two operators A and B, on Hubert spaces H and if, are similar in case there is a bounded linear transformation 5 from H onto K, with a bounded inverse, such that A = S^BS. There are two reasons for the insistence on positive weights in the definition of weighted shifts: (a) it seems more natural, and (b) nothing is gained by allowing arbitrary complex weights. The proof of (b) is the following assertion (Exercise 1) : if A and B are weighted shifts, with complex weight sequences (an) and (/3W), then a necessary and sufficient condition that A and B be unitarily equivalent is that \otn\ = I j8»| for n — 0, 1, 2, • • • . Consequence: if A is a weighted shift and if X is a complex number of modulus 1, then A is unitarily equi valent to \A. This kind of "circular symmetry" is an unusual but use ful property for an operator to have. More along the same lines can not be asked; modulus 1 is in the nature of things. That is (Exercise 2) : if a non-nilpotent operator A is similar to X.4, then |X| =1.
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 894 P. R. HALMOS [September
The unitary equivalence theory of weighted shifts is easy; their similarity theory is slightly more complicated, but still near the sur face. In the notation of the preceding paragraph, A is similar to B if and only if the sequence of quotients
\ ao ' ' • an \ I 0o * * • P. I is bounded away from 0 and from oo [20, Problem 76]. It is thus easy to tell when two weighted shifts are similar, but the circle of ideas centered at Problem 2 includes a question of a much more difficult type: when is an operator obtained by certain non-transparent con structions similar to a weighted shift? The formation of a part of a weighted shift is such a construction; there are others. Consider, for 2 instance, the (Cesaro) operator C0 on I defined by
C0(£o, £i, £2, • • • } = (170, Vh m, • • ' ), where 1 n Vn = —— Z) &• » + 1 <-o
The known spectral properties of C0 [7] suggest the question: is 1 — C0 similar to a weighted shift? The answer is not known. Another inter esting operator (Volterra) is defined on L2(0, 1) by Vf(x) =Jlf(y)dy\ i.e., Vf is, for each/, the indefinite integral of/. (Exercise 3) : is V sim ilar to a weighted shift? What are weighted shifts good for ?The answer is that they can be used for examples and counterexamples to illustrate many properties of operators. Among such properties are the existence of square roots, spectral behavior, irreducibility, the structure of invariant subspaces, and subnormality. Assertion: no weighted shift has a square root. Suppose, indeed, that A is a weighted shift, and suppose that B2 = A*. (The adjoint is easier to treat, and the result obviously comes to the same thing.) Since kerB C ker A *, and smeekere * 5^ 0, it follows that ker B = ker A *. Since ker ^4*C ran -4*, it follows that ker A*(Z ran B. Hence if e is in ker A*, e^O, then there exists a vector ƒ such that Bf = e> and it is easy to deduce (from ker B= ker ^4*) that e and ƒ are linearly independent. Conclusion: the dimension of ker B2 ( = ker^4*) must be at least 2, which is absurd [29], [20, Problem 114]. A weighted shift provides the easiest example of a kernel-free operator whose spectrum consists of 0 only [20, Problem 80 ], and the
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use i97o] TEN PROBLEMS IN HILBERT SPACE 895
more delicate spectral behavior of weighted shifts has also received quite a bit of attention [ló], [29], [34], [41 ]. Weighted shifts are irreducible. That is, if A is a weighted shift and if M is a subspace that reduces A, then M is either 0 or the entire space; equivalently, if P is a projection that commutes with A, then P =0 or P = 1. To prove this, all that is needed is that the kernel of A* is spanned by a single (non-zero) vector e which happens to be cyclic for A. Indeed: A*Pe = PA*e = 0 implies (since e spans ker A*) that Pe = Xe, and hence (since P is a projection) that either Pe = 0 or Pe = e. If Pe = 0, then ^4n^ =^4n(l —P)e = (1 —P)Ane, which belongs to ker P for all n, and therefore (since e is cyclic for A) P = 0; if Pe = e, then ^4ne==^4nP£=iMn£, which belongs to ran P for all n, and there fore P = 1. (This proof is due to K. J. Harrison.) A somewhat stronger result is known: every operator similar to a weighted shift is irre ducible [29], [51]. To what extent can the structure of the invariant subspaces of an operator be prescribed? Is it, for instance, possible that an operator has exactly one invariant subspace of each dimension between 0 and fc$o inclusive? The answer is yes, and the standard example (due to Donoghue) is given by the adjoint of a suitably weighted shift [20, Problem 151 ]. (The unilateral shift itself, unweighted, won't do.) There are many generalizations of the concept of normality; two important ones, in decreasing order of generality and increasing order of importance, are hyponormality and subnormality. An operator A is hyponormal in case A*A—AA*^0; an operator is subnormal in case it is part of a normal operator. (The verificaton that every subnormal operator is hyponormal is the easiest part of this otherwise tricky subject.) Which weighted shifts are normal, which are hyponormal, and which are subnormal? The first question is settled just by looking at kernels; there is no weight sequence that can make a weighted shift normal. The answer to the second question is also easy [20, Problem 160]: the weight sequence must be mono tone increasing. The answer to the third question is not easy. One formulation was offered by Stampfli [49]; the hitherto unpublished formulation below (due to C. Berger) is very different. Berger's condi tion is elegant and easy to state: a weighted shift is subnormal if and only if the squares of the partial products of the weights constitute the moment sequence of a probability measure in the unit interval. For the proof, suppose first that A is a subnormal weighted shift, and assume, with no loss of generality, that |J^4]| =1. Let B (on a Hilbert space K) be the minimal normal extension of A ; then ||5|| = 1. Since e0 is "bicyclic" for B (i.e., by minimality, there is no proper sub-
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 896 P. R. HALMOS [September
space of K that contains eo and reduces B), the spectral theorem im plies that B is unitarily equivalent to the position operator (f(z) —*zf(z)) on L2 of the closed unit disc D with respect to some proba bility measure p» The standard unitary equivalence, moreover, carries the vector e0 onto the constant function 1. Identify B with its unitarily equivalent transform on L2QJL), and, correspondingly, identify A with the restriction of B to the span of (the images of) the e vectors e0 ei, * ' ' • • Let v be the "marginal" measure induced by p. in the closed unit interval I. That is: if S:D—>I is the mapping defined by Sz = \z\, then v{E) =fx(S~1E) for each Borel subset E of 2". It follows [17, p. 163] that Jifdv—Ji) fS-dp whenever either side of the equation makes sense. If, in particular, ƒ(p) =p2n, n = 0, 1, 2, • • • , then
f p2ndv(p) = f \z\2ndp(z) = f \zn-l\2dp(z)
2 2 = ll^oll = |M"e0|| .
The last term is easy to compute: if po = l and p„+i —OLnpn) then n A eo = pneni and therefore 2n 2 p dv{p) = pn, n = 0, 1, 2, ƒ,J 2 The proof of the necessity of Berger's condition is complete; p n is the nth. moment of dv(\/p). To prove sufficiency, start with a probability measure v in J, and 2n write Jip dv(p) —pn> If X is normalized Lebesgue measure in the perimeter C of the unit circle, then *>XA in IXC induces (it is tempt ing to say "is") a measure p, in D. That is: if T: IXC—>D is the mapping defined by (p, u)-*pu, then p(E) =(^XX)(r_1E) for each 2 Borel subset E of D. Let B be the position operator on L {p)y and let 2 A be the restriction of B to the subspace i? (ju) spanned by {/0, /i, w /2, . . .}, where fn(z) =z , n = 0, 1, 2, ... . Since B is normal, it follows that A is subnormal. A straightforward computation shows that
(fn,fm) = f pn+mdv(p) • f un~md\(u), n, m = 0, 1, 2, • • . .
If »^m, the last factor vanishes, i.e., the/'s are pairwise orthogonal» If n = mf the last factor is 1, and therefore ii/»ir = #», » = 0,1,2,.• •.
If, therefore, en=fn/pn, n==0, 1, 2, • • • , then the e's form an ortho-
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use i97o] TEN PROBLEMS IN HILBERT SPACE 897
normal basis for H2(p). Since, finally, / 1 \ 1 pn+l Aen = A [ —ƒ„ ) = —/n+i = en+i, n = 0, 1, 2, • • • , \pn / Pn pn it follows that A is (unitarily equivalent to) the weighted shift whose weight sequence has exactly the p's for its partial products. The proof of Berger's theorem is complete. The preceding report on weighted shifts is a representative but not exhaustive summary of what is known. There is much that remains to be done, and, in particular, there are at least three directions of generalization that promise to be fruitful. (a) Two-way shifts (en—>anen+i for all integers ny not the non- negative ones only) have many properties in common with the one way kind, but not all. (Sample question with a known but not com pletely trivial answer: is the Vol terra operator similar to a two-way weighted shift?) (b) If the shift n—>n + l on the integers is replaced by a measure- preserving transformation on a more interesting measure space, the theory of weighted shifts becomes the theory of "weighted transla tions" ; although a few things about them are known [36], their theory is still largely undeveloped. (c) The space I2 is the direct sum of copies of a 1-dimensional Hu bert space; a generalization in the direction of higher multiplicities is obtained via the formation of direct sums of more general Hubert spaces. In that case the role of the numerical weights is played by operator weights. This is essentially virgin territory; the idea has been used to construct counterexamples [20, Problem 164], but a general theory does not yet exist. ANSWERS TO THE EXERCISES. (1) Given the a's and the jS's, put 5o = l and determine Sw recursively from anôw=j3w§n+i, n = 0f 1, 2, . . .; if always |an|=|j3«j, then always |S»| =1, and the "diagonal operator" determined by the S's transforms A onto B. This proves sufficiency. To prove necessity, suppose that A = W*BW, where W is unitary, so that An = W*BnW, and also A* = W*B*W. It follows that W* carries e0 (in the kernel of B*) onto a unit vector in the kernel of A*; assume with no loss of generality that W*eo — eo. It n n follows that W*B e0 = A eo, and hence (form norms and use induc tion) that |j3w| =\an\ forw = 0, 1, 2, . . . [20, Problem 75; 28]. (2) Since
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 898 P. R. HALMOS [September
it follows that 115-^*511 ^/M"!!17"-*!. « S-lAS=\A, then this implies that |X| =1 [28]. Note that if A is nilpotent, it is quite pos sible to have A be similar to \A for every non-zero complex number X; an example is A = (£ J). (3) The Volterra operator is not similar to a weighted shift. Rea son : if F is similar to A, then F* is similar to A * ; if A is a weighted shift, then ker -4*5^0, but an easy calculation shows that ker V* = 0. (This proof is due to J. G. Stampfli.)
3. INVARIANT Problem 3. If an intransitive operator has an inverse, is its inverse also intransitive? An operator is called intransitive if it leaves invariant some sub- space other than 0 or the whole space; in the contrary case it is transitive. The hope that non-trivial invariant subspaces always exist (i.e., that, except on a 1-dimensional space, every operator is in transitive) is perhaps still alive in the hearts of some. That existence theorem would be the first step toward a detailed structure theory for operators (possibly a generalization of the theory of the Jordan form in the finite-dimensional case). Repeated failure has convinced most of those who tried that the truth lies in the other direction, but the proof of that is, so far, just as elusive. There are many special classes of operators that have been proved intransitive. Problem 3 (due to R. G. Douglas) seems to be the simpl est problem of its kind (derive the intransitivity of something from that of something else). For the strong kind of invariance (that is, for reduction) the problem is easy: if a reducible operator has an in verse, then the inverse is also reducible. Indeed, more is true: if a subspace M reduces an invertible operator A, then the same M reduces A"1. (Proof: if a projection commutes with A, then it com mutes with A~l.) For plain invariance the more is false; the invari ance of a subspace under an invertible operator A does not imply its invariance under A~x. (Consider the bilateral shift.) A related question concerns square roots instead of inverses. Squaring preserves invariance (and reduction); what about the formation of square roots? If A2 is reducible, is A reducible? The answer is no (consider weighted shifts). If A2 is intransitive, is A intransitive? The answer is not known. Equally unknown is the answer to a mixed question (proposed by C. Pearcy): if A2 is re ducible, is A at least intransitive? The only thing along these lines that can be said has an undesirably special hypothesis (Exercise 1) : if A2 is normal, then A is intransitive.
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use i97o] TEN PROBLEMS IN HILBERT SPACE 899
If even the relatively elementary algebraic questions are un answered, there is no immediate hope for the much deeper questions of perturbation (in imitation of the theory of spectral perturbation), but they deserve to be put on record. What is known is that if A and B are Hermitian and B has rank 1, then A+iB is intransitive. The presently hoped-for generalization is obtained by replacing "rank 1" by "compact". Although large classes of compact operators are admissible here [47], not all are known to be. The mildest unproved perturbation statement appears to be this: if A is normal and B has rank 1, then A +B is intransitive. The generalization of this perturba tion problem to arbitrary intransitive operators in place of normal ones is not going to be easy to settle. Indeed: if A is an arbitrary operator, and if P is a projection of rank 1, then A =AP+A (1 —P) ; the first summand has rank 1, and the second has a non-trivial kernel. For normal operators the spectral theorem yields many invariant subspaces. The step from normal to subnormal is, however, large: it is not known whether every subnormal operator is intransitive. Replacement of the algebraic condition of normality by the analytic condition of compactness yields a famous and useful intransitivity theorem: every compact operator (on a space of dimension greater than 1, of course) has a non-trivial invariant subspace [2]. Do two commutative compact operators always have a non-trivial invariant subspace in common? The answer is not known. Two more invariant subspace theorems deserve mention; they are elementary, but they are sufficiently different from the preceding ones and from each other that they might suggest a new idea to some one. (i) The strictly algebraic version of the invariant subspace problem has a positive solution (Exercise 2) : every linear transformation (on a vector space of dimension greater than 1) has a non-trivial invariant linear manifold [46]. (ii) For every operator A there exists a hyperplane M (subspace of co-dimension 1) such that the compression of A to M has an eigen vector. (The compression of A to M is the operator B on M defined as follows: if P is the projection with range M, then Bf = PAf for each ƒ in M.) In fact more is true: for each non-zero vector/, there exists a hyperplane M containing/ such that the compression of A to M has ƒ as an eigenvector. Proof: if ƒ is an eigenvector for A, the assertion is trivial. If ƒ is not an eigenvector for A, then ƒ and Af span a 2-dimen- sional space; let g be a non-zero vector in that space orthogonal to/, and let M be the orthogonal complement of g. Clearly ƒ belongs to the hyperplane M. Since Af is a linear combination of/and g, the projec tion of Af into Af is a scalar multiple of/. This proof is due to L. J.
License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 900 P. R. HALMOS [September
Wallen. The result was suggested by the following theorem of C. Apostol's: if A is an operator such that 0 is in the spectrum of A but ker A = ker A* =0, then there exists an infinite-dimensional subspace M such that the compression of A to M is compact. All the results reported on so far concern intransitivity ; they assert that under certain conditions certain subspaces are invariant. The last two results to be mentioned in this context go in the direction of transitivity: certain subspaces are not invariant. (iii) Exercise 3: for each countable set of non-trivial subspaces, there exists an operator that leaves none of them invariant. (iv) There exists a linear transformation on a Hubert space H that leaves no non-trivial subspace of H invariant. Warning: this does not pretend to be a solution of the invariant subspace problem. "Sub- space" here means closed linear manifold, as always, but "linear transformation" does not mean operator, i.e., the linear transforma tion whose existence is asserted is not necessarily bounded. (There is, however, no fudging about domains: the assertion is about linear transformations that act on the entire space H.) The result is due to A. L. Shields [48] ; the proof goes as follows. Let $ be the smallest ordinal number with cardinal number c (continuum). If H is a separable infinite-dimensional Hubert space, then the set of all infinite-dimensional proper subspaces has cardinal number c, and is therefore in one-to-one correspondence a—>Ma with the predecessors of ^. Let ƒ o be a non-zero vector in itf0, and let g0 be a vector not in M"0. Suppose that fp and gp have been defined whenever ]8 License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use i97o] TEN PROBLEMS IN HILBERT SPACE 901 finite orbits. In all cases, A is a transformation on the Hamel basis E^JF, and, therefore, A has a unique extension to a linear trans formation on H. The linear transformation A can have no invariant infinite-dimen sional proper subspace (because Af« is never in Ma), and it can have no eigenvector (because it maps each finite linear combination of the vectors in the basis E\JF onto a linear combination that involves at least one new vector). 2 ANSWERS TO THE EXERCISES. (1) Since A commutes with A it follows that A commutes with all the spectral projections of A2. The only way this comment can fail to yield an invariant (in fact reducing) subspace for A is if A2 is a scalar. That case has to be examined separ ately; the examination is straightforward and the desired result follows easily. (2) Assume that the linear transformation A is such that, for some vector/, the set {ƒ, Af, A2f, • • • } is a linear basis; in all other cases the conclusion is obvious. Since every vector is a (finitely non-zero) n w linear combination of the lormYln-o License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 902 P. R. HALMOS [September been found useful in other parts of operator theory; it is reasonable to hope that this circle of ideas will have further applications. On a separable space (the only kind to be considered in this section from now on) if an operator A is diagonal, then there exists an in creasing sequence {Ew} of projections of finite rank such that En-*1 strongly and AEn=EnA for each w = l, 2, 3, •••. (The converse is not true. What the condition implies is that A is "block-diagonal". That is: the space is the direct sum of finite-dimensional reducing subspaces, which, however, need not be 1-dimensional.) A sur prisingly rich class of operators is obtained if commutativity is re placed by asymptotic commutativity. Call an operator A quasi- diagonal in case there exists an increasing sequence {£„} of projec tions of finite rank such that Ew~»l strongly and ||.4En —En^4||—»0. There are three kinds of basic results about quasidiagonal oper ators: characterization, closure, and inclusion. Characterization theorems are equivalences ; they replace the definition by something else that is sometimes easier to apply. Closure theorems assert that certain operations on quasidiagonal operators lead to other operators of the same kind. Inclusion theorems assert that certain other, more familiar, classes of operators are included among the quasidiagonal ones. The most elegant characterization theorem replaces the unnatural existence requirement in the definition of quasidiagonality by a simple equation. To describe that equation, note first that the set of all projections of finite rank is partially ordered by range inclusion, and, endowed with that order, it becomes a directed set. If 0 is a real net (i.e., a real-valued function) on that directed set, then the con vergence of 0 to a limit a, denoted by 0(E) —» a or lim 0(E) = a, means that for every positive number e there exists a projection E0 (of finite rank, of course) such that \ License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 197°] TEN PROBLEMS IN HILBERT SPACE 903 is (norm) closed. There are also algebraic closure theorems; e.g., every polynomial in a quasidiagonal operator, the adjoint of a quasidiagonal operator, and countable direct sums of quasidiagonal operators are again quasidiagonal. The most trivial inclusion theorem is that diagonal operators are quasidiagonal ; together with norm closure (and the spectral theorem) this yields the valuable inclusion theorem that normal operators are quasidiagonal. Another trivial inclusion theorem is that operators of finite rank are quasidiagonal; together with the characterization of compact operators as the limits of operators of finite rank [20, Problem 137] this yields the result that compact operators are quasidiagonal. The latter result can be strengthened; in fact, if A is compact, then lim||ilE- EA\\ = 0. (That is: lim inf can be replaced by lim.) This, in turn, implies that a compact operator will never interfere with the lim inf that establishes the quasidiagonality of some other operator; precisely, if A is quasi- diagonal and C is compact, then A +C is quasidiagonal. Noteworthy special case: the sum of a normal operator and a compact one is always quasidiagonal. Is the converse true? That is (Exercise 2): is every quasidiagonal operator the sum of a normal operator and a compact one? One of the facts that make quasidiagonal operators interesting is that every quasidiagonal operator is the sum of a block diagonal operator and a compact one. A slightly stronger statement is true: every quasidiagonal operator has a matrix such that, for a suitable way of dividing it into finite blocks, the result of replacing the blocks on the diagonal by O's is compact. To prove this, suppose that {En} is an increasing sequence of projections of finite rank such that En—>1 strongly and ||-4JS»—JS»^l||—>0. Choose a subsequence, if necessary, to justify the assumption that 53„||i4£n—E„i4|| < oo. Write D for the matrix formed of the diagonal blocks (£»—En„i)A(En—En_i) (where -Eo = 0) and write C — A— D. It is not obvious by inspection that D and C are operators, and it is even less obvious that C is com pact, but that is what the proof is about to show. Indeed: put Cn = En+i(AEn — EnA)En •— En{AEn •— EnA)En+i* Clearly \\Cn\\ S2\\AEn-EnA\\, and therefore ]£nCn converges in the norm to a compact operator; an obvious computation shows that that operator is equal to C. License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 904 P. R. HALMOS [September Weyl's theorem (the affirmative answer to Problem 4 for Hermitian operators) is an immediate consequence. Reason: if A is Hermitian, then so is each diagonal block in any matrix of A ; it follows that the block diagonal summand that the preceding result yields is, in fact, diagonal. The proof shows also what blocks the proof in the normal case; a diagonal block in a normal matrix may fail to be normal. For unitary operators J. A. Dyer (simplifying and generalizing an argu ment of I. D. Berg) proved that Weyl's theorem remains true; the trick is to represent a unitary operator as eiA, with A Hermitian, and apply the Hermitian result to A. Quasidiagonal operators are a special case of a concept that has received some attention in the literature [lO], [12], [22]; the more general operators are called quasitriangular. By definition an operator A is quasitriangular in case there exists an increasing sequence {En} of projections of finite rank such that En—>1 strongly and such that ||./4£w—En.4Ew||--K). (Motivation: replace asymptotic reduction by asymptotic invariance. Note that the appropriate definition of a triangular operator requires the existence of a sequence {En} of pro jections of finite rank such that AEn ~EnAEn.) Most of the characterization, closure, and inclusion theorems stated above for quasidiagonal operators are true for quasitriangular operators also [22]; the proofs for quasidiagonal operators are easy adaptations of the proofs in [22]. A case in point is the characterization theorem: A is quasitriangular if and only if lim imV*i||i4.E— E4E|| =0. A notable exception is the closure the orem : the class of quasidiagonal operators is closed under the forma tion of adjoints. For quasitriangular operators that is not true. In order to support the latter (negative) statement, it is, of course, necessary to have a technique for proving that something is not quasitriangular. The tool in [22 ] is the assertion that if A *A = 1 and ker A* 5*0, then A is not quasitriangular. A slightly sharper version of this is true [12]: if ^4*^4 ^1 and ker ^4*^0, then A is not quasi triangular. Suppose, indeed, that e^O and A*e = 0; let E0 be the pro jection onto the (1-dimensional) span of e. If E is a projection of finite rank with Eo^E, then the restriction of E A* E to ran E has e in its kernel; the finite-dimensionality of ran E implies the existence of a unit vector ƒ in ran E such that EAEf = 0. It follows that \\(AE-EAE)f\\=\\AEf\\=\\Af\\^ \\f\\, whence ||^4JS—E.4E||èl; this proves that A cannot be quasitri angular. Very small consequence: since the adjoint of the unilateral License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use i97o] TEN PROBLEMS IN HILBERT SPACE 905 shift is quasi triangular (in fact, triangular), the adjoint of a quasi tri angular operator can fail to be one. For all that has been said so far it could be true that, for every operator A, either A or A * is quasi triangular. A natural candidate for a counterexample is U® U* (where U, of course, is the unilateral shift), but the candidate fails; U®U* is, in fact, quasidiagonal. (Proof: it differs from the bilateral shift, which is normal, by an operator of rank 1.) Exercise 3: is there an operator A such that neither A nor A * is quasitriangular ? The sum theorem for quasidiagonal operators (block diagonal plus compact) is not the specialization of a theorem in [22], but it might as well be; a slight complication of the proof serves to show that every quasitriangular operator is the sum of a triangular operator and a compact one. Two further facts deserve at least brief mention, (i) Douglas and Pearcy [12] have proved that every operator with a finite spectrum is quasitriangular, and they conjectured that the same is true for operators with a countable spectrum, (ii) What happens if the lim inf in the definition of quasi triangularity is replaced by lim? Answer: a necessary and sufficient condition that lim^^i||-4£—Ei4£|| =0 is that A be the sum of a compact operator and a scalar [ll]. ANSWERS TO THE EXERCISES. (1) For every compact operator C there exists a separable reducing subspace whose orthogonal com plement is included in ker C. This fact is usually stated for compact normal operators only [20, Problem 133]; the general case follows by applying the usual statement to C*C and forming the smallest sub- space that reduces C and includes the separable part. Another ap plication of the same technique shows that if A = D + C, where C is compact, then there exists a separable subspace that reduces both D and C and whose orthogonal complement is included in ker C. If the space is non-separable and D is diagonal, then it follows that A has many eigenvectors. If, therefore, A has no point spectrum, then A admits no sum representation of the required kind. (2) The operators of the form "normal plus compact" do not ex haust all quasidiagonal operators. For an example, consider the direct sum A of infinitely many copies of the operator on C2 with matrix (? S); since A is block diagonal, it is quasidiagonal. Then commutator A* A—A A* is the direct sum of infinitely many copies of (J _?), which is about as far as possible from being compact; if, on the other hand, N is normal and C is compact, then (N+C)*(N-jrC) -(N+C)(N+C)* is compact. (3) If U is the unilateral shift and A =(£/-*) 0 (£/*+*), the License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 906 P. R. HALMOS [September neither A nor A* is quasi triangular. For the proof, consider A+i = U®(U*+2i) and note that it is bounded from below by 1 but its adjoint has a non-zero kernel. This answer to a question in [22] ap pears in [12]. 5. DILATED Problem 5. Is every subnormal Toeplitz operator either analytic or nor mall Toeplitz operators are the best known and analytically most im portant examples of compressions; they are compressions of the simplest "continuous" generalizations of diagonal matrices. If C is the unit circle ({sr: 12;| =l}) in the complex plane, endowed with normalized Lebesgue measure, then the functions en, defined by n 2 en(z)=z , n = 0y ±1, ±2, • • • , form an orthonormal basis for L 2 2 2 ( — L (C)) ; let H be the subspace of L spanned by {e0, e\, e2, • • • }. If (T+ej,€i) = (P(4>-ej)9ei) = (*•«/,*<); since the multiplication operator induced by e\ is the unilateral shift U, and hence isometric, it follows that (T+e^eù = (17(*-«y), Uet) = (0• «y+ii «*+0 = (P( License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use i97o] TEN PROBLEMS IN HILBERT SPACE 907 if and only if U*AU = A.) If a Toeplitz operator T4 is analytic (which, by the way, happens if and only if its matrix is lower triangular), then 0 • ƒ is in H2 when ever ƒ is in H2; the projection P is not needed. Consequence: if T4, is analytic, then it is a part of the multiplication operator induced by (r -A)' License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 908 P. R. HALMOS [September where S and Tare the positive square roots ofL —A A* and I—A*A respectively. The verification that B is unitary is trivial but not obvious [20, Problem 177]. Projections, that is to say perpendicular projections, or projec tions onto a subspace along its orthogonal complement, are the right things to look at in Hubert space, but, from a geometrical point of view, skew projections are just as good. In algebraic language skew projections are idempotent operators; perpendicular projections are the ones among them that happen to have unit norm. It is this normalization that forces every compression of a unitary operator to be a contraction; what happens if skew projections are allowed? Exercise 3: is it true that for every operator A on a Hubert space H there exists a unitary operator B on a larger Hilbert space K and a skew projection P from K onto H such that Af=PBf for each ƒ in H? What makes the theory of unitary dilations useful is Nagy's theorem [20, Problem 178] on power dilations. The assertion is that for every contraction A on H there exists a unitary operator B on a larger K such that Bn is a dilation of An simultaneously for every positive integer n. One of the most spectacular applications of unitary dilation theory is the proof of von Neumann's beautiful and powerful analytic theorem about contractions [20, Problem 180]. The assertion is that if jmi^l and if p is a polynomial such that \p(z)\ ^1 whenever | zI =1, then ||jf)(i4)||^l. For the proof, let B be a unitary power dilation of A to, say, 2£, let P be the (perpendicular) projection from K onto H, and note that, for each ƒ in H, \\P(A)A\-\\PP(U)A\* \\P(U)\\ -ll/NII/ll. The crucial inequality, the second one, is an elementary consequence of the functional calculus for normal operators; all that is needed is the observation that, since U is unitary, the spectrum of U is on the unit circle, where \p\ is, by assumption, bounded by 1. The success of this "one-variable" theory made it tempting to look for possible "several-variable" extensions. The first step was taken by Andô [l]; he proved that if Ai and A2 are commutative contractions on H, then there exist commutative unitary operators Bi and B% on a larger K such that B™B\ is a dilation of A™A% for every pair of positive integers m and n. Much to everyone's surprise, Ando's theorem turned out to be a characterization of the integer 2; Parrott [37] proved that for three or more contractions the corresponding statement is false. In fact License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 1907] TEN PROBLEMS IN HILBERT SPACE 909 even something much weaker is false: there exist three commuta tive contractions A0f A\y A% such that if B0, Bi, B2 respectively are isometric dilations (not necessarily unitary and not necessarily power dilations), then the set {B0t Bi, B2} cannot be commutative. The following presentation of Parrott's idea is due to Chandler Davis. Suppose that C0, Ci, C2 are isometries on some Hilbert space; write /0 0\ It is clear that the ^4's are contractions, and, since the product of any two of them is 0, it is clear that they commute. If B0, Bi, and B% are dilations of A0, Au and A2 respectively, to the same enlarged space, then they can be written in the form (0 0 *] Bi= Id 0 * , [Di Ei *J where the entries indicated by * need not be known. If each Bi is an isometry, so that II*<<ƒ, 0, 0>|| » = || <ƒ, 0,0>||» for all ƒ, then it follows that IM« + M'-IWI' for all/; since C» is an isometry, it follows that Di must be 0. A similar glance at (0,/, 0) shows that E» must be an isometry. It is to be proved that the Cs can be chosen so that the .B's do not commute. Assertion: if C0 = l and G and C2 do not commute (so that the Hilbert space they act on has dimension at least 2), then the desired result is achieved. Suppose, on the contrary, that the B's do commute. Since the entry in row 3 and column 1 of the product BiBj is EiCj, it follows that E1C2 = E2C1, EQCI = Ei, and EQC% = E%. This implies that E0C1C2 = E1C2 ^ E2C1 = E0C2C1, and therefore, since Eo is an isometry, that CiCi — CiCi) the con tradiction has arrived and the proof is complete. ANSWERS TO THE EXERCISES. (1) The only compact Toeplitz operator is 0. Indeed, if License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 910 P. R. HALMOS [September T4, is compact, then ||2V»||—»0 (since en—>0 weakly); it follows that (<£, eu) = 0 for all k (positive, negative, or zero), and hence that$ = 0. (2) If A = r+C, where T is a Toeplitz operator and C is compact, then U*A U~T+U*CU, where U is the unilateral shift, and there fore U^AU^A+Kj where K is compact. This implies that not every A has the form T+C. For an example, let A be the projection onto the span of {e0, e2, e^ • • • }, so that Aen = en when n is even and Aen = 0 when n is odd. If n is even, then U*A Uen = 0; if n is odd, then U*A Uen = en. Conclusion: A — U*A U is not compact. (3) If skew projections are allowed, then every operator has a unitary dilation. Indeed, given A on H, write K=H@H, identify H, as usual, with H@0, let B on K be given by (? J), and let P the skew projection (J 0) from K onto if. Since PB has the matrix (0 o)> the result follows. A generalization of this technique can be used to produce, for each positive integer n, a unitary operator B such that Bk is a skew dilation of Ak for each k = 1, • • • , n. For the typical case » = 2, write X=-ff©Heiî, '0 0 1] 1 il ^2 £ = 1 0 0 and P = 0 0 0 ,0 1 0, 0 0 0 These results are due to L. J. Wallen and J. S. Johnson. 6. SIMILAR Problem 6. Is every polynomially bounded operator similar to a contraction! There is a sense in which questions of similarity are not "natural" in Hubert space, but the few elegant results and interesting examples already known make it hard to resist the temptation to continue looking for an adequate general theory. As is often true, the easiest operators to begin with are the unitary ones. If U is unitary and A =5~1Z75', then, of course, An = S~1UnS, and therefore mn|| Sc for every positive integer n (where c = ||5~1|| • ||5J|); in other words, each operator similar to a unitary operator is power bounded. Since A is invertible, the inequality makes sense and is true for negative exponents also. One of the earliest non-trivial results about similarity is Nagy's converse [31 ]: if A is invertible and both A and A~l are power bounded, then A is similar to a unitary operator. (Caution: small diagonal matrices show that the inverse of a power bounded invertible operator may fail to be power bounded.) This is a satisfactory state of affairs; for similarity License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use i97o] TEN PROBLEMS IN HILBERT SPACE 911 to a unitary operator there is a simple and usable necessary and sufficient condition. Exercise 1: is every idempotent operator similar to a projection? What happens if the invertibility part of Nagy's condition is dropped? In other words: is there a simply describable class of operators such that power boundedness is a necessary and sufficient condition for being similar to one of them? Superficially it might seem that the class of isometries is the answer (drop the invertibility condition from the definition of unitary operators), but that is not right. (A quick way to see that it is not right is to recall that on finite-dimensional spaces all isometries are unitary.) A reasonable second guess might lead to the question: is every power bounded operator similar to a contraction? For some classes of operators the answer is yes. It is, for instance, an easy exercise in analysis to prove that every power bounded weighted shift is similar to a contraction. It is clear that every eigenvalue of a power bounded operator must have modulus less than or equal to 1. A finite Jordan block with eigenvalue of modulus 1 (and size greater than 1) is not power bounded; if the eigenvalue has modulus less than 1, then the block is power bounded and it is easily seen to be similar to a contraction. These considerations imply that in the finite-dimensional case every power bounded operator is similar to a contraction, and it follows, of course, that the same is true for operators of finite rank on spaces of arbitrary dimension. Nagy [32] extended the result to compact operators. For not necessarily compact operators the answer is no; a counterexample was constructed by Foguel [IS], [19], Exercise 2 : every power bounded subnormal operator is similar to a contrac tion. At this point it is reasonable to begin to think that the whole approach is wrong. The preceding two paragraphs encouraged a search for a class of operators whose conjugates (i.e., transforms by similarities) were characterized by a prescribed condition (power boundedness). It might be better to prescribe an interesting class of operators (e.g., contractions) and search for a characterization of their conjugates. With hindsight it might be said that two-sided power boundedness is a natural characterization of the conjugates of unitary operators in that it is a condition on the group of invertible operators. To characterize the conjugates of contractions (which may, of course, fail to be invertible) it seems reasonable to put a condition on opera tors considered as elements of an algebra. To find a good candidate for a characteristic condition, invert the problem: what can be said License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 912 P. R. HALMOS [September about the elements of the algebra generated by an operator similar to a contraction? In other words: if C is a contraction, A =5_1C»S', and p is a polynomial, what can be said about p(A)? Since p(A) — S~lp(C)S, and since (by the von Neumann theorem mentioned in §5) ||/>(Ol| ^||^|U ( = sup{ \p(z)\ : \z\ ^1}), one answer is the in equality |[^)||^||£|U (where c = ||5-1|| • ||S||). Operators satis fying this condition for all p are called polynomially bounded] Prob lem 6 asks whether they are exactly the conjugates of contractions. Foguel's work provides an operator that is power bounded but not similar to a contraction; if that operator were polynomially bounded also, Problem 6 would be solved (in the negative). Lebow [30 ] examined Foguel's operator and proved that it is not poly nomially bounded; Problem 6 is still unsolved. Some interesting special cases of Problem 6 are solved. (i) If r(A) License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use I97O] TEN PROBLEMS IN HILBERT SPACE 913 q+ and q~ be the polynomials defined by q+(z*)=±-(p(z) + p(~z)) and gr(*«) = 1 (f(*) - ƒ(-«)), Z Z2 so that £(s) = g+(;s2) +££-(s2). Since A2 is a contraction, the von Neu mann theorem on contractions applies and proves that hHA>)\\ = \\q±\\„ Consequence: ||XA)||s ML + Mil -llrll- = (i + IMIDIWU (The last inequality follows from the definitions of g+ and (p.) What the argument really proves is that if A2 is polynomially bounded, then so is A. The extension to Aw is routine. A final small comment about (ii) must be made before the subject can be abandoned: it is not the general case. That is: there exists a polynomially bounded operator A such that no power of A is a contraction. Example (A. L. Shields) : a unilateral weighted shift with weights 1 + 1/2", n = 0, 1,2, • - • . The norm of ,4nis ftt-o (1 + 1/2*), which is always strictly greater than 1 ; to prove that A is polynomially bounded, apply the general theory of similarity for weighted shifts [20, Problem 76] and infer that A is similar to the (unweighted) unilateral shift. (iii) Exercise 3: every polynomially bounded operator is the limit (in the norm) of operators that are similar to contractions. That is: granted that equality is not yet attainable, at least asympto tic equality can be achieved. Unitary operators are too special and contractions are too general, there are several interesting classes between. What, for instance, can be said about the conjugates of isometries? Answer: A is similar to an isometry if and only if the powers of A are uniformly bounded from both above and below. To say that the powers of A are uni formly bounded from above means, of course, that the supremum of the set of numbers {\\Af\\ :» = 0,1, 2, .-., ||/|| = 1} is finite (i.e., that A is power bounded) ; the corresponding condition from below is that the infimum of the same set of numbers is (strictly) positive. The proof of the assertion is a routine imitation of Nagy's proof for the unitary case. Although this characterization of the conjugates of isometries looks as if it might be awkward to apply, it does have some pleasant License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 914 P. R. HALMOS [September consequences. These consequences are minor, and each one can be proved directly, but it is more efficient to derive them from a com mon source. Sample: every square root of an isometry is similar to an isometry. (Isometries can have non-isometric square roots. A trivial example is (J_î); a non-trivial infinite-dimensional ex ample is a unilateral weighted shift with alternating weights 2 and |.) For the proof, observe that if A2 is an isometry, then A has a left inverse and, consequently, A is bounded from below; since An is alternately an isometry and A times an isometry, the desired result follows. The extension to higher powers is obvious. Another sample: if a normal operator A is similar to an isometry, then A is an isometry (and hence A is unitary). Proof: use the power characterization of the conjugates of isometries and the spectral theorem. Here is a small surprise: for subnormal operators the con clusion is false; a counterexample was constructed by Sarason [20, Problem 156]. The problem of characterizing the conjugates of partial isometries has apparently not been studied. 2 ANSWERS TO THE EXERCISES. (1) If £ = E, form A =2E — 1, note that An — l or A according as n is even or odd, and conclude that A is similar to a unitary operator U. Since Z72 = l (because A2 = l), it follows that U is Hermitian, and hence so is J (£7+1). Conclusion: |(£/+1) is a projection similar to E ( = §(^4+1)). This is an elegant application of Nagy's similarity theorem, but it has a flaw; the trouble is that the result is easier to prove directly. Indeed: decom pose the space into ranE (=ker(l—E)) and its orthogonal comple ment; the matrix corresponding to E then takes the form (J £). If S= (o f), then 5 is invertible and 5~1E5 = (J g). No harm done: the conclusion is worth mentioning in any discussion of similarity. The second proof is due to J. G. Stampfli. (2) Suppose that A is subnormal and power bounded, and let B be its minimal normal extension. If A denotes spectrum, then A(Bn) = (A(jB))n (spectral mapping theorem) C(A(^4))W (spectral inclusion theorem for subnormal operators) =A(An). If r denotes spectral radius, then it follows that ||S»|| = f(J5») ^ f(4n) g H^ll. Consequence: B is power bounded. Since the only power bounded complex numbers are the ones of modulus less than or equal to 1, the spectral theorem implies that B must be a contraction, and it follows that so is A, Conclusion: a power bounded subnormal opera tor is not only similar to a contraction, but actually is one. License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use i97o] TEN PROBLEMS IN HILBERT SPACE 915 (3) Every operator A with r(A) ^ 1 is the limit of operators similar to contractions. (Note that if A is polynomially bounded, then it is power bounded, and therefore r(A)^l.) Indeed: if ^4n = (l — l/n)A, then r(An) 7. NILPOTENT Problem 7. Is every quasinilpotent operator the norm limit of nil- potent ones? The similarity theory of linear transformations on finite-dimen sional spaces reduces to that of nilpotent ones, and the theory of nilpotent transformations turns out to be algebraically tractable. In the infinite-dimensional case nilpotence no longer plays the same central role, and the theory of the appropriately generalized concept is both algebraically and topologically refractory. If A is nilpotent, say An = 0, then, by the spectral mapping the orem, the spectrum of A consists of 0 alone. In the finite-dimensional case the converse is true; in the infinite-dimensional case it is not. The standard example [20, Problem 80 ] is any unilateral weighted shift whose weights tend to 0. If A is such a shift, then A is not nil- potent (in fact if Anf = 0 for some positive integer n, then ƒ = 0). The operator A is, however, quasinilpotent in the sense that ||.4n||1/n—*0. n 1/n Since limnm || is always equal to the spectral radius of A [20, Problem 74], the spectrum of a quasinilpotent operator is the single ton {o}. Problem 7 is important because it calls for the discovery of new techniques; the question itself is "wrong". What is wrong is that the condition whose necessity is in question is already known to be not sufficient; a limit of nilpotent operators may fail to be quasinilpotent. The basic example is due to Kakutani [20, Problem 87]. It is a uni lateral weighted shift A whose weight sequence {ao, cei, a^ • • • } is obtained as follows: every second a is equal to 1, (i.e., License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 916 P. R. HALMOS [September directed. The more honest formulation has to be more vague; it should be something like "what is the closure of the set of nilpotent operators?". As long as the subject has been raised: what is the closure of the set of quasinilpotent operators? There is an elegant partial solution of Problem 7 due to R. G. Douglas: every compact quasinilpotent operator is the limit of nil- potent ones. The main tool in the proof is the upper semicontinuity of the spectrum [20, Problem 86]. According to that result if A is quasinilpotent, then corresponding to each positive integer n there exists a positive number en such that every operator within ew of A has a spectrum smaller than 1/n. Assume, with no loss of generality, that €»—-K). If now A is compact (as well as quasinilpotent), then for each n there exists an operator Bn of finite rank such that ||^4 — 5»|| License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 1970] TEN PROBLEMS IN HILBERT SPACE 917 in a neighborhood of 0 such that f (A) = 0.) Exercise 3: If U is the unilateral shift and A is quasinilpotent, then ||£7-il||âl. Perhaps the most important place where quasinilpotent operators enter functional analysis is in Dunford's theory of spectral operators [13]. The concept belongs to Banach spaces, but in Hubert spaces, by virtue of an elegant theorem of Wermer [52], it has an especially simple characterization. An operator A (on a Hubert space) is spectral if and only if it can be represented as a sum, A =S+Q, where S is similar to a normal operator, Q is quasinilpotent, and S and Q commute; the representation is unique. The theory of the classical Jordan form shows that on a finite-dimensional Hilbert space every operator is spectral. A basic result about spectral operators is that if A~S+Q, as above, then the spectra of A and S are the same. To prove this, ob serve first that if T is an operator that commutes with Q, then QT is quasinilpotent (because ||(Qr)n|| =||önrw||), and it follows that if, moreover, T is invertible, then T+Q is invertible (because T+Q = T(1 + T~XQ)). Suppose now that X is not in the spectrum of 5 and write T = S—X; by what was just said, T+Q is invertible, so that X is not in the spectrum of S+Q. Since 5 and S+Q play symmetric roles, it follows that, indeed, their spectra are the same. This the orem has an ingenious generalization due to Colojoarà and Foiaç [8]. They define two operators (which might as well be called A and S) to be quasinilpotent equivalent in case I n /n\ II1/n Z(-i)*( M*sH ->o I fc-0 \k/ II and in case the same is true with A and S interchanged. The point is that if A and 5 happen to commute (which happens exactly when A—S and S commute), then the sum inside the norm is equal to (A— S)n. In the commutative case quasinilpotent equivalence re duces to having a quasinilpotent difference; the non-commutative case is a proper generalization. The Colojoarâ-Foiaç theorem is that any two operators that are quasinilpotent equivalent have equal spectra. If the imaginary part of a quasinilpotent operator is compact, does it follow that the operator itself is compact? The answer is yes, and the result plays an important role in the theory of invariant subspaces. The fact was discovered almost simultaneously by Ring- rose [42] and Schwartz [47]. Schwartz's proof is sophisticated but short; it goes as follows. Consider the algebra License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 918 P. R. HALMOS [September ideal 6 of compact operators, the quotient algebra (B/6, and the canonical homomorphism TT from (B to (B/C. If Q is quasinilpotent (in (B), then TT(Q) is quasinilpotent (in (B/e); if lm Q ( = (l/2i) ((? — (?*)) is compact, then Im 7r(<2) =0. Since (B/e is representable as an operator algebra, and since the image of Q in such a representa tion has now been proved to be quasinilpotent and Hermitian, it follows that 7r(<2) =0, so that Q is compact. A nilpotent operator is locally nilpotent also; that is, if An = 0, then Anf = 0 for each ƒ. The converse is true: if for each ƒ there is an n ( =^(/)) such that Anf = 0, then there is an n (independent of/) such that An = 0. Indeed, if A is locally nilpotent, then the space is the union of the subspaces ken4n, # = 1, 2, 3, • • • . The Baire category theorem implies that ken4n has a non-empty interior for at least one nt and a subspace with a non-empty interior equals the whole space. Note: local nilpotence on a dense set is not enough to guarantee nilpotence. The preceding paragraph extends elegantly to quasinilpotence, as follows. A quasinilpotent operator is locally quasinilpotent also; that is, if ||i4n||1/n—>0, then (|^4n/||1/n—>0 for each/. To prove the con verse, observe that if ||-4W/|| License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use i97o] TEN PROBLEMS IN HILBERT SPACE 919 proof is due to D. E. Sarason.) (3) If || U-A\\<1, then ||l- U*A\\ <1, so that U*A is invertible, and therefore A is left invertible. But if BA = 1, then BnAn = l (dis card BA's from the middle), and therefore 1 g||5n||1/nmw||1/n for all n. It follows that 1 ^r(B)r(A), which cannot happen when A is quasinil- potent. Corollary: the set of quasinilpotent operators is not dense. As far as the corollary is concerned, however, Exercise 3 is not needed. For that purpose it is enough to note that if A is quasinilpotent, then || 1— ^41| èl, and that is obvious: a quasinilpotent operator surely cannot be invertible. The corollary should be contrasted with the result that the set of all nilpotent operators of index 2 is strongly dense [20, Problem 91]. The statement of Exercise 3 is due to R. A. Hirschfeld. 8. REDUCIBLE Problem 8. Is every operator the norm limit of reducible ones? Recall that a subspace ikf of a Hubert space H reduces an operator A on H in case both M and ML are invariant under A ; equivalent!y, M reduces A in case the projection with range M commutes with A. The operator A is reducible in case it has a non-trivial reducing sub- space. On a finite-dimensional Hubert space the answer to the question is no: the set (R of reducible operators is closed. Suppose, indeed, that An is reducible and An-^A, and, for each nf let Pn be a non-trivial pro jection that commutes with An. Finite-dimensionality implies the compactness of the unit ball in the space of operators. There is, therefore, no loss of generality in assuming that Pn—*P, where, of course, P is a projection, and, clearly, AP—PA. If dim H = K, then 1 S trace Pn ^k — 1 ; since trace is continuous, it follows that P^ 0, 1. This proves that (R is closed; since not every operator is reducible (witness (? S))> it follows that (R cannot be dense. Exercise 1: on a non-separable Hubert space every operator is reducible. In view of the preceding comments, the solution of Problem 8 is negative for small spaces and trivially affirmative for large ones. The scope of the problem has been reduced to medium-sized Hubert spaces (infinite-dimensional but separable), and there it is unsolved. Irreducible operators on such spaces certainly do exist; the unilateral shift is one of them. The unilateral shift, however, does not yield a negative solution of Problem 8; it can be approximated by reducible operators. License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 920 P. R. HALMOS [September Another pertinent comment concerns bilateral weighted shifts. Such a shift is reducible if and only if its weight sequence is periodic [20, Problem 129]. If the weight sequence is not periodic but is the uniform limit of periodic ones, then, of course, the corresponding shift is the norm limit of reducible ones. If, however, the weight sequence is not the uniform limit of periodic ones (if, for instance, all the weights are 1 except the one with index 0, which is 2), then it does not follow that the shift is not reducibly approximate; all that follows is that the obvious approximation breaks down. The facts are not known. Irreducible operators not only exist, they exist in profusion: the set 3 of irreducible operators (on a separable Hubert space) is dense. To prove this, consider an arbitrary operator A and write A—B+iC with B and C Hermitian. Represent B as a multiplication on L2 over a finite measure space. (This is one place where separability comes in.) The (real) multiplier can be uniformly approximated by (real) simple functions. Multiplication by a real simple function is the direct sum of a finite set of real scalars, and consequently it is a diagonal operator; a Hermitian diagonal operator can obviously be approximated by one with no repeated eigenvalues. Call such an approximant B0, and consider the matrix of C with respect to a basis formed by the eigenvectors of B0. That matrix might have some en tries equal to 0, but in any event it can be approximated by a (Hermi tian) matrix with all non-zero entries; let C0 be the operator corre sponding to such a matrix. The operator Ao—Bo+iCo approximates A ; it remains to prove that A 0 is irreducible. If a subspace M reduces A ot then M is invariant under both B0 and C0. A subspace invariant under B0 is spanned by the subset of the eigenvectors of Bo that it contains ; this is a consequence of the distinctness of the eigenvalues of Bo and is proved by a standard and elementary computation. Such a subspace, however, cannot be invariant under Co, unless it is either 0 or the whole space ; this is a consequence of the non-zeroness of the matrix entries of Co. Conclusion: Ao is irreducible, and the proof of the density of 3 is complete. The first appearance of the theorem is in [2l]; the proof here presented is due to Radjavi and Rosenthal [39]. The proof works for all separable spaces, and, in particular, for the finite-dimensional ones. Since in the finite-dimensional case 3 is open (because its complement (ft is closed), and since the complement of a dense open set is nowhere dense, it follows that in the finite- dimensional case (R is, in the sense of topological size, very small indeed. It is of interest to note that for separable spaces (ft is always topologically small (meager, set of the first category). Explicitly: if License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 1970] TEN PROBLEMS IN HILBERT SPACE 921 the space is separable, then (R is an Fff [21 ]. The proof that License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 922 P. R. HALMOS [September separable infinite-dimensional Hilbert space) that have a reducing subspace of dimension 1 is not dense, but (Exercise 3): its closure contains every isometry. (ii) Every operator (on a separable Hilbert space) is the sum of two irreducible ones [14], [38]. ANSWERS TO THE EXERCISES. (1) If A is an operator on H and ƒ is a non-zero vector in H, then the smallest subspace of H that contains ƒ and reduces A is spanned by ƒ together with the set of all vectors of the form A\ • • • Anf> where n is an arbitrary positive integer and each Aj is either A or A*. It follows that that (non-zero) subspace is separable; if H is not separable, then the construction has yielded a non-trivial subspace that reduces A. (2) Given A, find an approximate eigenvector for A, use it as the first element of a basis, form the matrix of A, and then find an ap- proximant by replacing by 0 all but the first entry in the first column. Conclusion: every operator can be approximated by operators with eigenvalues. (3) It is to be proved that if U is an isometry, then U can be approximated by operators with reducing eigenvectors. The proof is similar to the one in (2). Let X be an approximate eigenvalue of U with modulus 1. It follows, by definition, that corresponding to each positive number e there exists a unit vector e such that || Ue— \e\\ 9. REFLEXIVE Problem 9. Is every complete Boolean algebra reflexivet The invariant subspace problem asks whether the set of all invariant subspaces of an operator can consist of the two extremes only, but what people would really like to know is what the set of all invariant subspaces of an operator, or, for that matter, of any set of operators can look like. A few necessary conditions are easy to ob tain. If, for instance, & is a set of operators (on H) and <£ is the set of all those subspaces (of H) that are invariant under every operator in ®, then it is clear that <£ is a lattice (i.e., that £ is closed under the formation of intersections and spans), and it is even clearer that £ contains 0 and H. In addition to these algebraic conditions, £ satisfies a somewhat less obvious topological condition, namely that it is strongly closed. (Explanation: the set of all projections whose ranges are in £ is strongly closed. Proof: if A is in Ct and {P„} is a net License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use î97ol TEN PROBLEMS IN HILBERT SPACE 923 of projections strongly convergent to P and satisfying APn~PnAPn for each n, then, because multiplication is jointly continuous on bounded sets, AP=PAP.) The topological condition has algebraic reverberations. Exercise 1: a strongly closed lattice is complete. (Caution: the converse is false. Example: discard a 1-dimensional space from the lattice of all subspaces of a 2-dimensional space; the remainder is a complete but non-closed lattice.) It is easy to show that the necessary conditions listed so far are nowhere near sufficient; the characterization problem for the set of all invariant subspaces of a set of operators is still open. There is a kind of Galois theory connecting sets of operators and sets of subspaces. Half of it was just described: to each set Cfc of operators there corresponds the set of those subspaces that are in variant under the elements of Q; call that set Lat®. The other half goes backwards : to each set £ of subspaces there corresponds the set Alg£ of those operators that leave invariant each element of £. It is clear that Alg<£ is always an algebra (closed under the formation of sums, products, and scalar multiples); it is even clearer that AlgJB contains 1 ; and the one-sided weak continuity of multiplication im plies that Alg<£ is weakly closed. The basic properties of Alg and Lat are easy; their proofs belong to universal algebra. The facts are that both Alg and Lat are order- reversing, and that if <£ is any set of subspaces and Q is any set of operators, then £ C Lat Alg£ and a C Alg Lata. (Exercise 2 : Alg £ = Alg Lat Alg £ and Lat a = Lat Alg Lat CL Anal ogy with linear algebra (when is an object equal to its own second dual?) suggests that if <£=Lat Alg<£, then the lattice £ be called reflexive; the same word is used for an algebra & such that Ct = Alg Lat Q. Word-saving convention: every lattice contains 0 and H and is strongly closed, every algebra contains 1 and is weakly closed. The theory of reflexive algebras is relatively new and, judging from the analytic techniques it has been using, quite deep. A typical result is Sarason's [45]: a commutative algebra of normal operators is reflexive. Example: the set of all matrices of the form (S°). In a different direction, Radjavi and Rosenthal [40] generalized a theorem of Arveson's [3] as follows: if d includes a maximal abelian self-adjoint algebra and if LatCt is a chain (totally ordered), then Ct is reflexive. Example: the set of all matrices of the form (£°). A typical example of a non-reflexive algebra is the set of all matrices of License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 924 P. R. HALMOS [September the form (J 2). A complete characterization of reflexive algebras is unknown even for finite-dimensional spaces. To ask about invariant subspace lattices of sets of operators is the same as to ask about reflexive lattices. Here are two trivial but illu minating examples. (i) If H is 2-dimensional and «£ consists of 0, H, and two distinct 1-dimensional subspaces of H} then <£ is reflexive. To say of an opera tor that it leaves invariant each subspace in £ is to say just that it has two prescribed eigenvectors, and hence that its matrix with respect to the basis they form is diagonal. Since the only subspaces simul taneously invariant under all such diagonal operators are the ones in £, the lattice £ is reflexive, (ii) Suppose again that H is 2-dimen sional, and let £ consist of 0, H, and three distinct 1-dimensional subspaces. It is very hard for an operator on H to have three distinct eigenvectors; the only operators that can do it are the scalars. Scalars, on the other hand, leave invariant every subspace of H; the lattice £ is not reflexive. The first non-trivial theorem about reflexive lattices is a sharpening of some related work of Kadison and Singer [27]; the statement, proved in complete generality by Ringrose [43], is that every com plete chain is reflexive. Here again "chain" means a lattice whose order (in general partial) is in fact total. No topological assumption is needed: it is easy to prove that a complete chain of subspaces is necessarily strongly closed. The smallest lattice that is not a chain is a 4-element Boolean algebra, i.e., a lattice of the form <£ = {0, M, N, H}, where the sub- spaces M and iVare such that ikffW==0 and M\/N = H. It is true but not obvious that each such £ is reflexive. To prove this it is sufficient (and necessary) to exhibit a set Œ of operators such that Lat Q, = £. Let P and Q be the projections onto M and N respectively and let (X be the set of all operators that are either of the form PA (1 — Q) or else of the form QA(l— P). Since PA(1— Q) annihilates N and maps everything into M, and, similarly, QA(\—P) annihilates M and maps everything into N, it follows that £ C Latö. It remains to prove the reverse inclusion. A useful lemma is that if K is in LatCt and K(£.M, then KZ)N. (Similarly, of course, if K is in LatCt and K(£N, then SDM.) For the proof, take a vector ƒ in K but not in M, and take an arbitrary vector g in N. Since ƒ is not in M, it follows that (1— P)/T^0, and hence that there exists an operator A such that A(l—P)/ = g. Since QA(1-P)f is in K (because K is in Lata) and QA(1-P)f=g (be cause g is in N), it follows that g is in K; this proves the lemma. License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 197°] TEN PROBLEMS IN HILBERT SPACE 925 Suppose now that K is in Lat &, K^Q, M, N; it is to be proved that K~H. The subspace K cannot be included in both M and N (because X^O). If, say, K(^M1 then, by the lemma, K'DN. Since, however, Kj^N, it is not the case that K License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 926 P. R. HALMOS [September infinite, the pentagon can occur, and in the one manifestation that has been studied it turns out to be reflexive. (Can it have a non- reflexive occurrence? The answer is not known.) As for the double triangle (= {0, L, Af, N, H], with LC\M = Lr\N = M(^N = 0 and L\/M — L\/N = M\/N = H)i all its known manifestations are non- reflexive. The known manifestations include all possible finite- dimensional ones; the unknown cases are all infinite-dimensional. The finite-dimensional facts are covered by a theorem of R. E. Johnson [25]: a finite lattice of subspaces of a finite-dimensional space is reflexive if and only if it is distributive. The "if" remains true for infinite-dimensional spaces (K. J. Harrison); the reflexive read ability of the pentagon shows that the "only if" is false for them. ANSWERS TO THE EXERCISES. (1) Suppose that <£ is a strongly closed lattice and that £o is a subset of «C. Consider the directed set of all finite subsets of <£o ordered by inclusion. If n is such a finite subset, let Mn be the intersection of the subspaces in n. Since n—>Mn is a de creasing net of subspaces, the corresponding net of projections is strongly convergent to the projection whose range is n<£0; this proves that C\£o is in «£. The proof for spans is similar. (2) Apply Lat to the relation CbCAlg Lata, and apply the relation JCCLat Alg<£ to Latö, in place of £. This proves that Lat G, = Lat Alg Lat &; the equation Alg<£ = Alg Lat Alg«£ is proved dually. Corollary: every lattice that is the Lat of something is the Lat of its own Alg. (3) If dim H< oo, and if £ = Lat A for some operator A on H, then <£ is self-dual, in the sense that <£ is isomorphic to a lattice in which all the order relations of £ are reversed. Proof: the formation of or thogonal complements proves that Lat A is anti-isomorphic to Lat A * ; since every matrix is similar to its transpose, and similar operators have isomorphic lattices, the proof can be completed by recalling that the matrix of A* is the conjugate transpose of that of A [S]. 10. TRANSITIVE Problem 10. Is every non-trivial strongly closed transitive atomic lattice either medial or self-conjugated This is the most awkward of the ten problems here proposed; the reason is that it points to the darkest area of ignorance. The crucial word is "transitive". A set & of operators is called transitive in case Lat(£ = <£min ( = the smallest lattice, the one con sisting of 0 and H only) ; a set £ of subspaces is transitive in case Alg£ = C£min ( = the smallest algebra, the one consisting of scalars only). License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 1970] TEN PROBLEMS IN HILBERT SPACE 927 If a set Ofc of operators is transitive, then so is every subset of 6, that generates the same weakly closed algebra. It follows that the search for transitive sets might as well be restricted to weakly closed algebras. (Word-saving convention: every algebra contains 1, every lattice contains 0 and H.) Problem 10 has a dual, a problem for algebras, that is easier to state: is every weakly closed transitive algebra equal to Œmax ( = the largest algebra, the one consisting of all operators)? It is implicit in the question that Œmax itself is transitive; if a subspace M of H is invariant under every operator on H, then M = 0 or M = H. Conjecture: the solution of this dual of Problem 10 is negative. In fact, there probably exists a commutative transitive algebra. This conjecture, in turn, is a special case of one already on record (in §3) : if a transitive operator exists (in present language an operator A is transitive in case Lat A = <£min) » then the weakly closed algebra it generates is a transitive algebra. All these conjectures seem to be out of reach at present. If H is small (finite-dimensional), then Burnside's classical theorem applies [24, p. 276] and says that the only transitive algebra is Cfcmax- If H is very large (non-separable), then an easy cardinality argument shows that no countably gen erated algebra can be transitive. The cases between these two ex tremes remain shrouded in mystery. The corresponding questions for lattices are newer, and, although the principal characterization problem is still unsolved, they promise to be somewhat more accessible. The first thing to settle is that the definition is not vacuous: are there any transitive lattices? Since a lattice larger than a transitive one is also transitive, the question reduces to a known projective geometric fact (Exercise 1): if £ = £m&x ( = the largest lattice, the one consisting of all subspaces), then £ is transitive. Once that's settled, it is natural to regard £m&x as a trivial example and to ask whether there exist any non-trivial strongly closed transi tive lattices. The answer is yes (J. E. McLaughlin), but that's not obvious. To get an interesting class of examples, consider an arbitrary conjugation Zona Hubert space H. (A conjugation is an involutory semilinear isometry, i.e., a mapping J oî H into itself such that 2 2 J = l, J(af+pg) =a*Jf+P*Jg, and ||//|| =||/||. If H is an L space, then an example of a conjugation can be obtained by writing /ƒ=ƒ*, and, conversely, every conjugation can be represented in this way. For a detailed discussion see [50, p. 357].) Call a subspace M oî H self-conjugate (with respect to /) in case Jf is in M whenever ƒ is in Jlf, and let £ ( = <£./) be the set of all self-con jugate subspaces. It is easy to verify that £ is a strongly closed transitive lattice. (The proof License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 928 P. R. HALMOS [September of transitivity uses the same technique as is needed to prove the transitivity of cCmax.) Call every £ obtained in this way a self-con- jugate lattice; there are as many of them as there are J's. Once that's settled, it is again natural to ask whether every transi tive lattice has been found by now. In other words, do there exist non-trivial strongly closed transitive lattices other than the self-con jugate ones? The answer is yes again, but that's even less obvious than anything that led up to it. To construct an example, let K be a Hilbert space, write H = K®K, and let £ be the 7-element lattice of subspaces of H consisting of 0, H, the two axes (K00 and Q@K), the diagonal (the set of all (ƒ, //s with ƒ in K), and the graphs of two operators S and T on K. Assertion : for suitable choice of S and T} the lattice «£ is transitive. Since H is given as a direct sum of two copies of K, every operator on iî is a two-by-two matrix of operators on K. The assumption that £ contains the two axes implies that every operator in Alg<£ has the form (S y) ; the assumption that £ contains the diagonal implies that The operators S and T are to be chosen so that their graphs are disjoint from one another and from the other non-tri vial elements of £ (i.e., from the axes and the diagonal). When can (ƒ, Sf) be equal to (g» Tg)ï Answer: exactly when f=g and (S— r)/=0. Consequence: graph £n graph T = 0 if and only if ker (S—T) =0. Since the hori zontal axis (K@Q) and the diagonal are graphs, and since every graph is disjoint from the vertical axis, it follows that the five non- trivial elements of £ are pairwise disjoint if and only if ker S = ker T = ker(l - S) = ker(l - T) = ker(5 - T) = 0. The consideration of the orthogonal complements of graphs shows that the five non-trivial elements of £ pairwise span H if and only if their adjoints satisfy the vanishing kernel conditions just found. (If K is finite-dimensional, the resulting conditions are equivalent to the original ones; in the infinite-dimensional case they are not. Note that the orthogonal complement of the graph of, say, 5 is the "co-graph " consisting of all vectors of the form ( — 5*/, ƒ).) When does (J x) leave invariant the graph of 5? An obvious com putation shows that a necessary and sufficient condition is that SX — XS. Consequence: £ is transitive if and only if ComSPi ComT = GW, where "Com" means commutant. The problem of finding a transitive lattice of the kind promised License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use i97o] TEN PROBLEMS IN HILBERT SPACE 929 above comes down to this: find two operators 5 and T satisfying the commutant condition just stated and such that both they and their adjoints satisfy the vanishing kernel condition. That is not difficult, but it does seem to lead to some matrix computation. One example (with dim K< oo) is obtained by letting S be the sum of a nilpotent Jordan block and a scalar, say, 2 (what is important about 2 is that it is neither 0 nor 1), and letting T be a diagonal matrix none of whose eigenvalues is 0, 1, or 2. The only condition whose verification is not obvious is the one involving ker (5— T). To prove it, compute T^S and note that it is a lower triangular matrix with all diagonal entries distinct from 1; it follows that 7*~lS— 1 is invertible, and hence so is S-T. A similar example can be constructed on infinite-dimensional spaces. Suppose, for instance, that K is L2 of the circle, let 5 be the bilateral shift (Sen~en+i for all n), and let T be a diagonal operator (Ten =X»en for all n). If the Vs are bounded, if none of them is 0 or 1, and if they are such that 00 00 2 E I lAi • • • Xft| < «) and El lA-i * • • M* < °°t n»l »-l then 5 and T satisfy all the requirements. The proof is straightfor ward. The lattices £> just constructed have the property that any two of their non-trivial elements are complements. Since in the lattice diagram of such an £ all the non-trivial elements occur on the same level, halfway between 0 and £T, the word medial might serve as an adequate description. P. Rosenthal reports that in an infinite-di mensional space a transitive medial lattice can be constructed with only four non-trivial elements. The construction uses, as above, the two axes and the diagonal, but only one graph, namely the graph of a suitably chosen unbounded transformation. It is not known whether three non-tri vial elements can do the job. In finite-dimen sional spaces (of dimension greater than 2) a medial lattice with four (or fewer) non-trivial elements can never be transitive; the proof proceeds by observing that any such lattice can be repre sented, as above, by two axes and two (or fewer) graphs, and then proving that there are too many degrees of freedom to make transi tivity possible. Exercise 2 : is every medial lattice with five (or more) non-trivial elements transitive? Each new construction brings with it the question whether it's the last. Do there exist non-trivial strongly closed transitive lattices License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 930 P. R. HALMOS [September other than the medial or the self-conjugate ones? Once again the answer is yes, but that yes is at the boundary of what is known today. K. J. Harrison has constructed a transitive lattice of 16 non-trivial elements (the dimension of the space is at least 8), of which 5 are atoms, 5 are co-atoms, and 6 are in the middle. It seems hard to describe a general class of which Harrison's example is an instance and which, together with the classes described before, has a chance of catching all transitive lattices. A reasonable guess is that atoms play a crucial role. Recall that a lattice is called atomic in case every ele ment is the span of all the atoms it includes. (Exercise 3: if 2 ^dim H < oo and if a lattice <£ of subspaces of H has exactly one atom, then £ is not transitive.) The trouble with Harrison's 18-element lattice is that it is not atomic. A fussy examination of low-dimensional cases indicates that transitive lattices want to be atomic. In the infinite- dimensional case atomicity still makes sense but it may be too special to play an important role. In any event, since no intelligent guess at the structure of all (strongly closed) transitive lattices is at hand, the best that can be done is to grasp at the atomic straw; that is what Problem 10 does. ANSWERS TO THE EXERCISES. (1) It is to be proved that if an operator A on H leaves invariant every subspace of H, then A is a scalar. The assumption implies that to every non-zero vector ƒ there corresponds a unique scalar X(f) such that Af=\(f)f. Since A(f+g) = Af+Ag, it follows that (X(/)-X(f+g))/+(X(g)-\(f+g))g = 0; this implies that if ƒ and g are linearly independent, then X(f) =X(g) (=X(f+g)). If, on the other hand, f=ag (with ce^O), then Af = aAg and therefore, again, X(f) =X(g), i.e., the function X is a constant. (2) There exist intransitive medial lattices with exactly five non- trivial elements. For a simple example in H — K@K (where K is separable and dim K ^2), let £ consist of the two axes, the diagonal, and the graphs of two operators S and T. If, for instance, both S and T are diagonal matrices, such that the set of all diagonal entries in both of them together is a set of distinct numbers all distinct from 0 and 1, then the vanishing kernel conditions are satisfied. It follows that £ is indeed a medial lattice. Since both Com 5 and Com T consist of all diagonal matrices, it follows that Alg«£ consists of all (**)* where X is diagonal; since dim K^2, this implies that Alg£ (3) If £ has exactly one atom Af0, then, since finite-dimensionality implies that every element of £ includes at least one atom, it follows that every element of £ includes M0. If M0 = H, then £ is not transi tive (because dim ff^2). If M^H^ then there exists a non-scalar License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use i97o] TEN PROBLEMS IN HILBERT SPACE 931 operator A on H with range M0. Such an A leaves invariant every M in£;indeedAMCAH=MoCM. BIBLIOGRAPHY 1. T. Andô, On a pair of commutative contractions. Acta Sci. Math. (Szeged) 24 (1963), 88-90. MR 27 #5132. 2. N. Aronszajn and K. T. Smith, Invariant subspaces of completely continuous operators, Ann of Math. (2) 60 (1954), 345-350. MR 16, 488. 3. W. B. Arveson, A density theorem for operator algebras, Duke Math. J. 34 (1967), 635-647. MR 36 #4345. 4. E. Bishop, Spectral theory for operators on a Banach space, Trans. Amer. Math. Soc. 86 (1957), 414-445. MR 20 #7217. 5. L. Brickman and P. A. Fillmore, The invariant subspace lattice of a linear trans formation, Canad. J. Math. 19 (1967), 810-822. MR 35 #4242. 6. A. Brown and P. R. Halmos, Algebraic properties of Toeplitz operators, J. Reine Angew. Math. 213 (1963/64), 89-102. MR 28 #3350; errata MR 30, 1205. 7. A. Brown, P. R. Halmos, and A. L. Shields, Cesâro operators, Acta Sci. Math. (Szeged) 26 (1965), 125-137. MR 32 #4539. 8. I. Colojoarà and C. Foias, Quasi-nilpotent equivalence of not necessarily com* muting operators, J. Math Mech. 15 (1966), 521-540. MR 33 #570. 9. , Theory of generalized spectral operators, Mathematics and its Applica tions, vol. 9, Gordon and Breach, New York, 1968. 10. D. Deckard, R. G. Douglas, and C. Pearcy, On invariant subspaces of quasi- triangular operators, Amer. J. Math. 91 (1969), 637-647. 11. R. G. Douglas and C. Pearcy, A characterization of thin operators, Acta Sci. Math. (Szeged) 29 (1968), 295-297. MR 38 #2628. 12. , A note on quasitriangular operators, Duke Math. J. 37 (1970), 177-188. 13. N. Dunford, A survey of the theory of spectral operators, Bull. Amer. Math. Soc. 64 (1958), 217-274. MR 21 #3616. 14. P. A. Fillmore and D. M. Topping, Sums of irreducible operators, Proc. Amer. Math. Soc. 20 (1969), 131-133. MR 38 #1549. 15. S. Foguel, A counterexample to a problem of Sz.-Nagy, Proc. Amer. Math. Soc. 15 (1964), 788-790. MR 29 #2646. 16. R. Gellar, Cyclic vectors and parts of the spectrum of a weighted shift, Trans. Amer. Math. Soc. 146 (1969), 69-85. 17. P. R. Halmos, Measure theory, Van Nostrand, Princeton, N. J., 1950. MR 11, 504. 18. , Normal dilations and extensions of operators, Summa Brasil. Math. 2 (1950), 125-134. MR 13, 359. 19. , On FogueVs answer to Nagy's question, Proc. Amer. Math. Soc. IS (1964), 791-793. MR 29 #2647. 20. , A Hilbert space problem book, Van Nostrand, Princeton, N. J., 1967. MR 34 #8178. 21. , Irreducible operators, Michigan Math. J. 15 (1968), 215-223. MR 37 #6788. 22. , Quasitriangular operators, Acta Sci. Math. (Szeged) 29 (1968), 283- 293. MR 38 #2627. 23. , Invariant subspaces, Abstract Spaces and Approximation, Proc. M. R. I. Oberwolfach, Birkhâuser, Basel, 1968, pp. 26-30. License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 932 P. R. HALMOS [September 24. N. Jacobson, Lectures in abstract algebra. Vol. II: Linear algebrat Van Nos- trand, Princeton, N. J., 1953. MR 14, 837. 25. R. E. Johnson, Distinguished rings of linear transformations, Trans. Amer. Math. Soc. Ill (1964), 400-412. MR 28 #5088. 26. R. V. Kadison, Strong continuity of operator functions. Pacific J. Math. 26 (1968), 121-129. MR 37 #6766. 27. R. V. Kadison and I. M. Singer, Triangular opeartor algebras. Fundamentals and hyperreducible theory, Amer. J. Math. 82 (1960), 227-259. MR 22 #12409. 28. G. K. Kalisch, On similarity, reducing manifolds, and unitary equivalence of certain Volterra operators, Ann. of Math. (2) 66 (1957), 481-494. MR 19, 970. 29. R. L. Kelley, Weighted shifts on Hubert space, Dissertation, University of Michigan, Ann Arbor, Mich., 1966. 30. A. Lebow, A power-bounded operator that is not polynomially bounded, Michi gan Math. J. 15 (1968), 397-399. MR 38 #5047. 31. B. Sz.-Nagy, On uniformly bounded linear transformations in Hubert space, ActaSci. Math. (Szeged) 11 (1947), 152-157. MR 9,191. 32. 1 Completely continuous operators with uniformly bounded iterates, Mag yar Tud. Akad. Mat. Kutató Int. Közl. 4 (1959), 89-93. MR 21 #7436. 33. B. Sz.-Nagy and C. Foiaç, Opérateurs sans multiplicité, Acta Sci. Math. (Szeged) 30 (1969), 1-18. 34. N. K. Nikol'skiï, Invariant subspaces of weighted shift operators, Mat. Sb. 74 (116) (1967), 171-190-Math. USSR Sb. 3 (1967), 159-176. MR 37 #4659. 35. R. S. Palais, Seminar on the Atiyah-Singer index theorem, Ann. of Math. Studies, no. 57, Princeton Univ. Press, Princeton, N. J., 1965. MR 33 #6649. 36. S. K. Parrott, Weighted translation operators, Dissertation, University of Michigan, Ann Arbor, Mich., 1965. 37. f Unitary dilations for commuting contractions, Pacific J. Math, (to appear), 38. H. Radjavi. Every operator is the sum of two irreducible ones, Proc. Amer. Math. Soc. 21 (1969), 251-252. MR 38 #6388. 39. H. Radjavi and P. M. Rosenthal, The set of irreducible operators is dense, Proc. Amer. Math. Soc. 21 (1969), 256. MR 38 #5042. 40, 1 On invariant subspaces and reflexive algebras, Amer. J. Math. 91 (1969), 683-692. 41. W. C. Ridge, Approximate point spectrum of a weighted shift. Trans. Amer. Math. Soa 147 (1970), 349-356. 42. J. R. Ringrose, On the triangular representation of integral operators, Proc. London. Math. Soc. (3) 12 (1962), 385-399. MR 25 #3372. 43. 1 Qn some algebras of operators, Proc» London Math. Soc. (3) 15 (1965), 61-83. MR 30 #1405. 44. P. M. Rosenthal, On lattices of invariant subspaces, Dissertation, University of Michigan, Ann Arbor, Mich., 1967. 45. D. E. Sarason, Invariant subspaces and unstarred operator algebras, Pacific J. Math. 17 (1966), 511-517. MR 33 #590. 46. H. H. Schaefer, Eine Bemerkung zur Existenz invarianter Teilràume linearer Abbildungen, Math. Z. 82 (1963), 90. MR 27 #4815. 47. J. Schwartz, Subdiagonalization of operators in Hubert space with compact imaginary part, Comm. Pure Appl. Math. 15 (1962), 159-172. MR 26 #1759. 48. A. L. Shields, A note on invariant subspaces, Michigan Math. J. (toappear). License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use i97o] TEN PROBLEMS IN HILBERT SPACE 933 49. J. G. Stampfli, Which weighted shifts are subnormal? Pacific J. Math. 17 (1966), 367-379. MR 33 #1740. 50. M. H. Stone, Linear transformations in Hubert space, Amer. Math. Soc. Colloq. Publ., vol. 15, Amer. Math. Soc, Providence, R. L, 1932. 51. N. Suzuki, On the irreducibility of weighted shifts, Proc. Amer. Math. Soc. 22 (1969), 579-581. 52. J. Wermer, Commuting spectral measures on Hubert space, Pacific J. Math. 4 (1954), 355-361. MR 16, 143. 53. H. Weyl, "Ober beschrànkte quadratische Formen deren Differ enz vollstetig ist, Rend. Circ. Mat. Palermo 27 (1909), 373-392. INDIANA UNIVERSITY, BLOOMINGTON, INDIANA 47401 NOTE ADDED IN PROOF. One of the consequences of the preprint system of scientific communication is that some part of almost everything that is printed is superseded by the time it is published. The present paper is no exception apparently. Solutions have been reported to two of the ten problems above: Problem 1, negative, J. G. Stampfli; Problem 4, affirmative, I. D. Berg. License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use