CHARACTERIZATIONS OF BOUNDED MEAN OSCILLATION ON THE POLYDISK IN TERMS OF HANKEL OPERATORS AND CARLESON MEASURES SARAH H. FERGUSON AND CORA SADOSKY* Abstract. A scale of BMO spaces appears naturally in product spaces corre- sponding to the different, yet equivalent, characterizations of the class of functions of bounded mean oscillation in one variable. S.-Y. Chang and R. Fefferman charac- terized product BMO, the dual of the (real) Hardy space H~e on product domains, in terms of Carleson measures. Here we describe two other BMO spaces, one con- tained in and the other containing product BMO, in terms of Carleson measures and Hankel operators. Both of these spaces play a significant role in harmonic analysis on the polydisk and in multivariable operator theory. 0 Introduction Recall that a function ~b, integrable on the circle T, is said to be of bounded mean oscillation, r ~ BMO(T) if I1r = sup T~ [r162 - r < oo, Ic_T I I Jz where Ct =~ft r162 is the mean value of the function r over the interval I, [JoNi]. In their seminal paper [FeSt], C. Fefferman and E. M. Stein gave alternative characterizations of the space BMO (T) in terms of duality and Carleson measures on the unit disk D. The famous duality result asserts that BMO(T) is the real dual of the Hardy space H~e (T) [Fe]. It is this characterization, together with Nehari's theorem, that connects the theory of BMO with the study of Hankel operators on H2( r). In passing from the disk to the polydisk, there are natural extensions of BMO in terms of bounded mean oscillation, duality and Carleson measures. An early example of Carleson [Carl] showed that these extensions are not equivalent. In *First author supported in part by a grant from the National Science Foundation. Second author supported in part by a grant from the Department of Energy. 239 JOURNALD'ANALYSE MATH[~MATIQUE, Vol. 81 (2000) 240 S. H. FERGUSON AND C. SADOSKY particular, the product analogue of the Carleson measure condition does not char- acterize the dual of H~e(T 2) [Fell. However, there is a description of the dual of H~e ('/~) in terms of Carleson measures (appropriately defined) on the bidisk, [Ch] and [Fell. Later in [ChFef], an atomic decomposition for H~e (~) (by atoms supported on open sets) was established, providing a version of bounded mean oscillation to characterize the dual of Hl(~). The space BMO(~) := H~e('Fd) *, d > 1, is called product BMO. The paper [ChFefl] is an excellent survey on this space and the product space program of Chang and Fefferman. The success in extending the central duality result to Hl(qrd), d > 1, which required deep analysis, deflected interest from other notions of bounded mean oscillation in product spaces, even at the cost of losing essential features of the one-dimensional theory. In fact, many of the defining properties of BMO, including bounded mean oscillation itself, do not carry over to product BMO. In this paper, we show how to recover the lost features of the theory in the context of the d-torus ql~, d > 1. For this, it is necessary to deal with two different spaces, whose definitions are natural analogues of BMO on the circle. The first space is the so-called little BMO space, denoted bmo(Td), studied by M. Cotlar and the second author in connection with the product Hilbert transform and weighted norm inequalities on the d-torus [CoSa]. Little BMO coincides with the class of functions which are of bounded mean oscillation on rectangles. A function r E Ll('Iff2) is in bmo(q~) if and only if [lr = sup _w Ir162215 dm(~)dm(A) < c~, LJCV II} IJ} where the constant Cr• is the average of r over the rectangle I • J. In one variable, bmo(T) = BMO('F); but, for all d > 1, we have the proper containment bmo(']~) C BMO(qI~), hence the name little BMO; cf. [CoSa2]. The second space considered here is the rectangular BMO space, denoted BMORect(qFd), which was introduced and studied in [Fet] and, more recently, in [Fefl] in connection with singular integral operators in product domains. A function r ~ L~(~) is in BMORect(Te) if [{~bllRect= z,J_CTsup ff[1 ~1 fxfJ 1r A) - ~bj(ff) - Cz(A) + ~bz•162 < ~, where for each Iffl = 1, Cj(~) = ~ fj r n)dm(~) and, similarly, Cz(A) is the mean value of r A) over the interval I. In terms of Carleson measures on the bidisk, BMORect(qF2) coincides with the space of functions whose associated measure (determined by the double Poisson integral) satisfies the Carleson condition on rectangles. Using a construction due BOUNDED MEAN OSCILLATION ON THE POLYDISK 241 to Carleson [Car I ], R. Fefferman showed directly that BMO (q~) c BMO n~ct(q~) by producing a function in rectangular BMO which did not determine a bounded linear functional on H~ (T2) (via the usual integral pairing) [Fef]. It turns out that both bmo(Td) and BMOnect(T a) can be described completely in terms of Hankel operators on H2(qI~). We motivate this by recalling the one- variable relation between bounded mean oscillation and norms of Hankel operators. Let P+ : L 2(~7) --+ H 2 (qF) be the orthogonal projection onto the Hardy space H 2 ('IF) and let pl = I - P+. For ~ E L2('F), the operator FO, densely defined on analytic polynomials by the formula I'of := Pl(~bf), is called a Hankel operator with symbol ~b. Note that the symbol of the Hankel operator Fo is not unique, since it is determined only by P• Nehari's theorem [Ne] says that Fo extends to a bounded operator, defined on all of H 2 (7), if and only if there is a function/3 ~ L ~ ('~) such that F~ = F~ and that furthermore, in this case, IIr ll = infpJ-~=v• 11/311 , The Hilbert transform H acts on L 2 (I?) as H = -~P+ + iP-L; and since H~(qF) is the subspace of functions that are in L 1(T) together with their Hilbert transform, the duality BMO(qF) = H~e(qF)* yields the identification BMO(qF) = L~ + H(L~ In terms of orthogonal projections, we see that a function ~b E L1($) is in BMO(qF) if and only if there are functions ~1,/32 E L~176('1~) such that P+~b = P+/31 and P• = P• Thus, by Nehari's theorem together with the fact that H2(T) • is the space of complex conjugates of functions in H02(T), the space BMO('F) coincides with the space of functions ~b E L ~(T) for which both Hankel operators, F~ and F~, determine bounded operators on H2(qr). This, in turn, is equivalent to the L 2 boundedness of the commutator [M~, H ], where M~ is multiplication by the function ~b. Thus the characterization of BMO(qF) as dual to H~(~) yields the equivalence of the seminorms, I1~11. (defined above) and max{llr, ll, IIr~ll}. We shall show that the exact analogue of the one-variable relationship between bounded mean oscillation, bounded commutators and Hankel operators carries over to the space bmo(~) and the big Hankel operators on H 2 (T t). Furthermore, using new norm estimates for these operators, we show that the big Hankel operator F~ is bounded on H2('I~) if and only if sup~ew liFq~K~[I < c~, where K~ denotes the normalized Szeg6 kernel on the polydisk. A proof of this in the one variable case can be found in [Bon] and [Nik]. This result provides us with a uniform Carleson criterion to describe the space bmo(~). The Carleson criterion is then used to characterize the analytic functions of bounded mean oscillation on rectangles in terms of analytic multipliers between H ~(T a) and the d spaces of partial derivatives of functions in H2(qra), denoted by O~H2(II~), j = 1,..., d. Comparing with the analytic versions of results in [Ch] and [Fef], which describe the space BMOA('Fa) ( = P+(L~176 ) as multipliers from Hz(Da) into the space of mixed partials 242 S. H. FERGUSON AND C. SADOSKY 01... OaH2(~)a), yields examples of analytic functions which act boundedly on H x(T 2) but are not of bounded mean oscillation on rectangles. We also give two descriptions of the pre-dual of bmo(Ta), one in terms of an atomic decomposition and the other in terms of weak factorization. The relationship between rectangular BMO and little Hankel operators is more subtle than that for little BMO and big Hankel operators. For a function, being in rectangular BMO is equivalent to the boundedness of its associated little Hankel operator applied to normalized Szeg6 kernels; however, it is not known whether this suffices to imply the boundedness of the little Hankel on all of H2(Ta). It is sufficient for the big Hankel operators (proven here for d > 1 and in [Bon] in the case d = 1), but may be insufficient for the little Hankel operators for d > 1. If it were sufficient, then it would follow that weak factorization does not hold for HI(Ta), as it does for H 1 of the unit ball in C a, d > 1 [CRW]. Weak factorization for H 1('1~ 2) was asserted in [Lin], using an argument based on the atomic decomposition of product BMO and a reduction to rectangular atoms. Unfortunately, there is a counterexample to that reduction [Bo], and the question is still open. A different approach to this problem is through the study of the different BMO spaces, as discussed in Section 3.