Kernel Theorems in Coorbit Theory

Home , Schwartz kernel theorem

TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY, SERIES B Volume 6, Pages 346–364 (November 14, 2019) https://doi.org/10.1090/btran/42

KERNEL THEOREMS IN COORBIT THEORY

PETER BALAZS, KARLHEINZ GROCHENIG,¨ AND MICHAEL SPECKBACHER

Abstract. We prove general kernel theorems for operators acting between coorbit spaces. These are Banach spaces associated to an integrable representation of a locally compact group and contain most of the usual function spaces (Besov spaces, modulation spaces, etc.). A kernel theorem describes the form of every bounded operator between a coorbit space of test functions and distributions by means of a kernel in a coorbit space associated to the tensor product representation. As special cases we recover Feichtinger’s kernel theorem for modulation spaces and the recent generalizations by Cordero and Nicola. We also obtain a kernel theorem for operators between the Besov ˙ 0 ˙ 0 spaces B1,1 and B∞,∞.

1. Introduction Kernel theorems assert that every “reasonable” operator can be written as a “generalized” integral operator. For instance, the Schwartz kernel theorem states that a continuous linear operator A : S(Rd) →S(Rd) possesses a unique distributional kernel K ∈S(R2d) such that (1) Af, g = K, g ⊗ f,f,g∈S(Rd) . If K is a locally integrable function, then Af, g = K(x, y)f(y)g(x)dydx, f, g ∈S(Rd), Rd and thus A has indeed the form of an integral operator. Similar kernel theorems hold for continuous operators from D(Rd) →D(Rd) [24, Theorem 5.2] and for Gelfand-Shilov spaces and their distribution spaces [21]. The importance of these kernel theorems stems from the fact that they oﬀer a general formalism for the description of linear operators. In the context of time-frequency analysis, Feichtinger’s kernel theorem [12] (see also [18] and [23, Theorem 14.4.1]) states that every bounded linear operator from the modulation space M 1(Rd) to the modulation space M ∞(Rd) can be represented in the form (1) with a kernel in M ∞(R2d). The advantage of this kernel theorem is

Received by the editors March 27, 2019, and, in revised form, May 23, 2019. 2010 Mathematics Subject Classiﬁcation. Primary 42B35, 42C15, 46A32, 47B34. Key words and phrases. Kernel theorems, coorbit theory, continuous frames, operator representation, tensor products, Hilbert-Schmidt operators. The ﬁrst and third authors were supported in part by the START-project FLAME (“Frames and Linear Operators for Acoustical Modeling and Parameter Estimation”, Y 551-N13) of the Austrian Science Fund (FWF). The second author was supported in part by the project P31887-N32 of the Austrian Science Fund (FWF).

c 2019 by the authors under Creative Commons Attribution-Noncommercial 3.0 License (CC BY NC 3.0) 346 KERNEL THEOREMS IN COORBIT THEORY 347 that both the space of test functions M 1(Rd) and the distribution space M ∞(Rd)= M 1(Rd)∗ are Banach spaces and thus technically easier than the locally convex spaces S(Rd)andS(Rd). Recently, Cordero and Nicola [8] revisited Feichtinger’s kernel theorem and proved several new kernel theorems that “do not have a counterpart in distribution theory”. They argue that “this reveals the superiority, in some respects, of the modulation space formalism upon distribution theory.” While we agree full- heartedly with this claim, we would like to add a more abstract point of view and argue that the deeper reason for this superiority lies in the theory of coorbit spaces and in the convenience of Schur’s test for integral operators. Indeed, we will prove kernel theorems similar to Feichtinger’s kernel theorem for many coorbit spaces. The main idea is to investigate operators in a transform domain after taking a short-time Fourier transform, a wavelet transform, or an abstract wavelet transform, i.e., a continuous transform with respect to a unitary group representation. In this new representation every operator between a suitable space of test functions and distributions is an integral operator. The standard boundedness conditions of Schur’s test then yield strong kernel theorems. The technical framework for this idea is coorbit theory, which was introduced and studied in [15–17, 22] for the construction and analysis of function spaces by means of a generalized wavelet transform. The main idea is that functions in the standard function spaces, such as Besov spaces and modulation spaces, can be characterized by the decay or integrability properties of an associated transform (the wavelet transform or the short-time Fourier transform). In the abstract set- ting, G is a locally compact group and π : G →U(H) is an irreducible, unitary, integrable representation of G. Leaving technical details aside, the coorbit space C p oπ Lw(G) consists of all distributions f in a suitable distribution space such that → p the representation coefficient g f,π(g)ψ is in the weighted space Lw(G). Next, let G1 and G2 be two locally compact groups, and let (π1, H1)and(π2, H2) be irreducible, unitary, integrable representations of G1 and G2, respectively. Let A be a bounded linear operator between Co L1 (G )andCo L∞ (G ). π1 w1 1 π2 1/w2 2 Our main insight is that such an operator can be described by a kernel in a coorbit space that is related to the tensor product representation π = π2⊗π1 of G = G1×G2 on the tensor product space H2 ⊗H1. The following non-technical formulation offers a flavor of our main result in Theorem 3: AlinearoperatorA is bounded from Co L1 (G ) to Co L∞ (G ) if and only π1 w1 1 π2 1/w2 2 ∈C ∞ × if there exists a kernel K oπ L −1⊗ −1 (G1 G2) such that w1 w2 (2) Aυ, ϕ = K, ϕ ⊗ υ ∈C 1 ∈C 1 for all υ oπ1 Lw1 (G1) ,ϕ oπ2 Lw2 (G2). This statement is not just a mere abstraction and generalization of the classical kernel theorem. With the choice of a specific group and representation one obtains explicit kernel theorems. For instance, using the Schrödinger representation of the Heisenberg group, one recovers Feichtinger’s original kernel theorem. The added value is our insight that the conditions on the kernel of [8] in terms of mixed modulation spaces [4] amount to coorbit spaces with respect to the tensor product representation. Choosing the ax + b-group and the continuous wavelet representation, one obtains a kernel theorem for all bounded operators between the Besov ˙ 0 ˙ 0 spaces B1,1 and B∞,∞ with a kernel in a space of dominating mixed smoothness. 348 P. BALAZS, K. GROCHENIG,¨ AND M. SPECKBACHER

This class of function spaces has been studied extensively [31,32] and is by no means artiﬁcial. By using suitable versions of Schur’s test, it is then possible to derive charac- terizations for the boundedness of operators between other coorbit spaces. For 1 1 example, in Theorem 7 we will prove the following, with p + q =1: ∞ (i) A:Co L1 (G ) →Co Lp (G ) bounded ⇔ K ∈Co Lp, (G ×G ), π1 w1 1 π2 w2 2 π 1/w ⊗w 1 2 ∞1 2 (ii) A:Co Lp (G ) →Co L∞ (G ) bounded ⇔ K ∈Co Lq, (G ×G ), π1 w1 1 π2 w2 2 π 1/w1⊗w2 1 2 p,q p,q where the mixed-norm Lebesgue spaces L and L on G1 × G2 are deﬁned in (23) and (24), respectively. The paper is organized as follows. In Section 2 we present the basics of tensor products and coorbit space theory. The theory of coorbit spaces of kernels with respect to products of integrable representations is developed in Section 3. Our main results, the kernel theorems, are proved in Section 4 and applied to particular examples of group representations in Section 5. We note that our proofs require a meaningful formulation of coorbit theory. One can therefore prove kernel theorems also in the context of other coorbit space theories [6, 9], e.g., for certain reducible representations.

2. Preliminaries on tensor products and coorbit spaces 2.1. Tensor products and Hilbert-Schmidt operators. The theory of tensor products is at the heart of kernel theorems for operators. Algebraically, a simple tensor of two vectors (in two possibly diﬀerent Hilbert spaces) is a formal product of two vectors f1 ⊗ f2, and the tensor product H1 ⊗H2 is obtained by taking the completion of all linear combinations of simple tensors with respect to the inner product

f1 ⊗ f2,g1 ⊗ g2 := f1,g1g2,f2 .

This tensor product is homogeneous in the following sense: α · (f1 ⊗ f2)=(αf1) ⊗ f2 = f1 ⊗(αf2). Note explicitly that the product f1 ⊗f2 is anti-linear in the second H factor. In some books this is done by introducing the dual Hilbert space 2 [25]. 2 2 2 If each Hilbert space is an L -space H1 = L (X, μ), H2 = L (Y,ν), then the simple tensor f ⊗ g is just the product (x, y) → f(x) · g(y), and the tensor product 2 2 2 becomes the product space H1 ⊗H2 = L (X, μ) ⊗ L (Y,ν)=L (X × Y,μ × ν). The connection between functions and operators arises in the analytic approach to tensor products. We interpret a function of two variablesasanintegralker- nel for an operator. Thus a simple tensor f1 ⊗ f2 of two functions becomes the rank one operator f →f,f2f1 with integral kernel f1(x)f2(y), and a general k ∈ L2(X ×Y,μ×ν) becomes a Hilbert-Schmidt operator from L2(Y,ν)toL2(X, μ). The systematic, analytic treatment of general tensor products of two Hilbert spaces often defines the tensor product as a space of Hilbert-Schmidt operators between H2 and H1. We note that his definition is already based on the characterization of Hilbert-Schmidt operators and thus represents a non-trivial kernel theorem [7]. Whereas the working mathematician habitually identifies an operator with its distributional kernel, we will make the conceptual distinction between tensor products and operators for our study of kernel theorems. In the sequel we will denote the (distributional) kernel of an integral operator by k and the abstract kernel in a tensor product by K. KERNEL THEOREMS IN COORBIT THEORY 349

2.2. Coorbit space theory. Let G be a locally compact group with left Haar H U H measure G ...dg,let be a separable Hilbert space, and let ( ) be the group of unitary operators acting on H. A continuous unitary group representation π : G → U(H)iscalledsquare integrable [1,11] if it is irreducible and there exist ψ ∈Hsuch that (3) |ψ, π(g)ψ|2dg < ∞ . G A non-zero vector ψ satisfying (3) is called admissible. For every square integrable representation there exists a densely deﬁned operator T such that ∀f1,f2 ∈ H,ψ1,ψ2 ∈ Dom (T ), one has

(4) f1,π(g)ψ1π(g)ψ2,f2dg = Tψ2,Tψ1f1,f2. G

For fixed ψ1 = ψ2 = ψ the representation coefficient f → Vψf(g):=f,π(g)ψ is interpreted as a generalized wavelet transform. The orthogonality relation (4) 2 then implies that Vψ is a multiple of an isometry from H to L (G). By using a weak interpretation of vector-valued integrals, (4) can also be recast as the inversion formula 1 (5) f = f,π(g)ψπ(g)ψdg . 2 Tψ G For the rest of this paper we assume without loss of generality that the chosen admissible vectors ψ are normalized, i.e., Tψ =1. ∗ 2 →H The adjoint operator Vψ : L (G) is formally defined by ∗ Vψ F := F (g)π(g)ψdg. G Other domains and convergence properties will be discussed later. ∗ With this notation (5) says that Vψ Vψ = IH for all admissible and normalized vectors ψ, which in the language of recent frame theory means that {π(g)ψ}g∈G is a continuous Parseval frame. By [5, Proposition 2.1] one can always assume that G is σ-finite since we assume H to be separable. In coorbit theory one needs much stronger hypotheses on π. The representation π is called integrable with respect to a weight w if there exists an admissible vector ψ ∈Hsuch that (6) |ψ, π(g)ψ|w(g) dg < ∞. G + Let g1,g2,g3 ∈ G.Wecallaweightw : G → R submultiplicative if w(g1g2) + w(g1)w(g2) and a function m : G → R w-moderate if it satisfies m(g1g2g3) p w(g1)m(g2)w(g3). If m is w-moderate, the weighted Lebesgue space Lm(G)isthen −1 invariant under left translation Lxf(y)=f(x y) and under the right translation Rxf(y)=f(yx). Throughout this paper, we will assume that the weight w satisfies (7) w(x) C max α(x),α(x−1),β(x), Δ(x−1)β(x−1) ,

p p p p where α(x):= Lx Lm(G)→Lm(G), β(x):= Rx Lm(G)→Lm(G), and Δ denotes the modular function of G. 350 P. BALAZS, K. GROCHENIG,¨ AND M. SPECKBACHER

Our standing assumption is that the representation π of G possesses an admis- 1 sible vector ψ such that Vψψ ∈ L (G). We denote the corresponding set by w A ∈H ∈ 1 w(G):= ψ ,ψ =0: Vψψ Lw(G) . For ﬁxed ψ ∈A (G) the linear version of A (G), w w H1 ∈H ∈ 1 (8) w := f : Vψf Lw(G) , H H1 ∼ H1 is dense in .Let( w) denote the anti-dual of w, i.e., the space of anti-linear H1 H1 H continuous functionals on w.As w is dense in , it follows that the inner H×H H1 ∼ ×H1 product on extends to ( w) w and so does the generalized wavelet transform. The coorbit space with respect to Lp (G) is then deﬁned by m C p ∈ H1 ∼ ∈ p oπ Lm(G):= f ( w) : Vψf Lm(G) and is equipped with the natural norm

p p f Coπ Lm(G) := Vψf Lm(G) . C p With our assumptions on π, ψ, m, the coorbit space oπ Lm(G)isaBanachspace C p ∞ 1 [16]. Alternatively, oπ Lm(G)forp< can be deﬁned as the completion of Hw with respect to this norm. Moreover, C 2 H C 1 H1 C ∞ H1 ∼ H∞ (9) oπ L (G)= , oπ Lw(G)= w, and oπ L1/w(G)=( w) = 1/w, and H1 ⊆C p ⊆H∞ (10) w oπ Lm(G) 1/w for 1 p ∞ and w-moderate weight m. In the context of coorbit space theory H1 H∞ the space w serves as a space of test functions, and 1/w is the corresponding distribution space. We quickly recall some of the fundamental properties of coorbit spaces; see for example [16, Theorems 4.1 and 4.2 and Proposition 4.3]. ∈A ∈C p ∈C q ∈ Proposition 1. Let ψ, φ w(G), f oπ Lm(G), g oπ L1/m(G),andF p Lm(G). Then the following properties hold: C p → p (i) Vψ : oπ Lm(G) Lm(G) is an isometry. Hp (ii) m is invariant with respect to π and p p p ∈ ∈C π(g)f Coπ Lm(G) w(g) f Coπ Lm(G) for all g G, f oπ Lm(G). ∗ p →C p (iii) Vψ : Lm(G) oπ Lm(G) is continuous. ∗ p (iv) Vψ Vψ = ICoπ Lm(G). ∈ p ∈C p (v) Correspondence principle:LetF Lm(G).Thereexistsf oπ Lm(G) such that F = Vψf if and only if F = F ∗Vψψ,where∗ denotes convolution on G. ∞ 1 1 C p ∗ C q (vi) Duality:For1 p< , p + q =1, we have ( oπ Lm(G)) = oπ L1/m(G), where the duality is given by

f,gC p C q = Vψf,Vψg p q . oπ Lm(G), oπ Lm(G) Lm(G),L1/m(G) C p (vii) The deﬁnition of oπ Lm(G) is independent of the particular choice of the A p p window function from w(G).Inparticular, Vψf Lm(G) Vφf Lm(G) for arbitrary non-zero φ, ψ ∈Aw(G). KERNEL THEOREMS IN COORBIT THEORY 351

We furthermore need a result on the existence of atomic decompositions for the C 1 space oπ Lw(G); see [15, Theorem 4.7].

Theorem 2. Let ψ ∈Aw(G). There exists a discrete subset {gi}i∈I ⊂ G and a collection of linear functionals λ : Co L1 (G) → C,i∈I, such that i π w (11) f = λ (f)π(g )ψ, with |λ (f)|w(g ) f C 1 , i i i i oπ Lw (G) i∈I i∈I C 1 and the sum converges absolutely in oπ Lw(G).

3. Frames and coorbit spaces via tensor products

Let G1,G2 be two locally compact groups with unitary square integrable representations π1 : G1 →U(H1)andπ2 : G2 →U(H2). For g := (g1,g2) ∈ G := G1×G2 the tensor representation π : G →U(H2 ⊗H1),

π(g):=π2(g2) ⊗ π1(g1), acts on a simple tensor Ψ := ψ2 ⊗ ψ1 ∈H2 ⊗H1 by

(12) π(g)(ψ2 ⊗ ψ1)=π2(g2)ψ2 ⊗ π1(g1)ψ1.

It follows immediately that π is a unitary representation of G on H2⊗H1.Moreover, π is irreducible (e.g., by [34, Section 4.4, Theorem 6]). Note that the order of indices is in agreement with the formulation of the kernel theorem in Theorem 3. If we interpret the simple tensor Ψ = ψ2 ⊗ ψ1 as the rank-one operator f → ψ (f)ψ with ψ ∈H , then we can write (12) as 1 2 1 1 · ⊗ π(g)(Ψ)(f)=(π1(g1)ψ1)(f) π2(g2)ψ2 = π2(g2)ψ2 π1(g1)ψ1 (f), → H where the contragredient representation π1 : G1 GL( 1)ofπ1 is deﬁned as −1 (π1(g1)ψ1)(f)=ψ1(π1(g1 )f); see [34, Section 3.1]. In case we treat the tensor product as a space of Hilbert-Schmidt operators, π acts on A ∈HS(H1, H2)as ∗ π(g)A = π2(g2)Aπ1(g1) .

The generalized wavelet transform of a simple tensor f2 ⊗ f1 with respect to a “wavelet” Ψ = ψ2 ⊗ ψ1 is given by

VΨ(f2 ⊗ f1)(g)=f2 ⊗ f1, (π2(g2) ⊗ π1(g1))(ψ2 ⊗ ψ1)

(13) = f2,π2(g2)ψ2f1,π1(g1)ψ2

= Vψ2 f2(g2)Vψ1 f1(g1). Thus, the wavelet transform of the tensor product representation factors into the product of wavelet transforms on G1 and G2. Strictly speaking, we would have to write V πi f to indicate the underlying representation, but we omit the reference to ψi i the group to keep notation simple. Throughout this paper we consider only separable weights w : G → R+ with w(g)=(w1 ⊗ w2)(g)=w1(g1)w2(g2)andm(g)=(m1 ⊗ m2)(g)=m1(g1)m2(g2), where wi is submultiplicative and mi is wi-moderate. Moreover we write (1/w)(g)= −1 (w1 ⊗ w2)(g) . It follows from (13) that the tensor representation π2 ⊗ π1 of two square-integrable representations is again square-integrable and that the tensor Ψ=ψ2 ⊗ ψ1 of two admissible vectors ψ2 and ψ1 is admissible for π. Likewise, if ⊗ ∈A ∈A ⊗ ∈A × w = w1 w2 and ψ1 w1 (G1), ψ2 w2 (G2), then ψ2 ψ1 w(G1 G2) 352 P. BALAZS, K. GROCHENIG,¨ AND M. SPECKBACHER

(where we assume that wi, i =1, 2, satisﬁes (7)). Therefore all deﬁnitions and results of Section 2.2 hold for the representation π = π2 ⊗ π1 and Ψ = ψ2 ⊗ ψ1 .In particular, the orthogonality relation (4), the inversion formula (5), Proposition 1, and Theorem 2 hold for suitable admissible vectors Ψ = ψ2 ⊗ ψ1.

4. Kernel theorems In this section we derive the general kernel theorems for operators between coorbit spaces. The basic idea comes from linear algebra, where a linear operator is identiﬁed with its matrix with respect to a basis. In coorbit theory the basic struc- ture consists of the vectors π(g)ψ. Thus in analogy to linear algebra we try to describe an operator A : H1 →H2 by the kernel (= continuous matrix)

(14) kA(g1,g2)=Aπ1(g1)ψ1,π2(g2)ψ2.

This can be seen as a continuous Galerkin-like representation of the operator A [2, 3]. The idea goes back to coherent state theory [30, Ch. 1.6]. One of its goals is to associate to every operator A a function or symbol kA, and (14) is one of the many possibilities to do so. Assume that A : Co L1 (G ) →Co L∞ (G )andf ∈Co L1 (G ) , i.e., A π1 w1 1 π2 1/w2 2 π1 w1 1 maps “test functions” to “distributions”. By using the inversion formula (5) for f and applying A to it, it follows formally that

Af = f,π1(g1)ψ1Aπ1(g1)ψ1dg1, G1 and furthermore that Vψ2 (Af)(g2)= Af, π2(g2)ψ2 = f,π1(g1)ψ1 Aπ1(g1)ψ1,π2(g2)ψ2 dg1 G1

(15) = f,π1(g1)ψ1kA(g1,g2)dg1 . G Let

(16) AF (g2)= F (g1)kA(g1,g2)dg1 G1 be the integral operator with the kernel kA. Then (15) can be written as

(17) Vψ2 Af = AVψ1 f or, equivalently,

∗ (18) A = Vψ2 AVψ1 . Using this factorization, the computation in (15) can be given a precise meaning on coorbit spaces. Identity (18) is the heart of the kernel theorems. The combi- nation of the properties of the generalized wavelet transform (Proposition 1) and boundedness properties of integral operators yields powerful and very general kernel theorems. KERNEL THEOREMS IN COORBIT THEORY 353

We will ﬁrst show the existence of a generalized kernel for operators mapping the space of test functions Co L1 (G ) into the distribution space Co L∞ (G ). π1 w1 1 π2 1/w2 2 Subsequently, we will characterize continuous operators in certain subclasses.

Theorem 3. Let G1 and G2 be two locally compact groups, and let (πj, Hj ) be A ∅ integrable, unitary, irreducible representations of Gj, such that wj (Gj) = for j =1, 2. ∈C ∞ × (i) Every kernel K oπ L1/w(G1 G2) deﬁnes a unique linear operator A : Co L1 (G ) →Co L∞ (G ) by means of π1 w1 1 π2 1/w2 2

(19) Aυ, ϕ = K, ϕ ⊗ υ

∈C 1 ∈C 1 for all υ oπ1 Lw1 (G1) and ϕ oπ2 Lw2 (G2) . The operator norm satis- ﬁes

(20) A Op K C ∞ oπ L1/w(G)

and

(21) kA = VΨK.

(ii) Kernel theorem: Conversely, if A : Co L1 (G ) →Co L∞ (G ) is π1 w1 1 π2 1/w2 2 ∈C ∞ × bounded, then there exists a unique kernel K oπ L1/w(G1 G2) such that (19) holds.

Proof. (i) Fix K ∈Co L∞ (G) with G = G × G , and let υ ∈Co L1 (G ), ϕ ∈ π 1/w 1 2 π1 w1 1 C 1 ⊗ ∈C 1 oπ2 Lw2 (G2) be arbitrary. By (13) it follows that ϕ υ oπ Lw(G). Therefore, the duality in (19) is well-deﬁned and

|K, ϕ ⊗ υ| K C ∞ ϕ ⊗ υ C 1 oπ L1/w(G) oπ Lw(G)

(22) = K Co L∞ (G) ϕ Co L1 (G ) υ Co L1 (G ) . π 1/w π2 w2 2 π1 w1 1

Therefore, if we ﬁx υ, the mapping ϕ →K, ϕ ⊗ υ is a bounded, anti-linear functional on Co L1 (G ) , which we call Aυ ∈Co L∞ (G ). Themap υ → Aυ is π2 w2 2 π2 1/w2 2 clearly linear, and (19) deﬁnes a linear operator A : Co L1 (G ) →Co L∞ (G ). π1 w1 1 π2 1/w2 2 The estimate (22) implies that

∞ ∞ 1 Av Coπ L (G2) K Coπ L (G) v Coπ L (G1) , 2 1/w2 1/w 1 w1 and thus

A Op K C ∞ . oπ L1/w(G) (ii) To prove the converse, we need to show that the mapping K → A is one-to- one and onto. K∈C ∞ Uniqueness: Let us assume that the kernel oπ L1/w(G) also satisﬁes

Aυ, ϕ = K, ϕ ⊗ υ = K,ϕ⊗ υ 354 P. BALAZS, K. GROCHENIG,¨ AND M. SPECKBACHER

∈C 1 ∈C 1 for every υ oπ1 Lw1 (G1), ϕ oπ2 Lw2 (G2) . By Theorem 2, there exists a { } ⊂ ∈C 1 discrete set γi i∈I G such that every F oπ Lw(G) can be written as F = λi(F )π(γi) ψ2 ⊗ ψ1 , i∈I

C 1 with unconditional convergence in oπ Lw(G)and |λ (F )|w(γ ) C F C 1 × . i i oπ Lw(G1 G2) i Since π(γi) ψ2 ⊗ ψ1 = π(γi,2)ψ2 ⊗ π(γi,1)ψ1, we conclude that K, F = λi(F )K, π(γi,2)ψ2 ⊗ π(γi,1)ψ1 ∈I i = λi(F )K,π(γi,2)ψ2 ⊗ π(γi,1)ψ1 i∈I = K,F.

∈C 1 K As this equality holds for every F oπ Lw(G), it follows that K = . Surjectivity: Let us assume that A : Co L1 (G ) →Co L∞ (G ) is bounded. π1 w1 1 π2 1/w2 2 ∞ × Then the kernel kA deﬁned in (14) is an element of L1/w(G1 G2), because

|kA(g)| = |Aπ1(g1)φ, π2(g2)ψ|

1 1 A Op π1(g1)φ Coπ L (G1) π2(g2)ψ Coπ L (G2) 1 w1 2 w2

1 1 A Op w1(g1) φ Coπ L (G1) w2(g2) ψ Coπ L (G2) . 1 w1 2 w2

We claim that kA is a generalized wavelet transform. Precisely, there exists K ∈ C ∞ × oπ L1/w(G1 G2) such that kA = VΨK. To prove this claim, we use Proposi- ∈C ∞ tion 1(v), which asserts that kA = VψK for some K oπ L1/w(G) if and only if kA = kA ∗ VΨΨ. 1 As kA · VΨ(π(g)Ψ) ∈ L (G1 × G2), we may choose the most convenient order of integration and apply the reproducing formula of Proposition 1(v) consecutively to the representations π1 and π2. Using (13) we obtain −1 (kA ∗ VΨΨ)(g)= kA(h)VΨΨ(h g) dh G −1 −1 = Vψ2 Aπ1(h1)ψ1 (h2)Vψ2 ψ2(h2 g2)dh2 Vψ1 ψ1(h1 g1)dh1 G1 G2 ∗ −1 = Vψ2 Aπ1(h1)ψ1 Vψ2 ψ2 (g2) Vψ1 ψ1(h1 g1)dh1 G1 −1 ∗ = Aπ1(h1)ψ1,π2(g2)ψ2 Vψ1 ψ1(h1 g1)dh1 =( ) . G1 At this point we note that by the assumption on A there exists a unique operator A : Co L1 (G ) →Co L∞ (G ) that satisﬁes π2 w2 2 π1 1/w1 1

Aυ, ϕ = υ, A ϕ KERNEL THEOREMS IN COORBIT THEORY 355

∈C 1 ∈C 1 ∗ for every υ oπ1 Lw1 (G1)andϕ oπ2 Lw2 (G2) . By its deﬁnition, A is weak - continuous. We continue with the integration over G1 and obtain ∗ −1 ( )= A π2(g2)ψ2,π1(h1)ψ1 Vψ1 ψ1(h1 g1)dh1 G1 ∗ = Vψ1 A π2(g2)ψ2 Vψ1 ψ1 (g1)= A π2(g2)ψ2,π1(g1)ψ1

= Aπ1(g1)ψ1,π2(g2)ψ2 = kA(g). ∈C ∞ × By Proposition 1(v) there exists a kernel K oπ L1/w(G1 G2) such that kA(g1,g2)=VΨK(g1,g2). By the ﬁrst part of the proof K deﬁnes an operator B : Co L1 (G ) →Co L∞ (G )bymeansofBv,φ = K, φ ⊗ v.Inparticu- π1 w1 1 π2 1/w2 2 lar,

Bπ1(g1)ψ1,π2(g2)ψ2 = K, π2(g2)ψ2 ⊗ π1(g1)ψ1 = VΨK(g1,g2)

= kA(g1,g2)=Aπ1(g1)ψ1,π2(g2)ψ2 .

Consequently, Bπ1(g1)ψ1 = Aπ1(g1)ψ1 for all g1 ∈ G1. This identity extends to C 1 all ﬁnite linear combinations of vectors π1(g1)ψ1 and by Theorem 2 to oπ1 Lw1 (G1). Thus B = A, and we have shown that the map from kernels to operators is onto. The map K → A is bounded and invertible. By the inverse mapping theorem we obtain that K C ∞ C A Op, which proves (20). oπ L1/w(G)

Remark 4. It is crucial to interpret the brackets in (19) correctly. For utmost precision, we would have to write

∞ 1 ⊗ ∞ 1 Aυ, ϕ Coπ L (G2), Coπ L (G2) = K, ϕ υ Coπ L (G), Coπ L (G) , 2 1/w2 2 w2 1/w w but we feel that this notation would distract from the analogy to distribution theory.

The injectivity of the mapping K → A from kernels to operators is closely C 1 related to an important property of the coorbit spaces oπ1 Lw1 (G1) . This so- called tensor product property has gained considerable importance in certain special cases [13, Theorem 7D] and [26]. We therefore state and prove a general version. Recall that the projective tensor product of two Banach spaces B1 and B2 is deﬁned to be ⊗ { ⊗ ∈ ∈ ∞} B1 B2 = f = φi ψi : φi B1,ψi B2 and φi B1 ψi B2 < . i∈I i∈I

The norm is given as f ⊗ =inf i∈I φi B1 ψi B2 over all representations of ⊗ f = i∈I φi ψi. The following identiﬁcation of the projective tensor product of Co L1 (G ) π1 w1 1 C 1 C 1 × and oπ2 Lw2 (G2) with the coorbit space oπ Lw(G1 G2) is a generalization of Feichtinger’s original result for modulation spaces [13, Theorem 7D].

Theorem 5. Under the general assumptions on the groups Gi and the representations (πi, Hi) we have C 1 × C 1 ⊗C 1 oπ Lw(G1 G2)= oπ2 Lw2 (G2) oπ1 Lw1 (G1) . 356 P. BALAZS, K. GROCHENIG,¨ AND M. SPECKBACHER

∈C 1 ⊗ Proof. Let F oπ Lw(G ). Then by Theorem 2 applied to π = π2 π1, F possesses ∈C 1 ∈ the representation F = i∈I λi(F )π(γi)Ψ oπ Lw(G) with γi =(γi,1,γi,2) G × G and |λ (F )|w(γ ) C F C 1 . Using Proposition 1(ii) we 1 2 i∈I i i oπ Lw (G) obtain that

1 1 λi(F )π1(γi,1)ψ1 Coπ L (G1) π2(γi,2)ψ2 Coπ L (G2) 1 w1 2 w2 ∈I i |λi(F )|w1(γi,1)w2(γi,2) ψ1 Co L1 (G ) ψ2 Co L1 (G ) π1 w1 1 π2 w2 2 i∈I

C F C 1 . oπ Lw (G) ∈C 1 ⊗C 1 C 1 Thus F oπ2 Lw2 (G2) oπ1 Lw1 (G1), and oπ Lw(G) is continuously embedded C 1 ⊗C 1 into oπ2 Lw2 (G2) oπ1 Lw1 (G1). Conversely, let F ∈Co L1 (G ) ⊗C o L1 (G ) . Choose a representation π2 w2 2 π1 w1 1 F = f ⊗ f with f C 1 f C 1 < ∞. Using Fubini’s i,2 i,1 i,1 oπ1 Lw (G1) i,2 oπ2 Lw (G2) i∈I i∈I 1 2 theorem and Proposition 1(ii) yields

F C 1 = |V F (g)| w(g)dg oπ Lw(G) Ψ G | | · | | Vψ1 fi,1(g1) w1(g1)dg1 Vψ2 fi,2(g2) w2(g2)dg2 ∈I G1 G2 i = fi,1 Co L1 (G ) fi,2 Co L1 (G ) < ∞. π1 w1 1 π2 w2 2 i∈I C 1 ⊗C 1 ⊆C 1 Thus, oπ2 Lw2 (G2) oπ1 Lw1 (G1) oπ Lw(G). The equivalence of the norms follows from the inverse mapping theorem.

Once the kernel theorem provides a general description of operators between test functions and distributions, we may try to characterize certain classes of operators by properties of their kernel. Since on the level of the generalized wavelet transform such operators correspond to integral operators (see diagram in Figure 1), we may translate the various versions of Schur’s test to kernel theorems for operators between coorbit spaces. Following the procedure in [8, Theorem 3.3], we first formulate a general version of Schur’s test and then derive the abstract kernel theorem. We introduce two classes of mixed norm spaces. For two σ-finite measure spaces (X, μ)and(Y,ν), 1 p ∞,andm : X × Y → R+, we define the spaces p,∞ × Lp,∞ × Lm (X Y )and m (X Y ) by the norms

1/p p p p,∞ | | (23) F Lm (X×Y ) := ess sup F (x, y) m(x, y) dμ(x) y∈Y X and 1/p p p p,∞ | | (24) F Lm (X×Y ) := ess sup F (x, y) m(x, y) dν(y) . x∈X Y The following version of Schur’s test is folklore and can be found in [33, Propo- sitions 5.2 and 5.4] or [27]. KERNEL THEOREMS IN COORBIT THEORY 357

Proposition 6. Let (X, μ) and (Y,ν) be σ-ﬁnite measure spaces, let 1 p ∞, 1 1 let p + q =1,andletT be the integral operator Tf(y)= X f(x)kT (x, y)dμ(x) with kernel kT : X × Y → C. 1 p ∈ (i) The operator T is bounded from Lm1 (X) to Lm2 (Y ) if and only if kT Lp,∞ × −1⊗ (X Y ). In that case m1 m2

1 p p,∞ (25) T L (X)→Lm (Y ) = kT L − (X×Y ) . m1 2 1⊗ m1 m2 p ∞ ∈ (ii) The operator T is bounded from Lm1 (X) to Lm2 (Y ) if and only if kT q,∞ × L −1⊗ (X Y ). In this case m1 m2

p ∞ q,∞ (26) T Lm (X)→L (Y ) = kT L − (X×Y ). 1 m2 1⊗ m1 m2 We now characterize the boundedness of operators between certain coorbit spaces.

∞ 1 1 Theorem 7. Let 1 p, q with p + q =1,andletmj be wj -moderate weights on G .IfA is a bounded operator from Co L1 (G ) to Co L∞ (G ) with kernel j π1 w1 1 π2 1/w2 2 K, then the following hold: C 1 C p (i) A is bounded from oπ1 Lm1 (G1) to oπ2 Lm2 (G2) if and only if its kernel C Lp,∞ × K is in oπ −1⊗ (G1 G2). Its operator norm satisﬁes m1 m2

p,∞ A Op K Coπ L − (G) . 1⊗ m1 m2 C p C ∞ (ii) A is bounded from oπ1 Lm1 (G1) to oπ2 Lm2 (G2) if and only if its kernel C q,∞ × K is in oπ L −1× (G1 G2). Its operator norm satisﬁes m1 m2

q,∞ A Op K Coπ L − (G) . 1⊗ m1 m2 Proof. Since Co L1 (G ) ⊆Co L1 (G )andCo Lp (G ) ⊆Co L∞ (G ) π1 w1 1 π1 m1 1 π2 m2 2 π2 1/w2 2 by (10), the kernel theorem is applicable to the operator A,andthereexistsa ∈C ∞ ⊗ kernel K oπ L1/w(G1 G2) such that

VΨK(g1,g2)=kA(g1,g2)=Aπ1(g1)ψ1,π2(g2)ψ2. ∈C Lp,∞ ∈Lp,∞ Assume ﬁrst that K oπ −1⊗ (G), which means that VΨK 1/m ⊗m (G). m1 m2 1 2 By Proposition 6, the integral operator A deﬁned by the integral kernel kA is bounded from L1 (G )toLp (G ). According to (18), A factors as A = V ∗ AV , m1 1 m2 2 ψ2 ψ1 where V is an isometry from Co L1 (G )toL1 (G ), and V ∗ is bounded ψ1 π1 m1 1 m1 1 ψ2 p C p from Lm2 (G2)to oπ2 Lm2 (G2) by Proposition 1. Consequently A is bounded from C 1 C p oπ1 Lm1 (G1)to oπ2 Lm2 (G2) . The boundedness estimate follows from

∗ p A Op Vψ Op A L1 (G )→L (G ) Vψ1 Op 2 m1 1 m2 2 p,∞ p,∞ C kA L − (G) = C K Coπ L − (G). 1⊗ 1⊗ m1 m2 m1 m2 358 P. BALAZS, K. GROCHENIG,¨ AND M. SPECKBACHER

C 1 C p Conversely, let A be bounded from oπ1 Lm1 (G1)to oπ2 Lm2 (G2). Then Aπ (g )ψ ∈Co Lp (G ) and the following estimates make sense: 1 1 1 π2 m2 2

p,∞ p,∞ p,∞ K Coπ L − (G) = VΨK L − (G) = kA L − (G) 1⊗ 1⊗ 1⊗ m1 m2 m1 m2 m1 m2 1/p p −1 =sup |Aπ1(g1)ψ1,π2(g2)ψ2m2(g2)| dg2 m1(g1) ∈ g1 G1 G2 −1 =sup Aπ1(g1)ψ1 C p m1(g1) oπ2 Lm2 (G2) g1∈G1 −1 A Op sup π1(g1)ψ1 Co L1 (G ) m1(g1) . π1 m1 1 g1∈G1

∈ 1 −1 Since Vψ1 ψ1 Lw1 (G1)andm1 is w1-moderate and thus satisﬁes m1(g1h)m1(g1) w1(h), the last expression is bounded by −1 −1 m1(h) 1 · | | sup π1(g1)ψ1 Coπ L (G1) m1(g1) =sup ψ1,π1(g1 h)ψ1 dh ∈ 1 m1 ∈ m (g ) g1 G1 g1 G1 G1 1 1 m1(g1h) | 1 =sup ψ1,π1(h)ψ1 dh Vψ1 ψ1 L (G1). ∈ w1 g1 G1 G1 m1(g1)

∈C Lp,∞ Thus K oπ −1⊗ (G). m1 m2 Part (ii) follows by using Proposition 6(ii) instead of (i) and is proved similarly.

The following diagram (Figure 1) shows the connection between the diﬀerent operators and spaces.

C 1 C p oπ1 Lm (G1) oπ2 Lm (G2) 1 A bounded 2

p,∞ Vψ ∈C L Vψ 1 K oπ −1⊗ (G) 2 m1 m2

VΨ

p,∞ k ∈L − (G) A m 1⊗m 1 1 2 p Lm (G1) Lm (G2) 1 A bounded 2

Figure 1

Using interpolation between Lp-spaces, Schur’s test can also be formulated as saying that an integral operator is bounded on all Lp simultaneously if and only if its kernel belongs to L1,∞ ∩L1,∞. The corresponding version for coorbit spaces is a consequence of Theorem 7 and an interpolation argument. KERNEL THEOREMS IN COORBIT THEORY 359

Corollary 8. The following conditions are equivalent: C p →C p ∞ (i) A : oπ1 Lm1 (G1) oπ2 Lm2 (G2) is bounded for every 1 p . C 1 →C 1 C ∞ → (ii) Both A : oπ1 Lm1 (G1) oπ2 Lm2 (G2) and A : oπ1 Lm1 (G1) Co L∞ (G ) are bounded. π2 m2 2 ∈C L1,∞ C 1,∞ (iii) K oπ −1⊗ (G) oπ L −1⊗ (G). m1 m2 m1 m2 Clearly one can now translate every boundedness result for an integral operator into a kernel theorem for coorbit spaces. As a simple but important example we offer a sufficient condition for regularizing operators, i.e., operators that map distributions to test functions. Theorem 9. Under the assumptions of Theorem 3, if the unique kernel of the operator A satisfies K ∈Co L1 (G),thenA is bounded from Co L∞ (G ) to π w π1 1/w1 1 C 1 oπ2 Lw2 (G2). Proof. Consider the integral operator A as in the proof of Theorem 7 and observe that V K = k ∈ L1 (G)isasufficientconditionforA : L∞ (G ) → L1 (G )to Ψ A w 1/w1 1 w2 2 be bounded by Schur’s test. 4.1. Discretization. Coorbit theory guarantees the discretization of the coorbit spaces via atomic decompositions and Banach frames. For our purposes, it is sufficient to state a shortened and simplified version of [22, Theorem 5.3]. Let Y be p p,∞ Lp,∞ one of the function spaces Lm(G),Lm (G), or m (G), and let Yd be the natural sequence space associated to Y . Proposition 10. If ψ satisfies

(27) sup |Vψψ(h)|w(g)dg < ∞, G h∈gQ for a compact neighborhood Q of e, then there exist a discrete subset Λ ⊂ G and constants C1,C2 > 0 such that ∈C (28) C1 f Coπ Y Vψf Yd C2 f Coπ Y for every f oπ Y.

Corollary 11. Let Λ=Λ1 × Λ2 ⊂ G be a discrete set such that {π(λ)Ψ}λ∈Λ C p,∞ C Lp,∞ satisﬁes (28) for oπ L −1⊗ (G) and oπ −1⊗ (G).IfA is a bounded operator m1 m2 m1 m2 from Co L1 (G ) to Co L∞ (G ) with kernel K, then the following hold: π1 w1 1 π2 1/w2 2 C 1 →C p (i) A : oπ1 Lm (G1) oπ2 Lm (G2) is bounded if and only if 1 2 1/p | −1 ⊗ |p ∞ (29) sup VΨK(λ)(m1 m2)(λ) < . λ1∈Λ1 λ2∈Λ2 C p →C ∞ (ii) Likewise A : oπ1 Lm (G1) oπ2 Lm (G2) is bounded if and only if 1 2 1/q | −1 ⊗ |q ∞ (30) sup VΨK(λ)(m1 m2)(λ) < . λ2∈Λ2 λ1∈Λ1 p,∞ Proof. (i) By Theorem 7 A has a kernel in Coπ L (G), and

p,∞ A Op K Coπ L (G).

p,∞ By (28), the expression in (29) is an equivalent norm for K Coπ L (G). The proof of (ii) works in exactly the same way. 360 P. BALAZS, K. GROCHENIG,¨ AND M. SPECKBACHER

5. Examples d d 5.1. Modulation spaces. The Weyl-Heisenberg group GWH = R × R × T is deﬁned by the group law

(x, ω, e2πiτ ) · (x,ω,e2πiτ )=(x + x,ω+ ω,e2πi(τ+τ −x·ω )).

2πiωt Let Txf(t):=f(t − x) denote the translation, and let Mωf(t):=e f(t)bethe 2πiτ modulation operator. The operator πWH(x, ω, τ)=e MωTx for (x, ω, τ) ∈ GWH 2 d deﬁnes a unitary square-integrable representation of GWH acting on L (R ), for which every non-zero vector in L2(Rd) is admissible. Since the phase factor e2πiτ is irrelevant for the deﬁnition of coorbit spaces, it is convenient to drop the trivial third component and consider the time-frequency shift π(x, ω)=πWH(x, ω, 1) = MωTx. Formally, we treat the projective representation π of R2d instead of the unitary representation πWH of GWH. The transform corresponding to π is nothing else but the short-time Fourier transform −2πiω·t 2 d Vψf(x, ω)=f,MωTxψ = f(t)ψ(t − x)e dt, f, ψ ∈ L (R ). Rd

The coorbit spaces associated to πWH coincide therefore with the coorbit spaces p Rd associated to π. These are the modulation spaces Mm( ) which were ﬁrst introduced by Feichtinger in [14] as certain decomposition spaces and subsequently C p were identiﬁed with the coorbit spaces of the Heisenberg group oπWHLm(GWH)= C p R2d p Rd oπLm( )=Mm( ) [18]. We refer to the standard textbooks [20, 23] for more information on time-frequency analysis. 1 Rd C 1 Theorem 3 asserts that every bounded operator from Mw( )= oπWHLw(GWH) C 1 R2d ∞ Rd C ∞ C ∞ R2d = oπ Lw( )toM1/w( )= oπWHL1/w(GWH)= oπ L1/w( ) possesses a ∞ ∈C ⊗ − − × ⊗ kernel K oπWH πWHLw 1⊗w 1 (GWH GWH) such that Af, g = K, g f ∈ 1 Rd for f,g Mw( ). Let us elaborate in detail what the kernel theorem asserts in this case: for gi =(xi,ωi,τi) ∈ GWH,i =1, 2, the tensor representation πWH ⊗ πWH 2 d 2 d ∼ acts on the simple tensor (ψ2 ⊗ ψ1)(t2,t1)=ψ2(t2)ψ1(t1) ∈ L (R ) ⊗ L (R ) = L2(R2d)as

− ⊗ ⊗ 2πi(τ1 τ2) πWH πWH(g2,g1)(ψ2 ψ1)(t2,t1)=e Mω2 Tx2 ψ2(t2)Mω1 Tx1 ψ1(t1) − 2πi(τ1 τ2) ⊗ = e M(ω2,−ω1)T(x2,x1)(ψ2 ψ1)(t2,t1) .

2πi(τ −τ ) Thus except for the phase factor e 1 2 the tensor representation πWH ⊗ πWH 2 R2d is just the time-frequency shift M(ω2,−ω1)T(x2,x1) acting on L ( ). Consequently, the coorbit spaces with respect to the product group GWH ⊗ GWH are again modulation spaces, this time on R2d. For the coorbit of L∞ we compare the norms ∞ 2d ⊗ K M (R ) =sup K, M(ω1,ω2)T(x1,x2)(ψ2 ψ1) 4d (x1,x2,ω1,ω2)∈R and ∞ 4d ⊗ ⊗ K Coπ⊗π L (R ) =sup K, π(x1,ω1) π(x2,ω2) (ψ2 ψ1) , 4d (x1,ω1),(x2,ω2) ∈R which are obviously equal. In this case Theorem 3 is therefore just Feichtinger’s kernel theorem: For A : M 1(Rd) → M ∞(Rd) there exists a unique kernel K ∈ M ∞(R2d) such that Af, g = K, g ⊗ f. KERNEL THEOREMS IN COORBIT THEORY 361

The recent extension of Feichtinger’s kernel theorem by Cordero and Nicola [8] can be seen in the same light. Let us explain the diﬀerence in the formulations. Our approach considers the generalized wavelet transform

VΨK(x1,ω1,x2,ω2)=K, πWH(x2,ω2, 1) ⊗ πWH(x1,ω1, 1)(ψ2 ⊗ ψ1) of the kernel. The conditions of Theorem 7 are formulated by mixed norms acting simultaneously on the variables (x2,ω2)and(x1,ω1). The treatment in [8] uses the short-time Fourier transform on R2d VΨK(x1,x2,ω1,ω2)= K, M(ω1,ω2)T(x1,x2)Ψ , which is the same transform, except for the order of the variables. In [8] it was therefore necessary to reshuﬄe the order of integration of time-frequency shifts and to use the notion of mixed modulation spaces, which were studied in [4, 29]. The new insight of our formulation is that the mixed modulation spaces are simply the coorbit spaces with respect to the tensor product representation. The special case of Theorem 7 for the Weyl-Heisenberg group and the weights s ms(x, ω, τ)=(1+|x| + |ω|) for s ∈ R states the following: Fix σ>0 and let A be 1 Rd ∞ Rd | | | | ∞ an operator from Mmσ ( )toMm−σ ( ). Then for r , s σ,1 p, q ,and 1/p +1/q =1wehave 1 Rd → p Rd ⇔ ∈C Lp,∞ R4d (i) A : Mm ( ) Mm ( ) bounded K oπ m− ⊗w ( ), s r ∞s r (ii) A : M p (Rd) → M ∞ (Rd) bounded ⇔ K ∈Co Lq, (R4d). ms mr π m−s⊗mr Regularizing operators from M ∞ to M 1 were recently studied by Feichtinger and Jakobsen [19]: they characterized a subclass of this space of operators by an integral kernel in M 1(R2d). The suﬃciency of this result in a coorbit version is contained in Theorem 9.

∗ 5.2. Wavelet coorbit spaces and Besov spaces. The affine group Gaff = R×R is given by the group law (x, a)·(y, b)=(x+ay, ab), where x, y ∈ R and a, b ∈ R\{0}. dxda | |−1/2 Its left Haar measure is given by a2 .LetDaf(t)= a f(t/a)denotethe dilation operator. Then (x, a) → πaff (x, a)=TxDa defines a unitary, square- 2 integrable representation of Gaff on L (R). Now let f,ψ ∈ L2(R). The continuous wavelet transform is defined as −1/2 −1 Wψf(x, a):=f,πaff (x, a)ψ = |a| f(t)ψ(a (t − x))dt, R and the admissibility condition (3) reads as dω |ψ (ω)|2 < ∞. R∗ |ω|

It is well-known that the coorbit spaces associated to the representation πaff are the homogeneous Besov spaces. See the textbooks [10, 28] for details and further expositions of wavelet theory. For brevity, we consider only the coor- p bit spaces with respect to the weighted L (Gaff )-spaces with the weight function −s p | | ∈ R − C νs(x, a)=νs(a)= a for s .Notethatν s =1/νs.Then oπaff Lνs (Gaff )= ˙ s−1/2+1/p R C 1 ˙ s+1/2 R Bp,p ( ) by [15, Section 7.2]. In particular oπaff Lνs (Ga)=B1,1 ( )and ∞ s−1/2 C ˙ ∞ ∞ R oπaff Lνs (Gaff )=B , ( ). In this example Theorem 3 states that an opera- ˙ s R → ˙ −r R tor A : B1,1( ) B∞,∞( ) is bounded if and only if its associated kernel K is ∞ 2 in Co ⊗ L (G ). At first glance not much seems to have been πaff πaff ν−s−1/2⊗ν−r−1/2 aff 362 P. BALAZS, K. GROCHENIG,¨ AND M. SPECKBACHER gained by this formulation, but it turns out that the coorbit spaces of the tensor ⊗ 2 product πaff πaff of Gaff are well understood in the theory of function spaces under the name of Besov spaces of dominating mixed smoothness. In particular, ∞ 2 Co ⊗ L (G ) can be identified with the Besov space of dominat- πaff πaff ν−s−1/2⊗ν−r−1/2 aff −s,−r R2 ing mixed smoothness S∞,∞ B( ). See [32, Definition A.4] and [31]. Moreover, Theorem 7 yields a characterization of continuous operators between certain Besov spaces: s r p,∞ 2 (i) A:B˙ (R)→B˙ (R) bounded ⇔ K ∈Co ⊗ L (G ), 1,1 p,p πaff πaff ν−s+1/2⊗νr+1/2−1/p aff s r q,∞ 2 (ii) A:B˙ (R)→B˙ (R) bounded ⇔ K ∈Co ⊗ L (G ). p,p ∞,∞ πaff πaff ν−s−1/2+1/p⊗νr+1/2 aff Thecase(i)forp = 1 was already formulated in a discrete version by Meyer [28, Section 6.9, Proposition 6].

j/2 j Theorem 12. Let {ψk,j}(k,j)∈Z2 be a wavelet basis with ψk,j(t)=2 ψ(2 t − k), and assume that ψ has compact support and satisfies sufficiently many moment conditions so that the assumption of Proposition 10 is satisfied. An operator A : ˙ 0 R → ˙ 0 R B1,1( ) B1,1( ) is bounded if and only if −j/2+j/2 sup Aψk,j,ψk,j 2 C. (k,j)∈Z2 (k,j)∈Z2

Proof. Set p =1,s = −1/2, recall that kA = VΨK, and apply Corollary 11. 5.3. The case of two distinct representations. For most applications it suffices to consider a single group G and its product group G × G. Our formulation with two different groups allows us to study operators acting between coorbit spaces associated with different group representations. Using the representations of the Weyl-Heisenberg group and the affine group of Sections 5.1 and 5.2, one can characterize the boundedness of operators between certain modulation spaces and Besov spaces by properties of their associated kernels. Theorem 7 now reads as follows: 1 Rd → ˙ r R (i) A : Mms ( ) Bp,p( ) bdd. p,∞ ⇔ K ∈Coπ ⊗π L (GWH × Gaff ), aff WH m −s⊗νr+1/2−1/p p Rd → ˙ r R (ii) A : Mms ( ) B∞,∞( ) bdd. q,∞ ⇔ K ∈Co ⊗ L (G × G ), πaff πWH m−s⊗νr+1/2 WH aff ˙ r R → p Rd (iii) A : B1,1( ) Mms ( ) bdd. p,∞ ⇔ K ∈Co ⊗ L (G × G ), πWH πaff m−r+1/2⊗νs a WH ˙ r R → ∞ Rd (iv) A : Bp,p( ) Mms ( ) bdd. q,∞ ⇔ K ∈Co ⊗ L (G × G ). πWH πaff ν−r−1/2+1/p⊗ms a WH As a special case one obtains a characterization of the bounded operators A from ˙ r R 2 Rd 2 Rd 2 Rd B1,1( )toL ( ). Since M ( )=L ( ), they are completely characterized by 2,∞ the membership of their kernel in Co ⊗ L (G × G ). πaff πWH 1⊗m−r+1/2 aff WH References

[1]S.T.Ali,J.-P.Antoine,andJ.-P.Gazeau,Square integrability of group representations on homogeneous spaces. I. Reproducing triples and frames (English, with French summary), Ann. Inst. H. Poincar´ePhys.Th´eor. 55 (1991), no. 4, 829–855. MR1144104 KERNEL THEOREMS IN COORBIT THEORY 363

[2] Peter Balazs, Matrix representation of operators using frames, Sampl. Theory Signal Image Process. 7 (2008), no. 1, 39–54. MR2455829 [3] P. Balazs and K. Gröchenig. A guide to localized frames and applications to Galerkin-like representations of operators. In I. Pesenson, H. Mhaskar, A. Mayeli, Q. T. L. Gia, and D.- X. Zhou, editors, Frames and Other Bases in Abstract and Function Spaces, Applied and Numerical Harmonic Analysis series (ANHA). Birkhauser/Springer, 2017. MR3700112 [4] Shannon Bishop, Mixed modulation spaces and their application to pseudodifferential operators, J. Math. Anal. Appl. 363 (2010), no. 1, 255–264, DOI 10.1016/j.jmaa.2009.08.032. MR2559061 [5] Marcin Bownik, Lyapunov’s theorem for continuous frames, Proc. Amer. Math. Soc. 146 (2018), no. 9, 3825–3838, DOI 10.1090/proc/14088. MR3825837 [6] Jens Gerlach Christensen and Gestur Olafsson,´ Coorbit spaces for dual pairs, Appl. Comput. Harmon. Anal. 31 (2011), no. 2, 303–324, DOI 10.1016/j.acha.2011.01.004. MR2806486 [7] John B. Conway, A course in functional analysis, 2nd ed., Graduate Texts in Mathematics, vol. 96, Springer-Verlag, New York, 1990. MR1070713 [8] Elena Cordero and Fabio Nicola, Kernel theorems for modulation spaces, J. Fourier Anal. Appl. 25 (2019), no. 1, 131–144, DOI 10.1007/s00041-017-9573-3. MR3901921 [9] S. Dahlke, F. De Mari, E. De Vito, D. Labate, G. Steidl, G. Teschke, and S. Vigogna, Coorbit spaces with voice in a Fréchet space, J. Fourier Anal. Appl. 23 (2017), no. 1, 141–206, DOI 10.1007/s00041-016-9466-x. MR3602813 [10] Ingrid Daubechies, Ten lectures on wavelets, CBMS-NSF Regional Conference Series in Ap- plied Mathematics, vol. 61, Society for Industrial and Applied Mathematics (SIAM), Philadel- phia, PA, 1992. MR1162107 [11] M. Duflo and Calvin C. Moore, On the regular representation of a nonunimodular locally compact group, J. Functional Analysis 21 (1976), no. 2, 209–243, DOI 10.1016/0022- 1236(76)90079-3. MR0393335 [12] Hans G. Feichtinger, Un espace de Banach de distributions tempérées sur les groupes locale- ment compacts abéliens (French, with English summary), C. R. Acad. Sci. Paris Sér. A-B 290 (1980), no. 17, A791–A794. MR580567 [13] Hans G. Feichtinger, On a new Segal algebra, Monatsh. Math. 92 (1981), no. 4, 269–289, DOI 10.1007/BF01320058. MR643206 [14] H. G. Feichtinger. Modulation spaces on locally compact abelian groups. In Proceedings of “International Conference on Wavelets and Applications” 2002, pages 99–140, Chennai, In- dia, 2003. Updated version of a technical report, University of Vienna, 1983. [15] Hans G. Feichtinger and Karlheinz Gröchenig, A unified approach to atomic decompositions via integrable group representations, Function spaces and applications (Lund, 1986), Lec- ture Notes in Math., vol. 1302, Springer, Berlin, 1988, pp. 52–73, DOI 10.1007/BFb0078863. MR942257 [16] Hans G. Feichtinger and K. H. Gröchenig, Banach spaces related to integrable group representations and their atomic decompositions. I, J. Funct. Anal. 86 (1989), no. 2, 307–340, DOI 10.1016/0022-1236(89)90055-4. MR1021139 [17] Hans G. Feichtinger and K. H. Gröchenig, Banach spaces related to integrable group representations and their atomic decompositions. II, Monatsh. Math. 108 (1989), no. 2-3, 129–148, DOI 10.1007/BF01308667. MR1026614 [18] Hans G. Feichtinger and Karlheinz Gröchenig, Gabor wavelets and the Heisenberg group: Gabor expansions and short time Fourier transform from the group theoretical point of view, Wavelets, Wavelet Anal. Appl., vol. 2, Academic Press, Boston, MA, 1992, pp. 359–397. MR1161258 [19] H. G. Feichtinger and M. S. Jakobsen, The inner kernel theorem for a certain Segal algebra, arXiv:1806.06307, 2018. [20] Gerald B. Folland, Harmonic analysis in phase space, Annals of Mathematics Studies, vol. 122, Princeton University Press, Princeton, NJ, 1989. MR983366 [21] I. M. Gelfand and G. E. Shilov, Generalized functions. Vol. 2, Spaces of fundamental and generalized functions; Translated from the 1958 Russian original [ MR0106409] by Morris D. Friedman, Amiel Feinstein and Christian P. Peltzer; Reprint of the 1968 English translation [ MR0230128], AMS Chelsea Publishing, Providence, RI, 2016. MR3469849 [22] Karlheinz Gröchenig, Describing functions: atomic decompositions versus frames, Monatsh. Math. 112 (1991), no. 1, 1–42, DOI 10.1007/BF01321715. MR1122103 364 P. BALAZS, K. GROCHENIG,¨ AND M. SPECKBACHER

[23] Karlheinz Gröchenig, Foundations of time-frequency analysis, Applied and Numerical Har- monic Analysis, Birkhäuser Boston, Inc., Boston, MA, 2001. MR1843717 [24] Lars Hörmander, The analysis of linear partial differential operators. I, Distribution theory and Fourier analysis, 2nd ed., Springer Study Edition, Springer-Verlag, Berlin, 1990. MR1065136 [25] Richard V. Kadison and John R. Ringrose, Fundamentals of the theory of operator algebras. Vol. I, Elementary theory, reprint of the 1983 original, Graduate Studies in Mathematics, vol. 15, American Mathematical Society, Providence, RI, 1997. MR1468229 [26] Viktor Losert, A characterization of the minimal strongly character invariant Segal algebra, Ann. Inst. Fourier (Grenoble) 30 (1980), no. 3, 129–139. MR597020 [27] Ivor J. Maddox, Infinite matrices of operators, Lecture Notes in Mathematics, vol. 786, Springer, Berlin, 1980. MR568707 [28] Yves Meyer, Wavelets and operators, translated from the 1990 French original by D. H. Salinger, Cambridge Studies in Advanced Mathematics, vol. 37, Cambridge University Press, Cambridge, 1992. MR1228209 [29] Shahla Molahajloo, Kasso A. Okoudjou, and Götz E. Pfander, Boundedness of multilinear pseudo-differential operators on modulation spaces, J. Fourier Anal. Appl. 22 (2016), no. 6, 1381–1415, DOI 10.1007/s00041-016-9461-2. MR3572906 [30] A. Perelomov, Generalized coherent states and their applications, Texts and Monographs in Physics, Springer-Verlag, Berlin, 1986. MR858831 [31] H.-J. Schmeisser and H. Triebel, Topics in Fourier analysis and function spaces,Mathematik und ihre Anwendungen in Physik und Technik [Mathematics and its Applications in Physics and Technology], vol. 42, Akademische Verlagsgesellschaft Geest & Portig K.-G., Leipzig, 1987. MR900143 [32] Winfried Sickel and Tino Ullrich, Tensor products of Sobolev-Besov spaces and applications to approximation from the hyperbolic cross,J.Approx.Theory161 (2009), no. 2, 748–786, DOI 10.1016/j.jat.2009.01.001. MR2563079 [33] T. Tao, Lecture Notes 2 for 247 A: Fourier Analysis, www.math.ucla.edu/~tao/247a.1.06f/ notes2.pdf. [34] Ernest B. Vinberg, Linear representations of groups, translated from the 1985 Rus- sian original by A. Iacob, reprint of the 1989 translation, Modern Birkhäuser Classics, Birkhäuser/Springer, New York, 2010. MR2761806

Acoustics Research Institute, Austrian Academy of Sciences, Wohllebengasse 12-14, 1040 Vienna, Austria Email address: [email protected] Faculty of Mathematics, University of Vienna, Oskar-Morgenstern-Platz 1, A-1090 Vienna, Austria Email address: [email protected] Institut de Mathematiques´ de Bordeaux, Universite´ de Bordeaux, 351, cours de la Liberation´ - F 33405 Talence, France Email address: [email protected]