arXiv:1702.06781v2 [cs.IT] 28 Feb 2020 ℓ ihsrcue priy e uscin11 nteohrhand, other the On 1.1. Subsection see sparsity, o structured numbers with Gelfand of study The details. (approximately) further for for 2.2 models Subsection as processing signal orsodn ntbl is ball unit corresponding en htw r ofotdwt aaee oseltoswher constellations parameter with confronted are we that means ihsalmxdsmoothness. mixed small with (2) eal nti oncincnb on nPooiin22adtediscu the and 2.2 Proposition ℓ in found be can connection this on details eptepai ncssweeete 0 either where cases on emphasis put We inlpoesn,Gladnmeso mednsntrlyappea naturally element embeddings an of of recovery numbers optimal Gelfand processing, signal uniytecmateso h embedding the of compactness the quantify emti on fve,teGladnmeso nebdigbetwe embedding an of numbers Gelfand the theo Y view, approximation of in point concepts geometric fundamental are numbers Gelfand in sudrml supin qiaett the to equivalent assumptions mild under is otnosmaueetmpand map measurement continuous (1) hr h nmmi ae vralmp fteform the of maps all over taken is infimum the where ore rwvltcecet of coefficients wavelet or Fourier 2 bd p b ie by given , ( e od n phrases. and words Key h i fti ae st nesadtebhvo fteGladnu Gelfand the of behavior the understand to is paper this of aim The Y hc ecnuti hsppr ilsgnrlproac bounds perfomance general yields paper, this in conduct we which , ℓ q d o instance, For . EFN UBR EAE OSRCUE PRIYADBESOV AND SPARSITY STRUCTURED TO RELATED NUMBERS GELFAND -pcs hc r qipdwt h ie (quasi-)norm mixed the with equipped are which )-spaces, ubro neetn aaee oseltos u e ma new Our of constellations. embeddings parameter interesting of number qai)omembeddings (quasi-)norm prei-eesvcos epciey nadto,w a we addition, In respectively. vectors, sparse-in-levels Abstract. and/or eaira nteuiait iuto.I h eann ca remaining cases the particular In some bound. situation. in univariate univariate the that the out in as turns behavior It multivar of smoothness. numbers Gelfand mixed the for estimates two-sided new c m I : (Id q PC MEDNSWT ML IE SMOOTHNESS MIXED SMALL WITH EMBEDDINGS SPACE X ≤ ecnie h rbe fdtriigteaypoi order asymptotic the determining of problem the consider We .Teecsstr u ob eae osrcue sparsity. structured to related be to out turn cases These 1. → f ℓ 1 b Y a eafnto nacranfnto pc n h iersmlsma samples linear the and space function certain a in function a be may ( ℓ efn numbers, Gelfand inf = ) 2 d and ) non-convex ℓ ℓ f 2 b n p b f ( ( ∈ ℓ x ℓ 1 d rsml ucinvle.The values. function simple or q d ∈ E ϕ into ) X JEDDRSNADTN ULLRICH TINO AND DIRKSEN SJOERD ) k B sup m m x ֒ → X rmfwarbitrary few from k ( ∩ The . : ,Y X, ℓ ℓ ℓ p b M R r b ℓ p ( 2 b ( ℓ ( m ℓ ( k q d ℓ u d ℓ q p < ) x 2 d =inf := ) ie that given ) -pcs opesdsnig lc priy sparsity-i sparsity, block sensing, compressed )-spaces, → ml piaiyasrin o h eoeyo block-spa of recovery the for assertions optimality imply ) k := 1. m X ֒ Y ℓ p b t iia os-aercvr error recovery worst-case minimal -th Y ≤ Introduction ( S : ℓ → X sa rirr ntncsaiylna)rcvr a.More map. recovery linear) necessarily (not arbitrary an is m i block-sparse 2 d =1 r0 or 1 b M ֒ -and )- k Y f sup k 1 rmteprpcieo prxmto hoyand theory approximation of perspective the From . → X p X j S pyorsapetmtsfor estimates sharp our pply =1 ≤ d ≤ m linear X q < 1 ℓ r | k = x 2 b aeBsvsaeebdig nrgmso small of regimes in embeddings space Besov iate e hydffra otb o o atrfrom factor log log a by most at differ they ses f lsdsbpc ihcodim( with subspace closed and ( cigbud o h efn ubr fthe of numbers Gelfand the for bounds tching ij ℓ ϕ ≤ − | p d q and m ape,weetercvr ro smeasured is error recovery the where samples, hs siae hwtesm asymptotic same the show estimates these -al ih0 with )-balls q S rbt 0 both or 1 m p/q ◦ m ≤ h mednsof embeddings the f 1t efn number Gelfand +1-th e N ( sparse-in-levels u yadBnc pc emty rma From geometry. space Banach and ry ℓ f hncnieigtepolmo the of problem the considering when r ihepai ncsswith cases on emphasis with , p b k · k m 1 ) ( /p k Here . ℓ nto(us-nre spaces (quasi-)normed two en Y fteGladnmeso mixed- of numbers Gelfand the of q d . so nRmr . below. 2.3 Remark in ssion -al perntrlyi wavelet in naturally appear )-balls ℓ , o h pia eoeyo signals of recovery optimal the for p b eoti hr onsi a in bounds sharp obtain We ( ℓ q d ,q p, < ) br febdig between embeddings of mbers N p < sol us-omadthe and quasi-norm a only is m ℓ p b ( : ℓ ≤ ≤ q d X inl,rsetvl,see respectively, signals, -pcst obtain to )-spaces perntrlyin naturally appear 1 ,wihi particular in which 1, → ℓ p b M ( R c ℓ -ees eo spaces Besov n-levels, 2 d m ) m and ) +1 m < salna and linear a is s and rse I : (Id p oss of consist y ≤ o ℓ , 2 b 1 ( X ℓ X p d → into ) and Y ) 2 SJOERD DIRKSEN AND TINO ULLRICH
characterizations of multivariate Besov spaces with dominating mixed smoothness. The study of embeddings b b between quasi-normed ℓp(ℓq)-spaces in this paper leads to new asymptotic bounds for Gelfand numbers of embeddings between the aforementioned Besov spaces in regimes of small smoothness, see Subsection 1.2. Interestingly, in some particular cases these estimates show the same asymptotic behavior as in the univariate situation. In the remaining cases they differ at most by a log log factor from the univariate bound. 1.1. Gelfand numbers and sparse recovery. We obtain matching bounds for the Gelfand numbers c (id : ℓb (ℓd) ℓb(ℓd )) m p q → r u for a number of interesting parameter ranges in this paper. The results in full detail can be found in Section 6. At this point, we wish to discuss only the most illustrative cases and highlight some interesting effects. In particular, we prove for 1 m bd and 0
Then, there exist constants c1,c2 > 0 depending only on C so that if s>c2, then m c (s log(eb/s)+ sd). ≥ 1 On the other hand, if for all x Rbd, ∈ inner σ (x) b d t ℓ2(ℓ1 ) (8) x ∆(Ax) ℓb (ℓd) D , k − k 2 2 ≤ √t
then there exist constants c1,c2 > 0 depending only on D such that m c bt log(ed/t), ≥ 1 provided that t>c2. GELFAND NUMBERS, STRUCTURED SPARSITY, BESOV SPACE EMBEDDINGS 3
To discuss the consequences of this result for structured sparse recovery, let us recall the following known results. We refer to Section 2.2 for a more extensive discussion of the literature. Let B be an m bd Gaussian matrix, i.e., a matrix with i.i.d. standard Gaussian entries and set A = (1/√m)B. For any 0 <× p,q , let ≤ ∞ ∆ℓ (ℓ )(Ax) = argmin Rbd z b d p q z∈ : Az=Axk kℓp(ℓq ) be the reconstruction via ℓp(ℓq)-minimization. It is known that if the number of measurements m satisfies
m & s log(eb/s)+ sd, then with high probability the pair (A, ∆ℓ1(ℓ2)) satisfies the stable reconstruction property in (7) (see [19] and also the discussion in [3, Section 2.1]). Moreover, this number of measurements cannot be reduced: [3, Theorem 2.4] shows that if for a given A Rm×bd one can recover every s-outer ∈ sparse vector x exactly from y = Ax via ∆ℓ1(ℓ2) then necessarily m & s log(eb/s)+ sd. However, one could still hope to further reduce the required number of measurements for outer sparse recovery by picking a better pair (A, ∆). Corollary 1.2 shows that this is not possible if we want the same stable reconstruction
guarantee as in (7). In this sense, the pair (A, ∆ℓ1(ℓ2)) with A = (1/√m)B is optimal.
It is worthwhile to note that the pair (A, ∆ℓ1(ℓ1)), A = (1/√m)B, with high probability satisfies the stable reconstruction guarantee ([10, Theorem 1.2 and Section 1.3], see also [11, 14] and [23, Theorem 9.13])
σk(x) b d ℓ1 (ℓ1 ) b d (9) x ∆ℓ1(ℓ1)(Ax) ℓ (ℓ ) D , k − k 2 2 ≤ √k whenever m & k log(ebd/k), where
σk(x)ℓb (ℓd) = inf x z ℓb(ℓd) : z is k -sparse . 1 1 {k − k 1 1 } As a consequence (by taking k = sd), (7) is satisfied under the condition m & sd log(eb/s). In conclusion, if we are only interested in stability of our reconstruction method with respect to outer sparsity (instead of sparsity) defects, then we can achieve this with fewer measurements (in a best worst-case scenario) by using
∆ℓ1(ℓ2) as a reconstruction map instead of ∆ℓ1(ℓ1). In the case of reconstruction of inner-sparse vectors, one might expect that a similar phenomenon occurs: if we are only interested in stability with respect to inner sparsity (instead of sparsity) defects, then we might be able to reconstruct from fewer measurements by using a reconstruction map that is especially suited for inner sparse recovery (a potential candidate would be
∆ℓ2(ℓ1), in analogy with the outer sparse case). Somewhat surprisingly, Corollary 1.2 shows that this is not the case. By taking k = bt in (9) and using that every t-inner sparse vector is bt-sparse (but not vice-versa),
one deduces that the pair (A, ∆ℓ1(ℓ1)), A = (1/√m)B, with high probability satisfies (8) if m & bt log(ed/t). This is optimal by Corollary 1.2. 1.2. Besov space embeddings with small mixed smoothness. As a second application of our work, we are interested in proving upper and lower bounds for the Gelfand numbers of compact embeddings between Besov spaces (10) Id : Sr0 B([0, 1]d) Sr1 B([0, 1]d) , p0,q0 → p1,q1 where 0 < p0 p1 , 0 < q0, q1 and r0 r1 > 1/p0 1/p1. By using a well-known hyperbolic wavelet discretization≤ ≤ machinery, ∞ see for≤ ∞ instance [63],− we can tran−sfer the problem to bounding the Gelfand numbers of id : sr0 b([0, 1]d) sr1 b([0, 1]d), p0,q0 → p1,q1 where (see (46) below for details) q/p 1/q r (r−1/p)qk¯jk1 p ¯ ¯ C r ¯ sp,qb := λ = λ¯j,k ¯j,k : λ sp,qb := 2 λ¯j,k < { } ⊂ k k d | | ∞ ¯j∈N k¯∈A¯ n h X0 Xj i o j j d r and A¯ = ([0, 2 1 1] [0, 2 d 1]) Z . We immediately observe that s b has an ℓ (ℓ )-structure. j − ×···× − ∩ p,q q p After a suitable block decomposition we use the subadditivity of Gelfand numbers (see (S2) below) to reduce the problem to finite-dimensional blocks. This strategy has for instance been pursued in the recent paper [46], where the case p0 = q0 > 1 has been treated. In our work, we are interested in the case min p0, q0 1, max p1, q1 2, which is connected to Theorem 6.1 below. As it turns out, there is a strong{ qualitative} ≤ difference{ } in ≤ the behavior of the Gelfand numbers depending on whether one is in the “large mixed smoothness” (r0 r1 > 1/q0 1/q1) or “small mixed smoothness” (r0 r1 1/q0 1/q1) regime. In the large smoothness− regime, sharp− bounds for the Gelfand numbers can be readily− ≤ derived:− using 4 SJOERD DIRKSEN AND TINO ULLRICH almost literally the arguments in [63, Thms. 3.18, 3.19] together with the well-known sharp estimates for the n n Gelfand numbers of the (non-mixed) embeddings id : ℓu ℓv where 0 0 depending≤ 0 ≤ 0 ≤ 1 ≤ 1 ≤ 0 − 1 0 − 1 only on p0,p1, q0, q1, d and r,
−r r d 0 d −r 1/q0 −1/q1 (13) cm cm(Id : S B([0, 1] ) S B([0, 1] )) Cm (log log m) . ≤ p0,q0 → p1,q1 ≤ 2
By taking p1 = q1 and q0 in the eighties. We refer to [64, Section 4] for a summary of these classical results and historical references. Let us only specifically mention the classical bound log(ebd/m) 1/2 (14) c (id : ℓb (ℓd) ℓb (ℓd)) min 1, , m 1 1 → 2 2 ≃ m n o which was obtained by Garnaev and Gluskin [26, 28] following important earlier advances of Kashin [36]. Note that one recovers this result from (4) by taking ‘b = bd’, d = 1. Some of the classical estimates were extended to the quasi-Banach range p< 1, see [22, 64], where it was shown that for 0 1 1/q, then ∞ − r− 1 + 1 logd−1 m 2 q d−1 (log m) q : 1 + 1 < 1, p 2, r Td Td m p q ≤ cm(Id : Sp,∞B( ) Lq( )) p,q,r,d r− 1 + 1 → ≃ logd−1 m p q d−1 (log m) q : 2 p < q. m ≤ The lower bound in Theorem 1.3 for the (larger) approximation numbers (linear widths) was proved recently by Malykhin and Ryutin. It closes a log log-gap in a result of Galeev [25] from 1996 . Let us also mention the results from [35, Thm. 3] and the remark following it. Let m < bd/2. First, c (id : ℓb (ℓd) ℓb (ℓd )) 1; m ∞ 2 → ∞ ∞ ≃ Second, for 1 p 2, ≤ ≤ b d b d 1 1 − 1 c (id : ℓ (ℓ ) ℓ (ℓ )) b p d p 2 . m ∞ 2 → p p ≃p Third, if 2 p< and in addition there exists a constant γ > 0 such that d γ log b, then ≤ ∞ ≥ 1 p b b d b d 1 . c (id : ℓ (ℓ ) ℓ (ℓ )) b p . 1− 1 γ,p m ∞ 2 p p (log b) p → ≤ 6 SJOERD DIRKSEN AND TINO ULLRICH Finally, we state a result of Vasileva from [62]. For 1 a 2 and 1 a b we define ≤ ≤ ≤ ≤ ≤ ∞ 1 1 a b λ(a,b) = min 1 − 1 , 1 n a − 2 o if a = 2 and λ(a,b) = 1 if a = 2. Note that if a< 2, then λ(a,b) < 1 if and only if b< 2. 6 Theorem 1.4 ([62]). Let 1 < p 2, 1 < q 2, p r , and q u . Then there exists an a = a(p, q) > 0 such that for all m,b,d≤ with m ≤abd ≤ ≤ ∞ ≤ ≤ ∞ ≤ c (id : ℓb (ℓd) ℓb(ℓd )) Φ (m,b,d,p,q,r,u) m p q → r u ≃p,q,r,u 0 provided that one of the following conditions holds: (i) u 2, r 2, and ≥ ≥ − 1 1− 1 1− 1 Φ = min 1,m 2 d q b p 0 { } (ii) λ(q,u) λ(p, r), λ(q,u) < 1, and ≤ − 1 1− 1 1− 1 λ(q,u) 1 − 1 − 1 1 1− 1 λ(p,r) Φ0 = min 1, (m 2 d q b p ) , d u q (m 2 d 2 b p ) (iii) λ(q,u) λ(p, r), λ(p, r) n< 1, and o ≥ − 1 1− 1 1− 1 λ(p,r) 1 − 1 − 1 1− 1 1 λ(q,u) Φ0 = min 1, (m 2 d q b p ) ,b r p (m 2 d q b 2 ) . 1.4. Outline. After fixing notationaln conventions we introduce several basic concepts,o such as Gelfand numbers and Gelfand widths and generalized notions of sparsity and compressibility in Section 2. In Section n 3 we set up some machinery to prove upper bounds for Gelfand widths of bounded subsets in (R , Y ) via random subspaces based on Gordon’s “escape through the mesh” theorem. For the lower boundsk·k we establish a connection to packing numbers of structured sparse vectors and give lower bounds for these packing numbers, see Sections 4 and 5. Section 6 establishes matching bounds for Gelfand numbers for b d embeddings between ℓp(ℓq )-spaces in several interesting parameter ranges. These results are applied in Section 7, where we prove Corollary 1.2 and establish new results for the Gelfand numbers of Besov space embeddings with small mixed smoothness. 1.5. Notation. As usual N denotes the natural numbers, N := N 0 , Z denotes the integers, R the real 0 ∪{ } numbers, and C the complex numbers. For a R we denote a+ := max a, 0 . We write log for the natural logarithm. Rm×n denotes the set of all m n-matrices∈ with real entries and{ R}n denotes the Euclidean space. R×n ¯ ¯ Nd Vectors are usually denoted with x, y , sometimes we use j, k 0 for multi-indices. For 0 < p n ∈ ∈ ≤ ∞ and x Rn we denote x := ( x p)1/p with the usual modification in the case p = . If X is a ∈ k kp i=1 | i| ∞ (quasi-)normed space, then BX denotes its unit ball and the (quasi-)norm of an element x in X is denoted P by x X . For any U X we let rX (U) = supx∈U x X denote the radius of U. In case X is a Banach k k ∗ ⊂ k k space, X denotes its dual. We will frequently use the quasi-norm constant, i.e., the smallest constant αX satisfying x + y α ( x + y ), for all x, y X. k kX ≤ X k kX k kX ∈ For a given 0 0 such that a cb for all n. We write a ⊂ b if both a . b and b . a . If α is a set of parameters, then we n ≤ n n ≃ n n n n n write an .α bn if there exists a constant cα > 0 depending only on α such that an cα bn for all n. N b d ≤ Let b, d . The bd-dimensional mixed space ℓp(ℓq ) is defined as the space of all bd-dimensional block ∈b Rbd Rd vectors (xi)i=1 , where xi for all 1 i b, equipped with the mixed (quasi-)norm defined in (2). Occasionally we∈ will identify x ∈ Rbd with the≤b ≤d matrix whose rows are the vectors x . We always refer ∈ × i to the ℓp-space supported on [b] := 1,...,b as the outer space and to the ℓq-space supported on [d] as the { } bd inner space. For any S [b] [d] and x R we define x to be the block vector with entries (xS )ij = xij for (i, j) S, (x ) = 0⊂ for (i,× j) Sc. ∈ ∈ S ij ∈ GELFAND NUMBERS, STRUCTURED SPARSITY, BESOV SPACE EMBEDDINGS 7 2. Preliminaries 2.1. Gelfand numbers and worst-case errors. We recall the definition of Gelfand numbers and the mild conditions under which Gelfand numbers can be considered as worst-case approximation/recovery errors. Let X, Y be (quasi-)Banach spaces and T : X Y be a continuous map. → Definition 2.1. Let X, Y be two quasi-Banach spaces and T (X, Y ). For m N the m-th Gelfand number is defined as ∈ L ∈ . c (T ) := inf TJ X : M ֒ X closed subspace with codim(M) (17) cm(K, Y ) := inf sup f Y : f K M . { ∩ ∋ M֒→Y,codim(M)≤m {k k The terms Gelfand numbers and Gelfand widths are often used interchangeably in the literature although they are defined in a fundamentally different way. In particular, it is important to note that in the definition of the Gelfand numbers c (id : X Y ) we take an infimum over subspaces M of X, whereas in the m → definition of Gelfand widths cm(BX , Y ) we take an infimum over subspaces M of Y . In particular, if X and Y are quasi-Banach spaces then it is not clear whether the above Gelfand numbers and Gelfand widths are equivalent. Only recently, some sufficient conditions have been determined under which c (id : X Y ) m → and cm(BX , Y ) are equivalent, see [16]. Obviously, if X and Y are both n-dimensional, then the Gelfand numbers and Gelfand widths coincide. (ii) A relation between Gelfand widths and a corresponding notion of minimal worst case errors, which are nowadays sometimes called compressive m-widths [23, Def. 10.2, Thm. 10.4], was first given in [59, Thm. 8 SJOERD DIRKSEN AND TINO ULLRICH 5.4.1]. These associated minimal worst-case errors are defined, for a given bounded set K in a (quasi-)Banach space Y , as (18) Em(K, Y ) := inf sup f Sm(f) Y Sm f∈K k − k m where the infimum is taken over all maps Sm = ϕm Nm, with Nm : Y R any linear and continuous measurement map and ϕ : Rm Y an arbitrary recovery◦ map. Note that→ the worst-case reconstruction m → error Em(X, Y ) in (1) and Em(BX , Y ) in (18) are defined in a subtly different manner: in the first case we m m consider measurement maps Nm : X R , whereas in the second case we consider maps Nm : Y R . The compressive m-width and the m-th→ Gelfand width are equivalent under mild conditions [23, Thm.→ 10.4]. In particular, if and are quasi-norms on Rn, then k·kX k·kY (19) α−1c (B , Y ) E (B , Y ) 2α c (B , Y ). Y m X ≤ m X ≤ X m X In conclusion, the m-th Gelfand number c (id : X Y ), the m-th Gelfand width c (B , Y ) and m → m X their associated minimal worst-case errors Em(X, Y ) and Em(BX , Y ), respectively, can in general be four non-equivalent quantities. However, if X and Y are finite-dimensional quasi-normed spaces of the same dimension, then these quantities are equivalent up to constants depending only the quasi-norm constants of b d b d X and Y . In particular, if X = ℓp(ℓq ) and Y = ℓr(ℓu), then the constants in the equivalences depend only on p,q,r and u. We will use these facts heavily in the sections that follow. 2.2. Generalized notions of sparsity and compressibility. We fix some terminology concerning struc- b Rdi tured sparsity. Consider a block vector x = (xi)i=1 with xi . As usual, we say that x is s-sparse if it has at most s nonzero entries. We say x is s-block sparse if∈ at most s blocks of x are non-zero. This type of sparsity occurs frequently in the signal processing literature (see e.g. [3, 17, 18, 19]). Block sparsity is strongly related to (and can in fact be viewed as a generalization of) the joint sparsity model, where one considers a signal consisting of several ‘channels’ (for instance, the three color channels of an RGB image) and assumes that nonzero coefficients appear at the same location within each of the channels (see e.g. [19, 20, 30] and [3, Section 1.2]). Block sparsity also plays an important role in high-dimensional statistics (see e.g. [8, Chapters 4 and 8], [65, Sections 2.1 and 3.2.1] and the references therein) and appears in machine learning, for instance in multi-task learning and multiple kernel learning (see e.g. [4, Sections 1.3 and 1.5] and the references therein). In this literature, it is sometimes of interest to consider group sparsity, a generalization of the block sparsity model where the signal consists of a small number of groups which are allowed to overlap. There is a well-developed theory available on sufficient conditions for recovery of block and, more generally, group sparse vectors via the group Lasso, see e.g. [8, 33, 41, 45]. As a third form of sparsity, we consider b sparsity-in-levels, which has emerged recently in the signal processing literature. A block vector x = (xi)i=1 b is called (ti)i=1-sparse-in-levels if each block xi is a ti-sparse vector. Empirically it has been observed that certain signals exhibit this type of structured sparsity after expansion in an appropriate basis. For instance, the wavelet coefficients of natural images often exhibit an (approximately) sparse-in-levels structure, where the levels correspond to the wavelet scales. We refer to e.g. [1, 5, 7, 40] for recent work on sufficient recovery guarantees for sparse-in-levels signals. To keep our exposition non-technical and to ensure that our sparsity concepts are linked to classical ℓp(ℓq)-spaces, we restrict ourselves to the study of block sparsity and sparsity-in-levels in the case where the block sizes and (in the case of sparsity-in-levels) the sparsity of the blocks are identical. Accordingly, we say b Rbd Rd that a block vector x = (xi)i=1 with xi for all i [b] is s-outer sparse if at most s blocks of x are non-zero and say that x is t-inner∈ sparse if∈ each block of x∈has at most t non-zero entries. The qualifiers ‘outer’ and ‘inner’ refer to the fact that the sparsity is defined with respect to the outer ℓp-space or inner ℓ -space, respectively, if x ℓb (ℓd). For our analysis it will also be important to consider a mixture of outer q ∈ p q and inner sparsity: we define x to be (s,t)-sparse if it has at most s non-zero blocks, each of which has at most t-non-zero entries. To illustrate why outer and inner sparsity should play an important role for the Gelfand numbers of b d ℓp(ℓq )-spaces where p and/or q are small, let us point out a generalization of “Stechkin’s lemma” [23, Prop. 2.3, Thm. 2.5] to the two different best s-term approximation errors (6) associated with inner and outer sparsity. Let us emphasize that although [23, Prop. 2.3, Thm. 2.5] is often attributed to S.B. Stechkin, he never formulated such a result. The first known version goes back to Temlyakov 1986 [58, p. 92] for p, q 1. Note, that there is indeed a well-known Stechkin lemma (or Stechkin criterion) from 1955 proved in [56],≥ GELFAND NUMBERS, STRUCTURED SPARSITY, BESOV SPACE EMBEDDINGS 9 see also [13, 7.4] for further historical comments. This criterion implies the inequality [23, Prop. 2.3] in case p = 1 and q = 2 but with a constant larger than 1. Let us now formulate two generalizations of [23, Prop. 2.3]. Their proofs are straightforward modifications. If 0 The bound (20) shows that vectors in B b d with p small can be approximated well by outer-sparse vectors ℓp(ℓq ) and, similarly, (21) implies that elements in B b d with q small can be approximated well by inner-sparse ℓp(ℓq ) vectors. The constant c = 1 in (20) and (21) is not optimal. It is actually below one, depending on p,q,r,u, see [13, Section 7.4] and the references therein. 3. A general upper bound for Gelfand widths of K Rn ⊂ In this section we set up some machinery to prove upper bounds for Gelfand numbers based on “Gordon’s escape through a mesh” theorem . To facilitate potential re-use, we write our results down in more generality than needed in Section 6 below. Below we will deal with Gelfand widths of bounded subsets in Rn, see (17) in Remark 2.3 . The main tool in the proof will be Gordon’s “Escape through a mesh” theorem [29, Cor. 3.4]. Recall that the Gaussian width of a set T Rn is defined by ⊂ w(T )= E sup g, x , x∈T h i where g is an n-dimensional standard Gaussian vector. Let us denote Γ((n + 1)/2) E := E g = √2 . n k k2 Γ(n/2) We will state the version of Gordon’s theorem from [23, Thm. 9.21]. Lemma 3.1. Let A Rm×n be a Gaussian random matrix and T be a subset of the unit sphere Sn−1 := x Rn : x =1 ∈. Then for every t> 0 we have { ∈ k k2 } −t2/2 P inf Ax 2 Em w(T ) t e . x∈T k k ≤ − − ≤ By the known relation m/√m +1 Em √m (see e.g. [23, Prop. 8.1]), Lemma 3.1 yields a non-trivial ≤ ≤ 2 result if Em > w(T ) and in particular if m & w(T ) . n n Proposition 3.2. Let 0 <ρ< , Y denote a quasi-norm on R and K R be a bounded symmetric set. Define the set ∞ k·k ⊂ K = x K : x >ρ ρ { ∈ k kY } k·k2 and let Kρ = x/ x : x K be the associated set of ℓ -normalized vectors. If { k k2 ∈ ρ} 2 (22) m & max w(Kk·k2 )2, 1 , { ρ } then c (K, Y ) ρ. m ≤ m×n Proof. To prove the assertion, we need to find an A R such that x Y ρ for all x ker(A) K. 2 ∈ k k ≤ ∈ ∩ This is satisfied if infx∈Kρ Ax 2 > 0. By Lemma 3.1, if A is an m n standard Gaussian matrix such that k·k2 k k × Em > w(Kρ ), then (23) γ(K,ρ; A) := inf Ax 2 > 0 k·k2 x∈Kρ k k 2 k·k2 with probability at least 1 exp( t /2), where 0 n m×n for a certain CY > 0, as all quasi-norms on R are equivalent. Hence, (22) ensures the existence of A R satisfying inf Ax > 0. ∈ x∈Kρ k k2 In conclusion, we can obtain an upper bound for the Gelfand width by estimating x k·k2 E w(Kρ )= sup ,g x∈K : kxkY >ρ x 2 Dk k E from above. 4. A lower bound for Gelfand widths via packing numbers We generalize the proof-by-contradiction technique from [22], which was used to prove lower bounds for n n cm(id : ℓp ℓq ). To this end, we establish a connection between Gelfand and packing numbers in Proposition 4.4, which→ requires some further preparation. Recall that for a quasi-normed space X, a bounded subset K X, and ε > 0, the packing number (K, X ,ε) is the maximal number of elements in K which have⊂ pairwise distances strictly larger than Pε. Thek·k following well-known lemma can be proven by a standard volume comparison argument. n Lemma 4.1. Let X be any quasi-norm on R with quasi-norm constant 1 αX < and let U be a subset of the unit ballk·kB = x Rn : x 1 . Then for any ε> 0, ≤ ∞ X { ∈ k kX ≤ } 2 n (U, ,ε) αn 1+ . P k·kX ≤ X ε m×n n n Lemma 4.2. Let A R and let X be any quasi-norm on R . Suppose that T R is a set of vectors satisfying, for∈ some c 1, k·k ⊂ ≥ −1 (24) inf z X c x X for all x T T. z∈Rn : Az=Ax k k ≥ k k ∈ − Let U T and let r (U) = sup x be its radius. Then, for any ε> 0, ⊂ X x∈U k kX 2cα r (U) log (U, ,ε) m log α + X X , P k·kX ≤ X ε where αX is the quasi-norm constant. Proof. Consider the quotient space Q = X/ker(A) equipped with its natural quasi-norm [x] Q := inf x v X . k k v∈ker(A) k − k Note that for every v ker(A), the vector z = x v satisfies Az = Ax. On the other hand, if z Rn satisfies Az = Ax, then v = x ∈ z defines a vector in ker(−A). By (24), this implies ∈ − [x] x c [x] for all x T T k kQ ≤k kX ≤ k kQ ∈ − and in particular (U, ,ε) (U,c ,ε)= (U, ,ε/c). P k·kX ≤P k·kQ P k·kQ Now apply Lemma 4.1 to obtain (U, ,ε) (r (U)B , ,ε/c) P k·kX ≤P X Q k·kQ 2cα r (U) rank(A) 2cα r (U) m α + X X α + X X . ≤ X ε ≤ X ε Taking logarithms yields the result. bd In what follows, we let X be a quasi-norm on R with quasi-norm constant 1 αX < . Moreover, we let 0 <β < denotek·k the smallest constant such that for all S [b] [d] and all≤x Rbd∞we have X ∞ ⊂ × ∈ x + x c β x . k SkX k S kX ≤ X k kX Clearly, for any S [b] [d] and x Rbd, ⊂ × ∈ x α ( x + x c ) α β x , k kX ≤ X k S kX k S kX ≤ X X k kX so αX βX 1. Note that β b d 2 depends only on p and q. ≥ ℓp(ℓq ) ≤ GELFAND NUMBERS, STRUCTURED SPARSITY, BESOV SPACE EMBEDDINGS 11 Lemma 4.3. Let be a collection of supports in [b] [d] and let A Rm×bd. Suppose that for every S × ∈ bd v Ker(A) 0 and S we have v < v c . Then, for any x R and S , ∈ \{ } ∈ S k SkX k S kX ∈ ∈ S −1 inf z X > (αX βX ) xS X . bd z∈R : Az=AxS k k k k Proof. Let x Rbd. If z = x satisfies Az = Ax , then v = x z Ker(A) 0 and therefore ∈ 6 S S S − ∈ \{ } α−1 x x z + z = v + z X k SkX ≤k S − SkX k SkX k SkX k SkX < v c + z k S kX k SkX = z c + z β z . k S kX k SkX ≤ X k kX Let us define the parameter xS X κX,Y,S = sup k k . Rbd x x∈ ,S∈S k SkY Proposition 4.4. Let be a collection of supports in [b] [d] and define + = S1 S2 : S1,S2 . Let be a quasi-norm onS Rbd satisfying × S { ∪ ∈ S} k·kY x x for all x Rn,S . k SkY ≤k kY ∈ ∈ S+ If −1 cm(BX , Y ) < (2αX κX,Y,S+ ) then for any U x : x Rbd, S and ε> 0, ⊂{ S ∈ ∈ S} 2α2 β r (U) log (U, ,ε) m log α + X X X . P k·kX ≤ X ε Proof. By definition of the Gelfand width, there exists an A Rm×bd such that for every x Ker(A) 0 ∈ ∈ \{ } x < (2α κ )−1 x . k kY X X,Y,S+ k kX For any S , ∈ S+ x κ x k SkX ≤ X,Y,S+ k SkY −1 1 κ x < (2α ) x ( x + x c ). ≤ X,Y,S+ k kY X k kX ≤ 2 k SkX k S kX Rearranging, we conclude that xS X < xSc X . Now apply Lemma 4.3 and subsequently Lemma 4.2 (with T = x : x Rbd, S , notingk k thatkT kT = x : x Rbd, S ) to obtain the conclusion. { S ∈ ∈ S} − { S ∈ ∈ S+} b d Proposition 4.4 can be used to prove lower bounds for the Gelfand widths c (B b d ,ℓ (ℓ )) via con- m ℓp(ℓq ) r u b d tradiction: one assumes that c (B b d ,ℓ (ℓ )) is small and then constructs a large packing consisting of m ℓp(ℓq ) r u structured sparse vectors to find a contradiction. This packing construction is the subject of the next section. 5. Construction of a large packing in B b d ℓp(ℓq ) We obtain a lower bound for the packing numbers (B b d , b d ,ε), where 0 < p,q,r,u . The P ℓp(ℓq ) k·kℓr(ℓu) ≤ ∞ proof, inspired by [3], is based on two combinatorial facts. The first combinatorial fact has been independently observed in various disciplines of mathematics. A proof can be found, e.g., in [23, Lemma 10.12], [38] or [50, Prop. 2.21, Page 219] . Lemma 5.1. Given integers ℓ The second combinatorial fact is a variation of Lemma 5.1, which is well known in coding theory [27, 61]. For given sets A ,...,A we let d denote the Hamming distance on A A , i.e., 1 ℓ H 1 ×···× ℓ ℓ dH ((x1,...,xℓ), (y1,...,yℓ)) = 1xi6=yi . i=1 X Lemma 5.2. (Gilbert-Varshamov bound) Fix θ > 1. Let A1,...,Aℓ be sets, each consisting of θ elements. Then, for any k [ℓ], ∈ θℓ (A1 Aℓ, dH , k) . P ×···× ≥ k−1 ℓ (θ 1)j j=0 j − b Rbd Let us recall that we defined a block vector x = (xi)iP=1 to be (s,t)-sparse if it has at most s non-zero blocks, each of which has at most t non-zero entries. In∈ the proof of the following proposition it will bd be notationally convenient to identify each x R with the b d matrix having the xi as its rows. Under this identification, the matrix x is (s,t)-sparse∈ if it has at most×s non-zero rows, each of which has at most t non-zero entries. Proposition 5.3. Let b, d N with b, d 8. For any 0 < p,q,r,u , s [ b/8 ], and t [ d/8 ], there ∈ ≥ 1/p 1/q≤ ∞ ∈ ⌊ ⌋ ∈ ⌊ ⌋ is a set W of (2s, 2t)-sparse vectors with r b d (W ) (2s) (2t) such that ℓp(ℓq ) ≤ s st 1/r 1/u b d (W, b d ,s (2t) ) . P k·kℓr(ℓu) ≥ 32s 8t t Proof. We use Lemma 5.1 to find a collection Ad,t of (d/(8t)) subsets of [d] such that each I Ad,t has cardinality 2t and ∈ card(I J) W = x Rb×d : x M, x =0 . ∈ Iα×[d] ∈ ([b]\Iα)×[d] α [ n o Observe that (d/(8t))st b s d st card(W ) = (b/(8s))scard(M) (b/(8s))s = . ≥ 4s 32s 8t For I and K let us denote α ∈ A xIα,K = eij . iX∈Iα jX∈Ki GELFAND NUMBERS, STRUCTURED SPARSITY, BESOV SPACE EMBEDDINGS 13 By construction, p/q 1/q 1/p 1/q xIα,K ℓb (ℓd) = (2s) (2t) , k k p q ≤ iX∈Iα jX∈Ki 1/p 1/q so rℓb (ℓd)(W ) (2s) (2t) . Moreover, for K,L with K = L, the property (25) implies p q ≤ ∈ A 6 r/u 1/r 1/r 1/u xI ,K xI ,L b d = s (2t) . k α − α kℓr(ℓu) ≥ iX∈Iα j∈KXi∆Li Finally, if I = I , then α 6 γ r/u r/u xIα,K xIγ ,L ℓb (ℓd ) = + k − k r u i∈XIα∩Iγ j∈KXi∆Li i∈XIα\Iγ jX∈Ki r/u 1/r + (2s)1/r(2t)1/u. ≥ i∈XIγ \Iα jX∈Li In conclusion, W is an s1/r(2t)1/u-packing. 6. Matching bounds for Gelfand numbers This section is devoted to the proof of the following extension of Theorem 1.1. Theorem 6.1. Let m bd. Then the following assertions hold: ≤ (i) If 0 Remark 6.2. The two-sided estimate in (iii) in the case c b log ebd m bd remains valid if 0 < q 1 p,q m ≤ ≤ ≤ and q p 2. This will be clear from the proof. ≤ ≤ Remark 6.3. Our proof of the upper bounds in Theorem 6.1 is non-constructive. As a matter of fact, our proof shows (see in particular the proof of Proposition 3.2) that the two-sided bounds in Theorem 6.1 are with high probability attained by the kernel of an m bd standard Gaussian matrix. × We start by proving the upper bound in (26). Consider the set Rbd Db,d,s = x : x ℓb (ℓd) √s, x ℓb (ℓd) 1 . { ∈ k k 1 2 ≤ k k 2 2 ≤ } We will need the following observation, which is known in the case d = 1 (see [51, Lemma 3.1] and [52, Lemma 2.9]). 14 SJOERD DIRKSEN AND TINO ULLRICH Lemma 6.4. Let Lb,d,s be the set of all s-outer sparse vectors in B b d . Then, ℓ2(ℓ2 ) (28) conv(L ) D 2conv(L ). b,d,s ⊂ b,d,s ⊂ b,d,s As a consequence, (29) w(Db,d,s) . s log(eb/s)+ √sd. p Proof. The first inclusion in (28) follows immediately, since by Cauchy-Schwarz, x ℓb (ℓd) √s if x Lb,d,s. k k 1 2 ≤ ∈ For the second inclusion, let x D . We write x = x , where the S are disjoint subsets of [b], ∈ b,d,s k≥0 Sk×[d] k satisfying Sk = s and xi· ℓd xj· ℓd if i Sk−1 and j Sk. In particular, for k 1, | | k k 2 ≥k k 2 ∈ P ∈ ≥ 1 x b d √s max xi· d √s min xj· d x b d . Sk×[d] ℓ2(ℓ2 ) ℓ2 ℓ2 Sk−1×[d] ℓ1(ℓ2 ) k k ≤ i∈Sk k k ≤ j∈Sk−1 k k ≤ √sk k b d b d Note, moreover, that xS0×[d] ℓ (ℓ ) x ℓ (ℓ ) 1 and therefore k k 2 2 ≤k k 2 2 ≤ 1 1 b d b d b d b d xSk ×[d] ℓ (ℓ ) 1+ xSk×[d] ℓ (ℓ ) 1+ xSk×[d] ℓ (ℓ ) =1+ x ℓ (ℓ ) 2. k k 2 2 ≤ k k 2 2 ≤ √sk k 1 2 √sk k 1 2 ≤ kX≥0 kX≥1 Xk≥0 Writing xSk×[d] b d x = xSk×[d] ℓ (ℓ ) 2 2 b d k k xSk×[d] ℓ (ℓ ) kX≥0 k k 2 2 b d and noting that xSk ×[d]/ xSk×[d] ℓ (ℓ ) is in Lb,d,s, we conclude that the second inclusion in (28) holds. This k k 2 2 immediately implies that w(Db,d,s) 2E sup y,g =2E sup gS b d =2E sup( gS b d E gS b d ) + 2 sup E gS b d , ℓ2(ℓ2 ) ℓ2(ℓ2 ) ℓ2(ℓ2 ) ℓ2(ℓ2 ) ≤ y∈Lb,d,sh i S∈S k k S∈S k k − k k S∈S k k where g is a standard Gaussian vector of length bd and = S˜ [d]: S˜ [b], card(S˜)= s S { × ⊂ } is the set of all s-outer sparse support sets in [b] [d]. Clearly, × E E 2 1/2 √ sup gS ℓb (ℓd) sup( gS ℓb(ℓd)) = sd. S∈S k k 2 2 ≤ S∈S k k 2 2 By the Gaussian concentration inequality for Lipschitz functions (see e.g. [6, Theorem 5.6]), for any S , ∈ S P E −t2/2 ( gS ℓb(ℓd) gS ℓb (ℓd) t) 2e (t> 0). |k k 2 2 − k k 2 2 |≥ ≤ E Thus, the gS ℓb(ℓd) gS ℓb(ℓd) are mean-zero and subgaussian and therefore (see e.g. [23, Proposition k k 2 2 − k k 2 2 7.29]), 1/2 E E 1/2 b sup( gS ℓb (ℓd) gS ℓb (ℓd)) . (log card( )) = log s log(eb/s). k k 2 2 − k k 2 2 S s ≤ S∈S p Combining these estimates yields (29). We will also use the following technical lemma. It formalizes, in a very special case, the idea that the functions x x/ log(1/x) and y y log(1/y) can be treated as ‘inverse functions up to constants’. 7→ 7→ Lemma 6.5. Let C 1, x> 0, 0 Proof. As y K, we find ≤ x 1 1 1 . y ≤ Ce log(eK/y) ≤ Ce Since t t log(t) is decreasing on [0, 1/e], we obtain 7→ x y Cex log(eK/y)= Cex log(eK/x)+ Cey log(x/y) ≥ y 1 Cex log(eK/x)+ Cey log(1/(Ce)) = Cex log(eK/x) y log(Ce). ≥ Ce − Rearranging yields the asserted bound. Proposition 6.6. Set 0 < p 1 and p < q 2. There is an absolute constant C 1 such that for all m bd we have ≤ ≤ ≥ ≤ log(eb/m)+ d 1/p−1/q c (id : ℓb (ℓd) ℓb(ℓd)) min 1,Ce . m p 2 → q 2 ≤ m n o Proof. If m Ce(log(eb/m)+ d) then we can trivially bound ≤ c (id : ℓb (ℓd) ℓb(ℓd)) id : ℓb (ℓd) ℓb (ℓd) =1, m p 2 → q 2 ≤k p 2 → p 2 k which is the desired estimate. b d Assume now that m>Ce(log(eb/m)+ d). We apply Proposition 3.2 for K = B b d and Y = ℓ (ℓ ) to ℓp(ℓ2 ) q 2 k·k2 deduce the result. To this end, we determine for a given s N a suitable 0 <ρ< 1 such that Kρ Db,d,s. ∈ k·k2 ⊂ This will allow for an upper estimate of the Gaussian width w(Kρ ) in terms of s based on Lemma 6.4. Afterwards, we choose s appropriately for the given m. Let 0 θ 1 satisfy 1/q = θ/p + (1 θ)/2, that is ≤ ≤ − 1 1 1 1 −1 θ = . q − 2 p − 2 If x Bℓb (ℓd), then for any x Kρ we have ∈ p 2 ∈ θ 1−θ 1−θ ρ< x b d x b d x b d x b d ℓq(ℓ2 ) ℓ (ℓ ) ℓ (ℓ ) ℓ (ℓ ) k k ≤k k p 2 k k 2 2 ≤k k 2 2 or −1 −1/(1−θ) x ℓb (ℓd) ρ . k k 2 2 ≤ Let 0 η 1 satisfy 1 = η/p + (1 η)/2, that is, ≤ ≤ − 1 1 1 −1 η = . 2 p − 2 k·k2 Then, for any y = x/ x ℓb (ℓd) Kρ we have k k 2 2 ∈ η 1−η x b d x ℓb (ℓd) x ℓb(ℓd) ℓ1(ℓ2 ) k k p 2 k k 2 2 −η −η/(1−θ) y b d = k k x b d ρ . ℓ1(ℓ2 ) ℓ (ℓ ) k k x ℓb (ℓd) ≤ x ℓb(ℓd) ≤k k 2 2 ≤ k k 2 2 k k 2 2 Note that η 1 1 1 −1 = . 1 θ 2 p − q − −(1/p−1/q) k·k2 Therefore, if we set ρ = s , then Kρ D and by Lemma 6.4, ⊂ b,d,s k·k2 √ w(Kρ ) . s log(eb/s)+ sd. Proposition 3.2 implies that p c (id : ℓb (ℓd) ℓb(ℓd)) s−(1/p−1/q) m p 2 → q 2 ≤ provided that s satisfies m & s log(eb/s)+ sd. Set C 1. Define s = m/(Ce(log(eb/m)+ d)) , so that s N and ≥ ⌊ ⌋ ∈ m m s = . ≤ Ce(log(eb/m)+ d) Ce log(ebed/m) 16 SJOERD DIRKSEN AND TINO ULLRICH Since m bed, we can now apply Lemma 6.5 with K = bed to obtain ≤ Ce Ce m s log(ebed/s)= (s log(eb/s)+ sd). ≥ 1 + log(Ce) 1 + log(Ce) Thus, the desired condition on m is satisfied if we pick C 1 large enough. ≥ We now prove the lower bound in (26). Proposition 6.7. Set 0 4s 1/p 4s 1/q x p (4s)1/p−1/q x q for all x ,...,x Rd k kk2 ≤ k kk2 1 4s ∈ Xk=1 kX=1 we see that 1/p−1/q κX,Y,S+ = (4s) . Therefore, if 1/p−1/q 1 −1 cp,q (2αℓb (ℓd)) , ≤ 4 p 2 then by assumption b d b d 1/p−1/q −1 −(1/p−1/q) cm(id : ℓp(ℓ2) ℓq(ℓ2)) Setting −1 1+1/p 2 cp = log(αℓb (ℓd) +2 α b d βℓb (ℓd)) , p 2 ℓp(ℓ2 ) p 2 it follows from (30) that m c s log(b/32s) c s log(c−(1/p−1/q)/32). ≥ p ≥ p p,q Hence, by picking c small enough, we can ensure that m 32s. Using this in (31) yields p,q ≥ s log(b/m)+(8e)−1sd mc−1. ≤ p We now rearrange to find 1 m c (s log(b/m)+(8e)−1sd) > µ(m)−1c (log(b/m)+(8e)−1d) ≥ p 2 p 1 −1 1 cp(log(b/m)+(8e) d) −1 = c (log(b/m)+(8e)−1d) min 1, 2 2 p m m, n o ≥ which is the desired contradiction. Remark 6.8. From the previous proof it is clear that for 0 be the set of all 2t-inner sparse support sets, so that + is the set of all 4t-inner sparse support sets. By H¨older’s inequality, we have the sharp inequality S x (4t)1/q−1/p x , for all S and x Rbd, k Skℓp(ℓq ) ≤ k Skℓp(ℓp) ∈ S+ ∈ and therefore 1/q−1/p κX,Y,S+ = (4t) . Hence, if 1/q−1/p 1 −1 cp,q < (2αℓb (ℓd)) 4 p q then by assumption b d b d 1/q−1/p −1 −(1/q−1/p) cm(id : ℓ (ℓ ) ℓ (ℓ )) bt d 1+1/p 2 (33) log m log(αℓb (ℓd) +2 αℓb (ℓd)βℓb (ℓd)). 32 8t ≤ p q p q p q Define −1 1+1/p 2 c˜ := log(α b d +2 α b d β b d ). p,q ℓp(ℓq ) ℓp(ℓq ) ℓp(ℓq ) By (32) we can pick cp,q small enough to arrange that 1 d log c˜ 8e 32 8t p,q ≥ and, as a consequence, m 8ebt. Combining this with (33) we find ≥ c˜ ebd c˜ ebd m p,q tb log > p,q µ(m)−1b log ≥ 32 m 64 m ebd c˜ ebd c˜p,qb log m −1 = p,q b log min 1, 64 m 64m m, n o ≥ which is the desired contradiction. Finally, we prove statement (iii) in Theorem 6.1. In contrast to the results in (i) and (ii), this is a truly mixed situation in which both the outer and the inner sparsity have a visible effect on the bounds. Proposition 6.10. Set 0 < q 1, q p 1 and m bd. Then, there is a constant c˜ , so that ≤ ≤ ≤ ≤ p,q 1 :1 m c˜ log ebd , ≤ ≤ p,q m 1/p−1/2 log(ebd/m) c (id : ℓb (ℓd) ℓb (ℓd)) :˜c log ebd m c˜ b log ebd , m p q → 2 2 ≃p,q m p,q m ≤ ≤ p,q m 1/q−1/2 b log(ebd/m) b1/2−1/p :˜c b log ebd m bd . m p,q m ≤ ≤ Proof. We define the constantc ˜p,q explicitly in (37). We distinguish three cases. ֒ (Case 1 m (˜c /2) log(ebd/m). The result trivially follows from the norm-1 embeddings ℓb(ℓd ≤ ≤ p,q q q → .(ℓb (ℓd) ֒ ℓb (ℓd), (S3) and (27 p q → p p Case (˜c /2) log(ebd/m) m (˜c b/16) log(ebd/m). Again, the upper bound is immediate from p,q ≤ ≤ p,q id : ℓb (ℓd) ℓb (ℓd) = 1, (S3) and (27). The lower bound follows using the proof technique used for k p q → p p k Proposition 6.7, but now setting t = 1 instead of t = d/(8e). We include the details for the reader’s convenience. We prove the lower bound by contradiction using Proposition 4.4, so suppose that c (id : ℓb (ℓd) ℓb (ℓd)) where 1 c˜p,q log(ebd/m) µ(m) := 2 m and c , c˜ are to be determined below. Set s = µ(m)−1 . By our assumption p,q p,q ⌊ ⌋ (˜c /2) log(ebd/m) m (˜c b/16) log(ebd/m), p,q ≤ ≤ p,q it follows that s N and ∈ 1 m b (34) s = 1 , ≤ µ(m) 2 c˜p,q log(ebd/m) ≤ 8 so we can apply Proposition 5.3 further below. Moreover, for x ker(A), x = 0, we have by assumption ∈ 6 1/p−1/2 1/q−1/2 1/p−1/2 1/q−1/2 −(1/p−1/2) x ℓb (ℓd) b d x ℓb (ℓd) b d cp,qs x ℓb (ℓd) k k p q ≤ k k 2 2 ≤ k k p q so in particular, − 1/q−1/2 (35) b/s c−1/(1/p−1/2)d 1/p−1/2 . ≥ p,q We apply Proposition 4.4 for X = ℓb (ℓd) and Y = ℓb (ℓd). Let be the set of all (2s, 2)-sparse support sets, p q 2 2 S so that is the set of all (4s, 4)-sparse support sets. By the sharp inequality S+ x (4s)1/p−1/241/q−1/2 x , for all S and x Rbd, k Skℓp(ℓq ) ≤ k Skℓ2(ℓ2) ∈ S+ ∈ it follows that 1/p−1/2 1/q−1/2 κX,Y,S+ = (4s) 4 . Therefore, if 1/p−1/2 1/q−1/2 1 1 −1 cp,q (2αℓb (ℓd)) , ≤ 4 4 p q then by assumption b d b d 1/p−1/2 −1 −(1/p−1/2) −(1/q−1/2) cm(id : ℓ (ℓ ) ℓ (ℓ )) 1 Since s> 2µ(m) , we now find 1 m c˜ s log(ebd/m) > µ(m)−1c˜ log(ebd/m)= m, ≥ p,q 2 p,q which is the desired contradiction. Case (˜c b/16) log(ebd/m) m bd. Both, the lower and the upper bound follow from factorization of p,q ≤ ≤ the identity and (S3). Namely, id : ℓb (ℓd) ℓb (ℓd) = b1/q−1/p and id : ℓb(ℓd) ℓb (ℓd) = b1/p−1/2. Part k p q → q q k k 2 q → p q k (i) and (ii) of Theorem 6.1 together with (S3) imply that b log(ebd/m) 1/q−1/2 b1/2−1/p min 1, b1/2−1/pc (id : ℓb (ℓd) ℓb (ℓd)) m ≃p,q m 2 q → 2 2 n o c (id : ℓb (ℓd) ℓb (ℓd)) ≤ m p q → 2 2 b1/q−1/pc (id : ℓb(ℓd) ℓb (ℓd)) ≤ m q q → 2 2 log(ebd/m) 1/q−1/2 b1/q−1/p min 1, . ≃p,q m The lower and upper bound match if m b log(ebd/m). n o ≥ Remark 6.11. There are several remaining open cases for which it would be interesting to prove matching bounds for the Gelfand numbers: (i) It would be very interesting to extend our result in (26) to the case of differing inner spaces. That b d b d is, one would like to have optimal bounds for cm(id : ℓp(ℓq ) ℓr(ℓu)) for p 1, p < r and q 7. Applications 7.1. Signal processing. From the perspective of signal processing, Gelfand widths (see Remark 2.3) are of interest due to their close connection to the compressive m-widths, defined in (18) above. As was discussed in Section 2.1, we have b d b d cm(B b d ,ℓ (ℓ )) p,q,r,u Em(B b d ,ℓ (ℓ )). ℓp(ℓq ) r u ≃ ℓp(ℓq ) r u Thanks to this equivalence, we can use Theorem 1.1 to derive Corollary 1.2. Its consequences for structured signal recovery were discussed before in the introduction. Proof of Corollary 1.2. By (7), outer σ (x) b d s ℓ1(ℓ2 ) D sup x ∆(Ax) b d D sup . ℓ2(ℓ2 ) x∈B b d k − k ≤ x∈B b d √s ≤ √s ℓ1(ℓ2 ) ℓ1(ℓ2 ) Theorem 1.1 therefore implies that for some c> 0 and C 1, ≥ Ce(log(eb/m)+ d) 1/2 D c min 1, c (id : ℓb (ℓd) ℓb (ℓd)) . m ≤ m 1 2 → 2 2 ≤ √s n o This means that either D2 D2 m s or s . ≤ c2 ≤ c2 Ce(log(eb/m)+ d) GELFAND NUMBERS, STRUCTURED SPARSITY, BESOV SPACE EMBEDDINGS 21 Thus, if we assume that s D2/c2, then we obtain ≥ D2 m D2 m s = ≤ c2 Ce(log(eb/m)+ d) c2Ce log(ebed/m) We now apply Lemma 6.5 with y = m, x = c2s/D2 and K = bed to obtain Ce c2s m log(eD2bed/c2s) & s log(eb/s)+ sd. ≥ 1 + log(Ce) D2 Suppose now that (8) holds. Then, inner σ (x) b d t ℓ2 (ℓ1 ) D sup x ∆(Ax) b d D sup . ℓ2(ℓ2 ) x∈B b d k − k ≤ x∈B b d √t ≤ √t ℓ2(ℓ1 ) ℓ2(ℓ1 ) Theorem 1.1 now yields for some c> 0 and C 1, ≥ 1/2 Ceb log(ebd/m) b d b d D c min 1, cm(id : ℓ (ℓ ) ℓ (ℓ )) . m ≤ 2 1 → 2 2 ≤ √t n o Therefore, either D2 D2 m t or t . ≤ c2 ≤ c2 Ceb log(ebd/m) Assuming that t D2/c2, we find ≥ D2 (m/b) t ≤ c2Ce log(ed/(m/b)) We now apply Lemma 6.5 with y = m/b, x = c2t/D2 and K = d to obtain m Ce c2t log(eD2bd/c2t) b ≥ 1 + log(Ce) D2 and as a consequence, m & bt log(ed/t). In the introduction we claimed that if B is standard Gaussian and A = (1/√m)B, then the pair (A, ∆ℓ1(ℓ1)) with high probability satisfies the stable reconstruction guarantee (8), provided that m & bt log(ed/t). By Corollary 1.2 this is optimal. Here we provide details on this claim for the convenience of the reader. We know that (A, ∆ℓ1(ℓ1)) with high probability satisfies ([10, Theorem 1.2 and Section 1.3], see also [11, 14] and [23, Theorem 9.13]) σk(x) b d ℓ1 (ℓ1 ) b d (38) x ∆ℓ1(ℓ1)(Ax) ℓ (ℓ ) D , k − k 2 2 ≤ √k whenever m & k log(ebd/k), where σk(x)ℓb (ℓd) = inf x z ℓb(ℓd) : z is k -sparse . 1 1 {k − k 1 1 } Set k = bt. By H¨older’s inequality and by noting that every t-inner sparse vector is bt-sparse, we find σbt(x)ℓb (ℓd) = inf x z ℓb(ℓd) : z is bt -sparse 1 1 {k − k 1 1 } √ b inf x z ℓb(ℓd) : z is bt -sparse ≤ {k − k 2 1 } √ √ inner b inf x z ℓb(ℓd) : z is t -inner sparse = bσt (x)ℓb (ℓd). ≤ {k − k 2 1 } 2 1 Therefore, by (38), inner σbt(x) b d σ (x) b d ℓ1(ℓ1 ) t ℓ2(ℓ1 ) b d x ∆ℓ1(ℓ1)(Ax) ℓ (ℓ ) D D , k − k 2 2 ≤ √bt ≤ √t provided that m & k log(ebd/k)= bt log(ed/t). 22 SJOERD DIRKSEN AND TINO ULLRICH 7.2. Gelfand numbers of Besov space embeddings with small mixed smoothness. The above results in finite dimensional mixed-norm spaces have a direct application to Besov space embeddings with small mixed smoothness. In this section we are interested in the Gelfand numbers of the embeddings (39) Id : Sr0 B(Ω) Sr1 B(Ω) , p0,q0 → p1,q1 where Ω Rd is a bounded domain (open set) and 0 < p p and 0 < q < q such that we ⊂ 0 ≤ 1 ≤ ∞ 0 1 ≤ ∞ are in the small smoothness regime, i.e. 1/p0 1/p1 < r0 r1 1/q0 1/q1 . The goal of this section is to show that the results in Theorem 6.1 imply − − ≤ − r0 r1 −(r0−r1) (40) cm(Id : S B(Ω) S B(Ω)) θ m p0,q0 → p1,q1 ≃ in some of the parameter constellations in the small smoothness regime. Here and below we write (41) θ := p ,p , q , q , r , r , d, Ω { 0 1 0 1 0 1 } to abbreviate the dependency on the various parameters. Observe that the decay behavior in (40) is typical for the univariate situation (d = 1). Usually, in the d- −(r0−r1) (d−1)η variate setting we encounter asymptotic orders such as m (log m) for some η := η(p0, q0,p1, q1, r0, r1), see Remark 7.8 and [13, 46]. The dimension d of the underlying Euclidean space enters the rate of conver- gence exponentially. Surprisingly, this is no longer the case in the small smoothness regime as Corollaries 7.5, 7.6 and 7.7 below will show. However, we emphasize that dimension d is still hidden in the constants. We start with some definitions. We denote by Lp(Ω), 0 < p , the space of all measurable complex- valued functions f :Ω C where ≤ ∞ → 1/p p f p := f(x) dx k k Ω | | Z is finite (with the usual modification if p = ). The space of tempered distributions on Rd is denoted with S′(Rd). For 0 < p,q and r R the Besov∞ space Sr B(Rd) is defined as ≤ ∞ ∈ p,q 1/q r d ′ d rqk¯jk1 −1 q (42) S B(R ) := f S (R ): f Sr B := 2 [ϕ¯ f] < , p,q ∈ k k p,q kF j F kp ∞ ¯j∈Nd n X0 o (with the usual modification in case q = ) where the system ϕ¯j ¯j∈Nd is the standard tensorized dyadic ∞ { } 0 decomposition of unity, see [63] and [55]. Important for our purpose is the following diagonal embedding result, see [55, Chapt. 2], (Sr0 B(Rd) ֒ Sr1 B(Rd (43) p0,q0 → p1,q1 for p0 p1 and r0 r1 > 1/p0 1/p1 . The embedding (43) is never compact which is the reason why we will≤ restrict to spaces− on bounded− domains in the sequel. This guarantees a decay of the corresponding Gelfand numbers of the embedding. Before doing so, let us comment on the discretization of the above function spaces on Rd developed in [63, Thm. 2.10]. Using sufficiently smooth wavelets with sufficiently many vanishing moments (and the notation from [63]) the mapping d d (44) f λ¯ ¯(f) := f, ψ¯ ¯ , ¯j N , k¯ Z , 7→ j,k h j,ki ∈ 0 ∈ r Rd represents a sequence spaces isomorphism between Sp,qB( ) and q/p 1/q r (r−1/p)qk¯jk1 p (45) s b := λ = λ¯ ¯ ¯ ¯ C : λ sr b := 2 λ¯ ¯ < p,q { j,k}j,k ⊂ k k p,q | j,k| ∞ ¯j∈Nd k¯∈Zd n h X0 X i o (with the usual modification in case max p, q = ) . We want to study Gelfand numbers of compact function space embeddings on domains. Let{ Ω} be an∞ arbitrary bounded domain in Rd, D(Ω) denote the set ′ r of test functions and D (Ω) the corresponding set of distributions. Then Sp,qB(Ω) is defined as the set of r Rd restrictions (in the distributional sense) of elements of Sp,qB( ) to Ω, i.e., Sr B(Ω) := f D′(Ω) : g Sr B(Rd) such that g = f p,q { ∈ ∃ ∈ p,q |Ω } and its (quasi-)norm is given by f r := inf g Sr B, see [63, Sect. 3.1], [31, 3.2.3] . The k kSp,qB(Ω) g|Ω=f k k p,q embedding (43) transfers to the domain Ω and is compact so that the Gelfand numbers decay and converge GELFAND NUMBERS, STRUCTURED SPARSITY, BESOV SPACE EMBEDDINGS 23 to zero. We are interested in establishing the decay rate especially for the embedding (39) in the important special case Ω = [0, 1]d, which is representative for all tensor product domains. Using the boundedness of certain restriction and extension operators, see [63, Sect. 4.5], [31, 3.2.3], together with the wavelet isomorphism (44) between (42) and (45), we can restrict ourselves to the study of embeddings r ¯ Nd ¯ Zd d −ji between the “restricted sequence spaces” sp,qb(Ω). For j 0 and k let Q¯j,k¯ = i=12 [ki 1, ki + 1]. Ω d ∈ ∈ × − Let A = k¯ Z : Q¯ ¯ Ω = . Consider the sequence space ¯j { ∈ j,k ∩ 6 ∅} r C sp,qb(Ω) := λ = (λ¯j,k¯)¯j∈Nd,k¯∈AΩ : λ sr b(Ω) < { 0 ¯j ⊂ k k p,q ∞} with (quasi-)norm given by q/p 1/q k¯jk1(r−1/p)q p (46) λ sr b(Ω) := 2 λ¯ ¯ . k k p,q | j,k| ¯j∈Nd k¯∈AΩ h X0 X¯j i Let us start with the following result, where the upper and lower bound only differ by a power of log log m. Proposition 7.1. Let 0 < q0 p0 1, 0 µ + d 1 ¯ (49) ♯ ¯j Nd : ¯j = µ = − µd−1 and ♯AΩ 2kjk1 { ∈ 0 k k1 } µ ≃d ¯j ≃Ω µ d−1 such that Dµ d,Ω 2 µ . Now we fix some J N and consider four different ranges in the sum on the right-hand side≃ in (48), i.e., ∈ J L M ∞ (50) c (id)̺ c (id )̺ + c (id )̺ + c (id )̺ + id ̺ . m ≤ mµ+1 µ mµ+1 µ mµ+1 µ k µk µ=0 X µ=XJ+1 µ=XL+1 µ=XM+1 Here we put L = J + (d 1) log J and M will depend on L as determined below. The main challenge is to − 2 choose the mµ suitably and estimate the corresponding cmµ (idµ) using the finite dimensional results from Section 6. Note, that we deal with ℓq(ℓp) spaces of complex numbers here. This technical issue only causes a multiplicative constant in the estimates below. Step 2. To estimate the first sum we choose mµ =2Dµ which yields (see (S4)) J ̺ (51) cmµ+1(idµ) =0 . µ=0 X 24 SJOERD DIRKSEN AND TINO ULLRICH In addition, J m 2J J d−1 . In case m 2µµd−1 we have as a consequence of (49) µ=0 µ ≃d,Ω µ ≤ P d−1 µ d−1 µ (52) c (id ) . 2(−r+1/p0−1/p1)µ c id : ℓµ (ℓ2 ) ℓµ (ℓ2 ) . mµ µ θ mµ q0 p0 → q1 p1 µ d−1 If mµ > 2 µ then (52) may fail since the right-hand side vanishes due to (S4). Using straightforward embedding arguments for finite-dimensional mixed-norm spaces, the Gelfand numbers on the right-hand side can be estimated in two different ways using (27). On the one hand, we get µ d−1 µ d−1 −(r−1/p0+1/p1)µ µ(1/q0−1/p0) µ(1/p1−1/q1) 2 µ 2 µ cm (idµ) .θ 2 2 2 cm (id : ℓ ℓ ) µ µ q0 → q1 (53) µ d−1 1/q0−1/q1 −rµ µ(1/q0−1/q1) log(e2 µ /mµ) .θ 2 2 . mµ On the other hand, µ d−1 µ d−1 −(r−1/p0+1/p1)µ 2 µ 2 µ cm (idµ) .θ 2 cm (id : ℓ ℓ ) µ µ p0 → p1 (54) µ d−1 1/p0−1/p1 −(r−1/p0+1/p1)µ log(e2 µ /mµ) .θ 2 . mµ Let us assume for the moment that r< 1/q 1/q . In the second sum in (50) we choose m =2µ2(L−µ)κ with 0− 1 µ κ< 1 such that r<κ(1/q 1/q ). Clearly, m 2µµd−1 and, due to κ< 1, L m 2L =2J J d−1. 0 − 1 µ ≤ µ=J+1 µ ≃κ Employing (53) gives P L L (1/q −1/q )̺ ̺ −µr̺ (µ−L)κ d−1 (µ−L)κ 0 1 (55) cmµ+1(idµ) .θ 2 2 (log eµ 2 ) . µ=XJ+1 µ=XJ+1 h i Since we assumed r< 1/q 1/q , we obtain 0 − 1 L (56) c (id )̺ . 2−Lr̺(log L)(1/q0−1/q1)̺ (2J J d−1)−r̺(log J)(1/q0 −1/q1)̺ . mµ+1 µ θ ≃θ µ=XJ+1 In the case r =1/q 1/q we choose m =2J µd−1 in the range J<µ L. This gives m 2µµd−1 and 0 − 1 µ ≤ µ ≤ L L (57) m 2J J d−1 1 2J J d−1 log J. µ ≃d ≃d µ=XJ+1 µ=XJ+1 Instead of (55) we obtain L L (1/q0−1/q1)̺ c (id )̺ . 2−J µ−(d−1)(µ J) mµ+1 µ θ − (58) µ=J+1 µ=J+1 X X h i −r̺ 2J J d−1 (log J)(1/q0−1/q1)̺ log J. ≃θ h i Now we discuss the third sum in (50), which we only need in case p0 have mµ = 0. Using (54) we obtain M M µ d−1 (1/p0−1/p1)̺ ̺ −(r−1/p0+1/p1)µ̺ log(e2 µ /mµ) cmµ+1(idµ) .θ 2 mµ µ=XL+1 µ=XL+1 ∞ (1/p −1/p )̺ −rµ̺ (µ−L)β (µ−L)β d−1 0 1 .θ 2 2 log(2 µ ) µ=L+1 (59) X h i ∞ (1/p0−1/p1)̺ . 2−rµ̺ 2(µ−L)β((µ L)β + log µ) θ − µ=XL+1 h i 2−rL̺(log L)(1/p0−1/p1)̺ ≃θ (2J J d−1)−r̺(log J)(1/p0−1/p1)̺ . ≃θ Let us finally care for the fourth sum in (50). It holds ∞ ∞ id ̺ . 2−(r−1/p0+1/p1)µ̺ k µk θ µ=M+1 µ=M+1 (60) X X 2−(r−1/p0+1/p1)M̺ ≃θ 2−L(r−1/p0+1/p1)β/(β−1)̺ . ≃θ Note, that r>β(1/p 1/p ) for β > 1 implies (r 1/p +1/p )β/(β 1) > r. Then it holds 0 − 1 − 0 1 − ∞ (61) id ̺ . 2−Lr̺ (2J J d−1)−r̺ . k µk θ ≃θ µ=XM+1 Altogether, (51), (56), (59), and (61) yield in case 1/p 1/p ∞ (63) m := m 2J J d−1 J µ ≃θ µ=0 X ∞ and a standard monotonicity argument to fill the gaps between mJ J=1. Similarly, in the endpoint case r =1/q 1/q the estimates (57) and (58) imply { } 0 − 1 r 0 −r 1/q0−1/q1 r+1/̺ (64) cm(id : s b(Ω) s b(Ω)) .θ m (log log m) (log log m) . p0,q0 → p1,q1 2 2 Step 3 (lower bound). The argument given by Vyb´ıral in [63, Thm. 3.18, Step. 2] for proving the same lower bound for entropy numbers almost literally applies to Gelfand numbers by taking the lower bound in Theorem 6.1, (27) into account (instead of [63, Lem. 3.11]). In case p0 = p1 = 2 we obtain the following sharp result which turns out to be a consequence of our new improved block result in Theorem 6.1, (26). The main benefit is that we do not produce the term 1/q0−1/q1 (log log2 m) , see (47), which is an artefact of the log-term in (27). Compared to Proposition 7.1, we can get beyond the restriction p 1, but now need to impose p = p = 2 . 0 ≤ 0 1 Proposition 7.2. Let 0 < q0 1, q0 < q1 2 and 0 Proof. We return to (50) where we drop the third sum, i.e., M = L = J +(d 1) log J. The range µ =0, ..., J − 2 is treated completely analogous as above. In the range µ = L +1, ... we chose mµ = 0 and estimate the sum over the operator (quasi-)norms of idµ. This gives ∞ ∞ (66) id ̺ . 2−µr̺ (2J J d−1)−r̺ . k µk θ ≃θ µ=XL+1 µ=XL+1 Let us deal with the range µ = J +1, ..., L . By Theorem 6.1, (26) we have, in contrast to (53) above, the improved bound µ d−1 1/q −1/q −rµ 2 + log(µ /mµ) 0 1 (67) cmµ (idµ) θ 2 , ≃ mµ which results in L L ̺ −µr̺ (µ−L)κ(1/q0−1/q1)̺ cmµ+1(idµ) .θ 2 2 (68) µ=XJ+1 µ=XJ+1 2−Lr̺ (2J J d−1)−r̺ , ≃θ ≃θ µ (L−µ)κ when plugging in mµ =2 2 . The relations (51), (66), and (68) together with (63) imply (65) . What concerns the endpoint case r =1/q 1/q we are able to complement Propositions 7.1, 7.2 in case 0 − 1 p0 = p1 = 2 as follows . Proposition 7.3. Let 0 < q0 1, q0 < q1 2 and r =1/q0 1/q1. Let further ̺ = min 1, q1 . For m 2, we have ≤ ≤ − { } ≥ −r r 0 −r r+1/̺ (69) m .θ cm(id : s b(Ω) s b(Ω)) .θ m (log log m) . 2,q0 → 2,q1 2 J d−1 Proof. We insert mµ =2 µ into (67) and take (57) into account. Let us state our main result in this subsection. Theorem 7.4. Under the same restrictions as in Propositions 7.1, 7.2 and 7.3 the bounds for the Gelfand numbers literally transfer to the corresponding embedding Id : Sr B(Ω) S0 B(Ω) . p0,q0 → p1,q1 Proof. The result is a direct consequence of Propositions 7.1, 7.2 and 7.3 together with the machinery described in the proof of [63, Thm. 4.11] . All that is needed for this machinery are the properties (S1)-(S5), which are satisfied by both Gelfand and dyadic entropy numbers. Let us emphasize the following special cases. In connection with Besov space embeddings of mixed smoothness the target space Lp plays a particular role. Corollary 7.5. Let 0 < q 1 and 0 Let us also state the following endpoint result as a special case of Proposition 7.3 (take q0 = q, q1 = 2). Corollary 7.6. Let 0 < q 1 and r =1/q 1/2. Then we have for m> 2 ≤ − m−r . c (Id : Sr B(Ω) L (Ω)) . m−r(log log m)r+1 . θ m 2,q → 2 θ 2 Let us finally mention the following non-sharp bound. Corollary 7.7. Let 0 < q0 Remark 7.8. (i) In the case of entropy numbers Vyb´ıral proved in [63, Thm. 4.9] that for any ε> 0 there is a constant Cε > 0 such that (70) cm−(r0−r1) e (id : sr0 b(Ω) sr1 b(Ω)) C m−(r0−r1)(log m)ε , m 2, ≤ m p0,q0 → p1,q1 ≤ ε ≥ for 0 r0 r1 −(r0−r1) (d−1)(r0−r1−1/q0+1/q1) (72) em(id : s b(Ω) s b(Ω)) θ m (log m) . p0,q0 → p1,q1 ≃ r0 r1 ∗ In fact, the entropy numbers in (70) can be bounded from above by em(id : sp0,q b(Ω) sp1,q1 b(Ω)) if ∗ ∗ ∗ ∗ → q q0. Now choose q1 > q > q0 such that 1/q 1/q1 + ε/(d 1) = r0 r1 > 1/q 1/q1 which, together ≥ ∗ − − − − with (72) and q0 replaced by q , implies (70). The gap in (70) will be closed in a forthcoming paper [43] also −(r0−r1) in the endpoint case r =1/q0 1/q1. The correct order is m . (ii) Using almost literally the− arguments in [63, Thms. 3.18, 3.19] together with Theorem 6.1, (27) we can prove a direct counterpart of (72) for Gelfand numbers in the situation max p0, q0 1, max p1, q1 2, p p and (71), which is { } ≤ { } ≤ 0 ≤ 1 r0 r1 −(r0−r1) (d−1)(r0−r1−1/q0+1/q1) (73) cm(id : s b(Ω) s b(Ω)) θ m (log m) , m 2 . p0,q0 → p1,q1 ≃ ≥ However, the argument described after (72) does not apply for Gelfand numbers. Consider the situation of determining the Gelfand numbers of the embedding c (id : sr b(Ω) s0 b(Ω)) for r < 1/2. We m 2,1 → 2,2 choose q∗ > 1 such that 1/q∗ 1/2 = r. The problem reduces to the Gelfand numbers of the embedding r 0 − cm(id : s2,q∗ b(Ω) s2,2b(Ω)). Note that this situation is not covered by (73). Nevertheless, we may reduce → b d b d further to the finite-dimensional embedding id : ℓq∗ (ℓ2) ℓ2(ℓ2). The critical point here is that in case → b d b d min p0, q0 > 1 the entropy and Gelfand numbers of embeddings id : ℓ (ℓ ) ℓ (ℓ ) may differ in order { } q0 p0 → q1 p1 (already if 1 Remark 7.9. Note that in Corollary 7.7 the case p0 = q0 is excluded. In [47] Nguyen and Sickel studied Weyl numbers of embeddings of tensor product Besov spaces (p0 = q0) with small mixed smoothness into Lp1 . Replacing xn by cn in the proof of [47, Lem. 6.19] we obtain for 0 Remark 7.11. We doubt that log log m-terms in Propositions 7.1, 7.3 and Corollaries 7.6, 7.7 are necessary. Except in Proposition 7.2, where we actually use the sharp mixed-norm estimate (26), we reduce the problem to the classical finite-dimensional non-mixed situation (27) via standard embeddings. By doing so, we produce an additional log which becomes a log log at the end. To get an improvement, one needs to refine these estimates by working directly with the mixed norms. An associated mixed-norm result in the spirit d d d of (26) where the inner ℓ2-spaces are replaced by ℓp0 and ℓp1 would solve this issue. But this seems to be rather involved for Gelfand numbers and we leave it as an open problem. However, in the case of entropy numbers this is possible as we will show in a forthcoming paper, [43]. 28 SJOERD DIRKSEN AND TINO ULLRICH Acknowledgement The authors wish to thank S. Mayer for several fruitful discussions on this paper. They also acknowledge the fruitful comments by G. Byrenheid, V.K. Nguyen, E. Novak and H. Rauhut on the topic of this paper. The authors thank the reviewers for their detailed comments, which improved the presentation of the paper. T.U. gratefully acknowledges support by the German Research Foundation (DFG) Ul-403/2-1 and the Emmy- Noether programme, Ul-403/1-1. References [1] B. Adcock, A. C. Hansen, C. Poon, and B. Roman. Breaking the coherence barrier: asymptotic inco- herence and asymptotic sparsity in compressed sensing. CoRR, abs/1302.0561, 2013. [2] T. Aoki. Locally bounded linear topological spaces. Proc. Imp. Acad. Tokyo, 18:588–594, 1942. [3] U. Ayaz, S. Dirksen, and H. Rauhut. Uniform recovery of fusion frame structured sparse signals. Appl. Comput. Harmon. Anal., 41(2):341–361, 2016. [4] F. Bach, R. Jenatton, J. Mairal, and G. Obozinski. Optimization with sparsity-inducing penalties. Foundations and Trends R in Machine Learning, 4(1):1–106, 2012. [5] A. Bastounis and A. C. Hansen. On the absence of the RIP in real-world applications of compressed sensing and the RIP in levels. CoRR, abs/1411.4449, 2014. [6] S. Boucheron, G. Lugosi, and P. Massart. Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford University Press, 2013. [7] C. Boyer, J. Bigot, and P. Weiss. Compressed sensing with structured sparsity and structured acquisi- tion. Applied and Computational Harmonic Analysis, to appear, 2017. [8] P. B¨uhlmann and S. Van De Geer. Statistics for high-dimensional data: methods, theory and applications. Springer Science & Business Media, 2011. [9] G. Byrenheid and T. Ullrich. Optimal sampling recovery of mixed order Sobolev embeddings via discrete Littlewood-Paley type characterizations. Anal. Math., 43(2):133–191, 2017. [10] E. J. Cand´es, J. K. Romberg, and T. Tao. Stable signal recovery from incomplete and inaccurate measurements. Communications on Pure and Applied Mathematics, 59(8):1207–1223, 2006. [11] E. J. Cand´es and T. Tao. Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE Transactions on Information Theory, 52(12):5406–5425, Dec 2006. [12] A. Cohen, I. Daubechies, R. DeVore, G. Kerkyacharian, and D. Picard. Capturing ridge functions in high dimensions from point queries. Constr. Approx., 35(2):225–243, 2012. [13] D. D˜ung, V. Temlyakov, and T. Ullrich. Hyperbolic cross approximation. Advanced Courses in Math- ematics. CRM Barcelona. Birkh¨auser/Springer, to appear. arXiv:1601.03978v2. [14] D. Donoho. Compressed sensing. IEEE Trans. Inform. Theory, 52(4):1289–1306, 2006. [15] D. L. Donoho. For most large underdetermined systems of linear equations the minimal l1 solution is also the sparsest solution. Commun. Pure Appl. Math., 59(6):797–829, 2006. [16] D. E. Edmunds and J. Lang. Gelfand numbers and widths. J. Approx. Theory, 166:78–84, 2013. [17] Y. Eldar and H. B¨olcskei. Block-sparsity: Coherence and efficient recovery. In ICASSP, pages 2885– 2888. IEEE, 2009. [18] Y. Eldar, P. Kuppinger, and H. B¨olcskei. Block-sparse signals: uncertainty relations and efficient recovery. IEEE Trans. Signal Process., 58(6):3042–3054, 2010. [19] Y. C. Eldar and M. Mishali. Robust recovery of signals from a structured union of subspaces. IEEE Trans. Inform. Theory, 55(11):5302–5316, 2009. [20] M. Fornasier and H. Rauhut. Recovery algorithms for vector valued data with joint sparsity constraints. SIAM J. Numer. Anal., 46(2):577–613, 2008. [21] M. Fornasier, K. Schnass, and J. Vybiral. Learning functions of few arbitrary linear parameters in high dimensions. Found. Comput. Math., 12(2):229–262, 2012. [22] S. Foucart, A. Pajor, H. Rauhut, and T. Ullrich. The Gelfand widths of ℓp-balls for 0 < p 1. J. Complexity, 26(6):629–640, 2010. ≤ [23] S. Foucart and H. Rauhut. A Mathematical Introduction to Compressive Sensing. Birkh¨auser, Boston, 2013. [24] E.` M. Galeev. Kolmogorov diameters of classes of periodic functions of one and several variables. Izv. Akad. Nauk SSSR Ser. Mat., 54(2):418–430, 1990. GELFAND NUMBERS, STRUCTURED SPARSITY, BESOV SPACE EMBEDDINGS 29 [25] E.` M. Galeev. Linear widths of H¨older-Nikol′ski˘ıclasses of periodic functions of several variables. Mat. Zametki, 59(2):189–199, 317, 1996. [26] A. Y. Garnaev and E. D. Gluskin. The widths of a Euclidean ball. Dokl. Akad. Nauk SSSR, 277(5):1048– 1052, 1984. [27] E. Gilbert. A comparison of signalling alphabets. Bell System Tech. Jour., 31:504–522, 1952. [28] E. D. Gluskin. On some finite-dimensional problems in the theory of widths. Vestnik Leningrad Univ. Math, 14(163-170):163–170, 1982. [29] Y. Gordon. On Milman’s inequality and random subspaces which escape through a mesh in Rn. In Geometric aspects of functional analysis (1986/87), volume 1317 of Lecture Notes in Math., pages 84– 106. Springer, Berlin, 1988. [30] R. Gribonval, H. Rauhut, K. Schnass, and P. Vandergheynst. Atoms of all channels, unite! Average case analysis of multi-channel sparse recovery using greedy algorithms. J. Fourier Anal. Appl., 14(5):655–687, 2008. [31] M. Hansen and W. Sickel. Best m-term approximation and Sobolev-Besov spaces of dominating mixed smoothness—the case of compact embeddings. Constr. Approx., 36(1):1–51, 2012. [32] A. Hinrichs, A. Kolleck, and J. Vyb´ıral. Carl’s inequality for quasi-Banach spaces. J. Funct. Anal., 271(8):2293–2307, 2016. [33] J. Huang and T. Zhang. The benefit of group sparsity. The Annals of Statistics, 38(4):1978–2004, 2010. [34] A. D. Izaak. Kolmogorov widths in finite-dimensional spaces with mixed norm. Mat. Zametki, 55(1):43– 52, 156, 1994. [35] A. D. Izaak. Widths of H¨older-Nikol′ski˘ıclasses and finite-dimensional sets in spaces with a mixed norm. Mat. Zametki, 59(3):459–461, 1996. [36] B. S. Kashin. Diameters of some finite-dimensional sets and classes of smooth functions. Izvestiya Rossiiskoi Akademii Nauk. Seriya Matematicheskaya, 41(2):334–351, 1977. [37] B. S. Kashin. Widths of Sobolev classes of small-order smoothness. Vestnik Moskov. Univ. Ser. I Mat. Mekh., (5):50–54, 1981. [38] T. K¨uhn. A lower estimate for entropy numbers. J. Approx. Theory, 110(1):120–124, 2001. [39] T. K¨uhn, S. Mayer, and T. Ullrich. Counting via entropy: new preasymptotics for the approximation numbers of Sobolev embeddings. SIAM Journ. Num. Analysis, 54(6):3625–3647, 2016. [40] C. Li and B. Adcock. Compressed sensing with local structure: uniform recovery guarantees for the sparsity in levels class. CoRR, abs/1601.01988, 2016. [41] K. Lounici, M. Pontil, S. Van De Geer, and A. B. Tsybakov. Oracle inequalities and optimal inference under group sparsity. The Annals of Statistics, 39(4):2164–2204, 2011. [42] Y. V. Malykhin and K. S. Ryutin. The product of octahedra is badly approximated in the ℓ2,1-metric. Mat. Zametki, 101(1):85–90, 2017. [43] S. Mayer and T. Ullrich. On a theorem of Edmunds and Netrusov and implications for entropy numbers of Besov space embeddings in regimes of small mixed smoothness. Preprint Bonn, 2017. [44] S. Mayer, T. Ullrich, and J. Vyb´ıral. Entropy and sampling numbers of classes of ridge functions. Constr. Approx., 42(2):231–264, 2015. [45] S. N. Negahban, P. Ravikumar, M. J. Wainwright, and B. Yu. A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers. Statist. Sci., 27(4):538–557, 11 2012. [46] V. Nguyen. Gelfand numbers of embeddings of mixed Besov spaces. J. Complexity, 41:35–57, 2017. [47] V. Nguyen and W. Sickel. Weyl numbers of embeddings of tensor product Besov spaces. J. Approx. Theory, 24(200):170–220, 2015. [48] A. Pietsch. Operator ideals, volume 20 of North-Holland Mathematical Library. North-Holland Publish- ing Co., Amsterdam-New York, 1980. Translated from German by the author. [49] A. Pietsch. Eigenvalues and s-numbers, volume 13 of Cambridge Studies in Advanced Mathematics. Cambridge University Press, Cambridge, 1987. [50] A. Pinkus. N-widths in approximation theory, volume 7 of Ergebnisse der Mathematik und ihrer Gren- zgebiete (3) [Results in Mathematics and Related Areas (3)]. Springer-Verlag, Berlin, 1985. [51] Y. Plan and R. Vershynin. One-bit compressed sensing by linear programming. Comm. Pure Appl. Math., 66(8):1275–1297, 2013. 30 SJOERD DIRKSEN AND TINO ULLRICH [52] Y. Plan and R. Vershynin. Robust 1-bit compressed sensing and sparse logistic regression: a convex programming approach. IEEE Trans. Inform. Theory, 59(1):482–494, 2013. [53] S. Rolewicz. On a certain class of linear metric spaces. Bull. Acad. Polon. Sci. Cl. III., 5:471–473, XL, 1957. [54] W. Rudin. Functional analysis, second ed. McGraw-Hill, New York, St.Louis, San Francisco, 1991. [55] H.-J. Schmeisser and H. Triebel. Topics in Fourier analysis and function spaces. A Wiley-Interscience Publication. John Wiley & Sons, Ltd., Chichester, 1987. [56] S. B. Stechkin. On absolute convergence of orthogonal series. Dokl. AN SSSR, 102:37–40, 1955. [57] V. Temlyakov. Constructive sparse trigonometric approximation for functions with small mixed smooth- ness. Constr. Approx., 45(3):467–495, 2017. [58] V. N. Temlyakov. Approximation of functions with bounded mixed derivative. Trudy Mat. Inst. Steklov., 178:1–112, 1986. [59] J. F. Traub, G. W. Wasilkowski, and H. Wo´zniakowski. Information-based complexity. Computer Science and Scientific Computing. Academic Press, Inc., Boston, MA, 1988. With contributions by A. G. Werschulz and T. Boult. [60] M. Ullrich and T. Ullrich. The role of Frolov’s cubature formula for functions with bounded mixed derivative. SIAM J. Numer. Anal., 54(2):969–993, 2016. [61] R. Varshamov. Estimate of the number of signals in error correcting codes. Dokl. Acad. Nauk SSSR, 117:739–741, 1952. [62] A. Vasil’eva. Kolmogorov and linear widths of the weighted Besov classes with singularity at the origin. Journ. Approx. Theory, 167:1–41, 2013. [63] J. Vyb´ıral. Function spaces with dominating mixed smoothness. Dissertationes Math. (Rozprawy Mat.), 436:73, 2006. [64] J. Vyb´ıral. Widths of embeddings in function spaces. J. Complexity, 24(4):545–570, 2008. [65] M. J. Wainwright. Structured regularizers for high-dimensional problems: Statistical and computational issues. Annual Review of Statistics and Its Application, 1:233–253, 2014. RWTH Aachen University, Lehrstuhl C fur¨ Mathematik (Analysis), Pontdriesch 10, 52062 Aachen, Germany E-mail address: [email protected] University of Bonn, Hausdorff Center for Mathematics, Endenicher Allee 62, 53115 Bonn, Germany E-mail address: [email protected]