Native Banach Spaces for Splines and Variational Inverse Problems∗

Native Banach spaces for splines and variational inverse problems∗

Michael Unser and Julien Fageot†

April 25, 2019

Abstract We propose a systematic construction of native Banach spaces for general spline-admissible operators L. In short, the native space for L and the (dual) norm 0 is the largest space of functions f : d k · kX R R such that Lf 0 < , subject to the constraint that the growth-restricted→ nullk spacekX of∞L be ﬁnite-dimensional. This space,

denoted by L0 , is specified as the dual of the pre-native space L, which is itself obtainedX through a suitable completion process. TheX main difference with prior constructions (e.g., reproducing kernel Hilbert spaces) is that our approach involves test functions rather than sums of atoms (e.g., kernels), which makes it applicable to a much broader class of norms, including total variation. Under specific admissibility and compatibility hypotheses, we lay out the direct-sum topology of

L and L0 , and identify the whole family of equivalent norms. Our constructionX X ensures that the native space and its pre-dual are endowed with a fundamental Schwartz-Banach property. In practical terms,

this means that L0 is rich enough to reproduce any function with an arbitrary degreeX of precision.

Contents

1 Introduction 2 arXiv:1904.10818v1 [math.FA] 24 Apr 2019

2 Preliminaries 6 2.1 Schwartz-Banach spaces ...... 6 2.2 Characterization of linear operators and their adjoint . . . . . 10 ∗The research leading to these results has received funding from the Swiss National Science Foundation under Grant 200020-162343. †Biomedical Imaging Group, École polytechnique fédérale de Lausanne (EPFL), Sta- tion 17, CH-1015, Lausanne, Switzerland ([email protected]).

1 3 Spline-admissible operators 12 3.1 Biorthogonal system for the null space of L ...... 14 3.2 Speciﬁcation of a suitable pseudo-inverse operator ...... 17

4 Direct-sum topology of the native space 20 4.1 Pre-dual of the native space ...... 21 4.2 Native space ...... 26 4.3 Equivalent topologies and biorthogonal systems ...... 30

5 Link with classical results 36 5.1 Operator-based solution of diﬀerential equations ...... 36 5.2 Total variation and BV ...... 37 5.3 Sobolev/Beppo-Levi spaces with d = 1 ...... 37 5.4 Reproducing-kernel Hilbert spaces ...... 38 5.5 Connections with kernel methods and splines ...... 39

1 Introduction

d Given a series of data points (xm, ym) R R, the basic interpolation d ∈ × problem is to find a function f : R R such that f(xm) = ym for m = → 1,...,M. In order to make the problem well posed, one needs to impose additional constraints on f; for instance that f is in the linear span of a finite number of known basis functions (standard regression problem), or that the desired function minimizes some energy functional. The variational formulation of splines follows the latter strategy and ensures the existence and unicity of the solution for energy functionals of the form Lf 2 with L k kL2 a suitable differential operator, the prototypical example being L = Dn with D the derivative operator [30, 24, 10]. We adopt in this paper an abstract formulation that encompasses the great majority of variational theories of splines that have been considered in the literature such as [9, 5] to give some notable examples. Given a 1 0 0 suitable pair of Banach (or Hilbert) spaces ( , L) whose elements are d 0 X c. X 0 functions on R , a regularization operator L: L , and a vector-valued 0 M X −−→X measurement functional ν : L R , f ν(f) = ( ν1, f ,..., νM , f ), X → 7→ d h i h i we define the generalized spline interpolant fint : R R as the solution of → 1Our notation reflects the property that the two spaces are the topological duals of two d 0 primary spaces (X , XL). X is a classical function space such as Lp(R ), while XL is the corresponding native space for L; that is, the largest Banach space such that kLfkX 0 is well-defined.

2 the variational linear inverse problem

fint = arg min Lf 0 s.t. ν(f) = (y1, . . . , yM ). (1) 0 X f∈XL k k

0 0 The simplest case occurs when there is an isomorphism between L and , X Xc. meaning that the regularization operator L has a stable inverse L−1 : 0 0 X0 −−→ . In particular, when νm = δ( xm): f f(xm) and = is a XL · − 7→ H XL reproducing-kernel Hilbert space (RKHS) such that f, g H = Lf, Lg = ∗ h i 0 h di L Lf, g for all f, g , then the solution of (1) with = L2(R ) is hexpressiblei as ∈ H X

M X fint = amh( , xm), (2) · m=1 where

∗ −1 h( , xm) = (L L) δ( xm) (3) · { · − } is the (unique) reproducing kernel of [10, 4]. The bottom line is that H M (2) is a linear model parametrized by a R . In addition, we have that 2 T M×M ∈ fint = a Ga, where G R is the positive-definite Gram matrix k kH ∈ whose entries are given by [G]m,n = h(xm, xn); this then yields the solution −1 a = G y of our initial interpolation problem with y = (y1, . . . , yM ). The theory of RHKS also works the other way around in the sense that, d instead of the operator L, one can specify a positive-definite kernel h : R d 0 × R R [1]. One then constructs the native space = by considering → XL H the closure of the vector space specified by (2) with varying M N and d ∈ xm R ; i.e., = span h( , x) x∈ d [4, 15]. ∈ H { · } R This kind of result is extendable to the scenario where L has a finite- N0 dimensional null space L = span pn n=1, in which case the solution takes the generic parametric formN { }

M N X X0 fint = amh( , xm) + bnpn · m=1 n=1

M with expansion coefficients a = (a1, . . . , aM ) R and b = (b1, . . . , bN0 ) [10, 37]. The delicate point there is to properly∈ define the corresponding d d native space since the underlying kernel h : R R R, which is still given by (3), isH no longer positive-definite [22, 28, 29].× The→ underlying native space then has the structure of a semi-RKHS [23, 2], a concept that was already present implicitly in the early works on variational splines [10, 12].

3 0 d While the aforementioned results with = L2(R ) are classical, there X 0 d has been recent interest in a variant of Problem (1) with = (R ) (the d X M space of Radon measures on R ) [34, 16, 7]. It turns out that the latter is much better suited to compressed sensing and to the resolution of inverse problems in general [18]. In particular, when L is shift-invariant, it has been shown [34, Theorem 2] that the minimum of Lf M is achieved by an k k adaptive L-spline2 of the form K N X X0 f = akρL( τk) + bnpn, (4) · − k=1 n=1 −1 where ρL = L δ , with the twist that the intrinsic parameters of the { } spline—the number K of knots and their locations τk—are adjustable with K (M N0). Remarkably, the generic form (4) of the solution is preserved ≤ − 0 M for arbitrary continuous linear measurement functionals ν : L R , far beyond the pure sampling framework of RKHS, which makesX the→ result applicable to linear inverse problems. While there have been attempts to gener- alize the -norm (or total variation) variant of the reconstruction problem (1) [16, 7,M 6], the part that has remained elusive is the specification of the corresponding native space in the multidimensional scenario (d > 1) when the null space of L is nontrivial. The difficulty there is essentially the same as the one that was encountered initially by Duchon with L = ( ∆)γ (frac- d − tional Laplacian) and = L2(R ) [12]; namely, the need to properly restrict the native space in orderX to exclude the harmonic components of the null space that grow faster than the underlying Green’s function. A systematic approach for specifying native spaces was later given by Schaback [28, 29], but it is restricted to the RKHS framework and to kernels that are conditionally positive-definite. In this paper, we develop an alternative Banach-space formulation that is applicable to a whole range of norms X 0 and operators L. In a nutshell, we are proposing a systematic approachk·k for constructing the largest Banach 0 space that ensures that Lf X 0 is well defined, subject to the constraint XL k k that the null space of L should be finite-dimensional. The main benefits of our new formulation are as follows: The extension of the notion of generalized spline via (1) through the ap- • propriate pairing of a regularization operator L and a space 0 that is X 2 d d A function f : R → R is said to be an L-spline with knots τ1,..., τK ∈ R if PK L{f} = k=1 akδ(· − τk) with a1, . . . , aK ∈ R. For instance, if d = 1 and L = D (resp. L = D2), then f is piecewise-constant (resp. piecewise-linear and continuous) with jumps (resp. breaks) at the τm.

4 the continuous dual of a primary Banach space . The two aforemen- X tioned theories with R(f) = Lf 2 (RKHS or Tikhonov regulariza- k kL2 tion) and R(f) = Lf M (generalized TV regulatization) are covered k d k d d by taking = L2(R ) and = C0(R ) the pre-dual of (R ) , but our frameworkX is considerablyX more general. M

The precise identification of the class of spline-admissible operators • L (see Definition 2). In essence, these are differential-like operators d that are injective on (R ) (Schwartz’ class of test functions) and that admit a well-definedS Green’s function, which is symbolically denoted −1 by (x, y) gL(x, y) = L δ( y) (x), where δ( y) is the Dirac 7→ { d · − } · − impulse at the location y R . ∈ The proposal of a Banach counterpart to the notion of conditional • positive definiteness from the theory of semi-RKHS: The critical hy- −1∗ pothesis here is the continuity of some pseudo-inverse operator Lφ (see Definition 4). In a companion paper, we shall demonstrate the usefulness of this criterion and how it can be readily tested in practice [14].

A systematic way of constructing the native space for (L, 0), denoted • 0 X by L, via a completion process that involves test functions and oper- atorsX rather than linear combinations of kernels. This is a significant extension of the usual approach as it applies to a much larger family of d primary spaces, including non-reflexive Banach spaces such as (R ). M 0 The guarantee of universality: The native space L specified by Theo- • rem 4 is rich enough to represent any function withX an arbitrary degree of precision3. It is also sufficiently restrictive for the null space of L to be finite-dimensional, which is non-obvious for d > 1 since the “unre- stricted” null space of partial differential operators is either trivial or infinite-dimensional [19]. The proposed approach is a convenient alternative to the more traditional use of Beppo-Levi spaces that involve composite norms with partial derivatives [3, 11, 4, 38].

The explicit characterization of L (the pre-dual of the native space • 0 X L), as given in Corollary 2. The practical signiﬁcance of this result isX that it precisely delineates the domain of validity of representer theorems for the solutions of Problem (1) or variants thereof. Indeed, a

3 0 0 d This is due to XL being dense in S (R ).

5 sufficient condition for existence is the weak* continuity of the measure- 0 M ment functional ν : R , as in [34]. This is equivalent to νm L XL → ∈ X for m = 1,...,M [25]. For instance, the requirement for the well- possedness of a regularized interpolation problem is δ( xm) L for d · − ∈ X 0 any xm R . The latter condition is automatically satisfied when L is a RKHS.∈ However, it can fail when switching to non-euclidean normsX 0 such as, for example, the critical configuration (L, ) = (D, (R)) which corresponds to the popular total-variation regularizationX M with R(f) = TV(f) = Df M [26, 8]. On the other hand, we have that k k 0 2 δ( xm) L for (L, ) = (D , (R)), which justifies the use of second-order· − ∈ X total variationX for theM regularization of deep neural networks [31].

The paper is organized as follows: In Section 2, we lay out the functional context while introducing the notion of Schwartz-Banach space, which is fundamental to our approach. In Section 3, we give the mathematical conditions for L to be spline-admissible and describe an effective way to stabilize its inverse via the use of a biorthogonal system of L (the null space of L). The final ingredient is given by some norm-compatibilityN conditions that enable the specification of the pre-Banach space L. The completion of L in the P P XL -norm yields our pre-native space L whose properties are revealed in kthe · k first part of Section 4 (Theorem 3).X This then allows us to characterize 0 the native space L (Theorem 4) and to establish its embedding properties. We also identifyX the complete family of equivalent norms, which yields a complete understanding of the underlying direct-sum topology. Finally, in Section 5, we illustrate the compatibility of our extended formulation with the specification of many classical spaces; in particular, RKHS and Lp-type Solobev spaces.

2 Preliminaries

2.1 Schwartz-Banach spaces The Banach spaces that we shall consider are embedded in Schwartz’ space of 0 d 0 d tempered distributions denoted by (R ). Formally, an element f (R ) S ∈ S is a continuous linear functional f : ϕ f, ϕ that associates a real number 7→ h i d denoted by f, ϕ to each test function ϕ (R ) (Schwartz’ space of smooth and rapidlyh decayingi functions). For instance,∈ S the Dirac impulse at location d 0 d x0 R is the generalized function δ( x0) (R ) defined as ∈ · − ∈ S ϕ δ( x0), ϕ = ϕ(x0). 7→ h · − i 6 d Similarly, any slowly increasing and locally integrable function f : R R R → specifies a distribution by way of the integral f, ϕ = d f(x)ϕ(x)dx. h i R Our construction relies on the pairing of an operator L and a Banach space 0. The latter is the dual of a primary space that is appropriately X d 0 d X sandwiched between (R ) and (R ), in accordance with Definition 1. S S Definition 1 (Schwartz-Banach space). A normed vector space ( , X ) 0 d X k·k ⊆ (R ) is said to be a Schwartz-Banach space if it can be specified as the S d completion of (R ) in the X -norm or, equivalently, if is a Banach S k · k d d. d. 0 d X space with the property that (R ) , , (R ) (continuous and dense embeddings). S −→X −→S

The reader is referred to Appendix A for the precise definition of the underlying notions of embeddings and a review of supporting mathematical results. d Prominent examples of Schwartz-Banach spaces are Lp(R ) with p d ∈ [1, ) and C0(R ) (the class of functions that are continuous, uniformly bounded∞ and decaying at infinity). By contrast, the Schwartz-Banach prop- d d erty holds neither for L∞(R ) nor for (R ) (the space of bounded Radon d M measures on R ). These two spaces, however, retain relevance for our purpose d d 0 as duals of (non-reflexive) Schwartz-Banach spaces; i.e., L∞(R ) = L1(R ) d d 0 and (R ) = C0(R ) . M Proposition 1. The dual 0 of a Schwartz-Banach space is a Banach space with the following properties:X X

d 0 d. 0 d (R ) , , (R ) •S −→X −→S 0 0 d hw,ϕi = w (R ) : supϕ∈S( d)\{0} = w X 0 < + . •X { ∈ S R kϕkX k k ∞} 0 0 d d The duality product for ( , ) is compatible with that of (R ), (R ) , • which allows us to writeX X S S

f, g X 0×X = f, g (5) h i h i for any (f, g) 0 . ∈ X × X If, in addition, is reﬂexive, then 0 is itself a Schwartz-Banach space, which then also yieldsX X

00 0 d g, ϕ = = g (R ): g X = sup h i < + . X X { ∈ S k k d ϕ X 0 ∞} ϕ∈S(R )\{0} k k

7 0 d The prototypical example that meets the last statement is = Lq(R ) = d 0 1 1 X Lp(R ) with + = 1 and 1 < p < . The property that is lost when p q ∞ X is not reflexive e.g., for (p, q) = (1, ) is the denseness of the continuous d 0 ∞ embedding (R ) , . S −→X d 0 d. 0 d Proof. The embedding (R ) , , (R ) follows from the Schwartz- Banach property, TheoremS 6−→X on dual−→S embeddings, and the reflexivity of d d d. 0 (R ). When is reflexive, we also get that (R ) , , which proves thatS 0 is a Schwartz-BanachX space. By definition,S 0 −→Xis the Banach space associatedX with the dual norm X

M w X 0 = sup w, ϕ X 0×X < + . k k kϕkX ≤1: ϕ∈X h i ∞

Now, the fundamental observation is that w, ϕ X 0×X = w, ϕ for any 0 d 0 0 d h i h i (w, ϕ) , (R ) since (R ), while the determination of the ∈ X S X ⊆ S d supremum can be restricted to ϕ (R ) in reason of the denseness of d ∈ S (R ) in . This allows us to rewrite the dual norm as S X w, ϕ w X 0 = sup w, ϕ = sup h i. (6) k k d h i d ϕ X kϕkX ≤1: ϕ∈S(R ) ϕ∈S(R )\{0} k k 0 0 d Since , (R ) and the expression on the right-hand side of (6) is well- X −→S 0 d 0 0 d defined for any w (R ), we have that = w (R ): w X 0 < ∈ S X ⊆ W { ∈ S k k . To establish the converse inclusion—and, hence, 0 = —we use (6) ∞} 0 d X W d to infer that, for any w (R ), the linear functional w : (R ) R is bounded by ∈ W ⊆ S S → w, ϕ w X 0 ϕ X |h i| ≤ k k k k d for all ϕ (R ). We then invoke Theorem 1 below with = R to deduce ∈ S Y that w has a unique continuous extension to the completed space ∈d W 0 = ( (R ), X ), which proves that . The same argument also givesX aS precisek meaning · k to the right-handW side ⊆ of X (5). Finally, in the reflexive case, we reapply the first part of Proposition 1 to 0 to obtain the announced X characterization of 00 = . X X The important point here is that the two (dual) formulas for the norms 0 d in Proposition 1 are valid for any tempered distribution w (R ). In effect, this provides us with a simple criterion for space membership∈ S (resp. 0 0 exclusion): w X 0 < w (resp. w / w X 0 = ). Likewise, k k ∞ ⇔00 ∈ X ∈ X ⇔ k k ∞ we have that g = g X < , although the equivalence holds true in the reflexive∈ case X only.X ⇔ This k k explains∞ the greater difficulty in developing

8 a general theory for non-reﬂexive native spaces4—as opposed to, say, the d traditional RKHS that go hand-in-hand with = L2(R ). A signiﬁcant advantage of working with Schwartz-BanachX spaces is the compatibility property expressed by (5). In addition, when f 0 and g are both ordinary functions, the duality product has an explicit∈ X transcription∈ X as the integral Z f, g = f(x)g(x)dx. (7) h i Rd In order to control the algebraic rate of growth/decay of such functions, we rely on weighted Banach spaces such as

d n d o L∞,α(R ) = f : R R measurable s.t. f ∞,α < + → k k ∞ with norm

M −α −α f ∞,α = (1 + ) f ∞ = ess sup (1 + x ) f(x) , k k k k · k k x∈Rd k k | |

+ where the order α R puts a cap on the rate of growth of f at infinity. ∈ d d 0 d In conformity with the relation L∞(R ) = L1(R ) , L∞,α(R ) is the con- d tinuous dual of L1,−α(R ): the Schwartz-Banach space associated with the weighted L1 norm Z M α α g 1,−α = (1 + ) g 1 = (1 + x ) g(x)dx k k k k · k k Rd k k where the switch to a negative exponent α 0 now demands that g de- − ≤ d cays at infinity; e.g., a sufficient requirement for g L1,−α(R ) is g(x) C ∈ | | ≤ (1+kxk)α+d+ with > 0. Finally, since our primary interest is with ordinary functions that are well-defined pointwise, we also consider the Banach space

d n d o Cb,α(R ) = f : R R continuous and s.t. f ∞,α < + → k k ∞

4 d iso. d iso. d For instance, the non-reflexive spaces C0(R ) ,−−→ Cb(R ) ,−−→ L∞(R ) are all equipped with the same k · k∞-norm. The condition that g is bounded is not sufficient d to ensure that g ∈ C0(R ) (the only Schwartz-Banach space in the chain); it is also required that g be continuous and decaying at infinity. The boundedness criterion for space d membership applies to L∞(R ) alone because it is the dual of the Schwartz-Banach space d L1(R ).

9 d that shares the same norm as L∞,α(R ), but whose elements (functions) are constrained to be continuous. The hierarchy between these various spaces is described by the embedding relations

d iso. d 0 d Cb,α(R ) , L∞,α(R ) , (R ) −→ −→S d d. d iso. d 0 (R ) , L1,−α(R ) , L∞,α(R ) , S −→ −→ with additional explanations given in Appendix A.

2.2 Characterization of linear operators and their adjoint A linear operator is represented by an upright capital letter such as L or G. Let and be two topological vector spaces with continuous duals X Y c. 0 and 0, respectively. Then, the notation G: indicates that X Y X −−→Y G continuously maps . The adjoint of G is the unique operator ∗ 0 c. 0 X → Y 0 ∗ 0 G : such that Gx, y Y×Y0 = x, G y X ×X 0 for any x and Y −−→X h i h i ∈ X y0 0. Moreover, if and are both Banach spaces, then the norm of the∈ operator Y is preservedX [27],Y as expressed by

M Gϕ Y ∗ G X →Y = sup k k = G Y0→X 0 . k k ϕ X k k ϕ∈X \{0} k k d d. The attractiveness of the Schwartz-Banach setting with (R ) , is the convenience of being able to specify the operator in the moreS constrained—−→X d c. 0 d but mathematically foolproof—scenario G: (R ) , (R ). The critical property, then, is the existence of a boundS of−−→Y the form−→S

d Gϕ Y C ϕ X for all ϕ (R ), (8) k k ≤ k k ∈ S which allows one to extend the domain of the operator to the complete space c. by continuity; i.e., G: with G X →Y C. This is a powerful extensionX principle that reliesX −−→Y on the bounded-linear-transformationk k ≤ (B.L.T.) theorem.

Theorem 1 ([25, Theorem I.7, p. 9]). Let G be a bounded linear transformation from a normed space ( , Z ) to a complete normed space ( , Y ). Z k · k Y k · k Then, G has a unique extension to a bounded linear transformation (with the same bound) from the completion of ( , Z ) to ( , Y ). Z k · k Y k · k The other foundational result that supports the present construction is Schwartz’ kernel theorem [17], which states that the application of the op- d c. 0 d erator G: (R ) (R ) to ϕ yields a tempered distribution G ϕ : S −−→S { } 10 d (R ) R specified by S → ψ G ϕ , ψ = g( , ), ψ ϕ , (9) 7→ h { } i h · · ⊗ i 0 d d d where g( , ) (R R ) is the kernel of the operator and ψ ϕ (R d · · ∈ S × M ⊗ ∈ S × R ) with (ψ ϕ)(x, y) = ψ(x)ϕ(y). This property is often symbolized by the formal “integral”⊗ equation Z G: ϕ g( , y)ϕ(y)dy (10) 7→ Rd · with a slight abuse of notation. Conversely, the right-hand side of (10) d c. 0 d 0 d d defines a continuous operator (R ) (R ) for any g( , ) (R R ). The association between theS kernel−−→S and the operator is unique· · ∈ S and,× hence, transferable to G: c. by continuity5. Likewise, we may describe the ∗ 0 c. X 0−−→Y0 d adjoint G : , (R ) by means of the relation Y −−→X −→S Z G∗ : ϕ g(y, )ϕ(y)dy 7→ Rd · where the flipping of the indices of the kernel is the infinite-dimensional counterpart of the transposition of a matrix. The bottom line is that the consideration of a Schwartz-Banach space allows for a concrete and unambiguous description of linear functionals and linear operators in terms of generalized functions and generalized kernels, respectively. Proposition 2 (Representation of functionals and operators). Let be a X Schwartz-Banach space. Then, any continuous linear functional g : R 0 d X → is uniquely characterized by a single element d : the restriction g S(R ) (R ) d | ∈ S of g to (R ) R. Likewise, any continuous linear operator G: S → 0 d X → Y where is a Banach subspace of (R ) is uniquely characterized by a single Y 0 d d S element g( , ) (R R ), which is the Schwartz kernel of the restriction · · d∈ S × 0 d G d : (R ) , (R ). |S(R ) S → Y −→S d d. Proof. The Schwartz-Banach property implies that (R ) , . Hence, S −→Xd both g and G are fully characterized by their restriction on (R ). As such, 0 d S g is identified as an element of (R ), while G is uniquely characterized 0S d d by its Schwartz kernel g( , ) (R R ) when seen as an operator from d 0 d · · ∈ S × (R ) to (R ). S S

5The kernel is also called the generalized impulse response of the operator. Formally, this is indicated as g(·, y) = G{δ(·−y)} with a slight abuse of notation when δ(·−y) ∈/ X .

11 The common practice is to indicate the correspondences in Proposition 2 by (7) and (10), respectively. One should keep in mind, however, that the rigorous interpretation of these relations involves the limit/extension process: Z M g, f = g(y)f(y)dy = lim g, fi i h i Rd h i Z Z M g( , y)f(y)dy = lim g( , y)fi(y)dy i Rd · Rd · d for any sequence (fi) in (R ) such that f fi X 0 as i , with the implicit understandingS that the underlyingk − “integrals”k → are the→ symbolic ∞ representations of continuous linear functionals acting on f (resp. fi).

3 Spline-admissible operators

The identification of the native space for an admissible operator L essentially boils down to the characterization of the solutions of the linear differential equation Lf = w for a suitable class of excitations w 0. This requires ∈ X that L be invertible in an appropriate sense. As a minimum, we ask that L be injective (with a well-defined inverse L−1) when we restrict its domain to d (R ). S d 0 d Definition 2 (Admissible operator). A linear operator L: (R ) (R ) S → S is called spline-admissible if there exist an order α R of algebraic growth, −1 ∈ N0 an inverse operator L , and a finite-dimensional space p = span pn n=1 such that N { }

∗ d d 1. L and L continuously map (R ) L1,−α(R ). (The continuity of S → the adjoint is required for the extended version of the operator L: d 0 d L∞,α(R ) (R ) in Condition 4 to be well deﬁned.) → S −1 −1∗ d d 2. L and L continuously map L1,−α(R ) L∞,α(R ). (The as- −1 d c. d → sumption L : L1,−α(R ) L∞,α(R ) is actually suﬃcient as it −1∗ d−−→c. d implies that L : L1,−α(R ) L∞,α(R ) by duality.) −−→ −1∗ ∗ −1 d 3. Invertibility: L L ϕ = ϕ and L Lϕ = ϕ for all ϕ (R ). ∈ S d 4. Restricted null space: p = L = p L∞,α(R ):L p = 0 . N N { ∈ { } } A preferred scenario is when L and L−1 are both linear-shift-invariant— that is, convolution operators—with respective frequency response Lˆ(ω) and

12 ˆ −1 M R 1/L(ω). Then, ϕ L ϕ = ρL ϕ = d ρL(y)ϕ( y)dy, where 7→ ∗ R · − ( ) −1 1 ρL(x) = (x), F Lˆ(ω)

0 d which is the (generalized) inverse Fourier transform of 1/Lˆ (R ) under the implicit assumption that the latter is a well-defined tempered∈ S distribution. In that case, we refer to ρL, which satisfies the formal property L ρL = δ, as the canonical Green’s function of L. { } d ∗ Example 1. The derivative operator D = dx with D = D is spline- c. − admissible with N0 = 1 and α = 0. Indeed, D: (R) (R) , L1(R), S −−→S −→ while its null space is D = span p1 Cb(R) L∞,0(R) = L∞(R) with N { } ⊂ −1 ⊆ p1(x) = 1. Its canonical inverse is given by D : ϕ ρD ϕ with ρD(x) = 1 −1 c. 7→ ∗ sign(x) and, hence, such that D : L1(R) L∞(R). 2 −−→ Let us now briefly comment on the assumptions in Definition 2. Con- ditions 1 and 2 ensures that the composition of operators in Condition 3 −1∗ ∗ d c. d c. 0 d −1 d c. (i.e., L L : (R ) L1,−α(R ) (R ) and L L: (R ) d c. S0 d −−→ −−→S S −−→ L1,−α(R ) (R )) are legitimate. Condition 1 limits the framework to operators that−−→S do not drastically affect decay. However, it does not penal- ize a loss of regularity. In particular, it is met by all constant-coefficient d (partial) differential operators, which happen to be continuous (R ) d d S → (R ) , L1,−α(R ). The condition with an adequate α is also satisfied S −→ by the fractional derivative operators Dγ or ( ∆)γ/2 (fractional Laplacian) − whose impulse responses decay like 1/ x γ+d. This decay property can be k k used to show that the fractional derivatives Dγ with γ > 0 are continuous d d (R ) L1,−(γ+d)(R ) [32], while their Green’s functions are included in S →d L∞,α(R ) with α = (γ d), which is a favourable state of affairs in the context of Condition 1. − An important observation is that the left-invertibility of L∗ in Condition −1 d −1 d 3 is equivalent to LL ϕ = ϕ for all ϕ (R ) with L ϕ L∞,α(R ) and d 0 d ∈ S ∈ L: L∞,α(R ) (R ) extended accordingly by duality. Indeed, for any d → S ϕ˜ (R ), we have that ∈ S ϕ, ϕ˜ = ϕ, L−1∗L∗ϕ˜ = LL−1ϕ, ϕ˜ , (11) h i h i h i where the factorization is legitimate, in reason of the compatible range and d domain of the underlying operators. We then invoke the denseness of (R ) 0 d S0 d in (R ) (see Proposition 6) to extend the validity of (11) for all ϕ˜ (R ), whichS yields the desired result. ∈ S

13 d While Condition 3 tells us that L is invertible on (R ), it does not guar- S d antee that the property still holds when the domain is extended to L∞,α(R ) because the corresponding null space L = p in Condition 4 may be nontrivial. We shall resolve this ambiguityN by factoringN out the null-space components. Our approach is based on the construction of a projection operator d c. ProjNp : L∞,α(R ) p, as described next. The latter is then used in Section 3.2 to specify−−→N a stable pseudo-inverse operator that is a corrected (or regularized) version of L−1.

3.1 Biorthogonal system for the null space of L The fundamental hypothesis here is that the growth-restricted null space d L = f L∞,α(R ):L p = 0 admits a basis p = (p1, . . . , pN0 ) of N { ∈ { } } ﬁnite dimension N0. The idea then is to select a set of analysis functionals 0 d φ1, . . . , φN0 (R ) that satisfy the biorthogonality relation ∈ Y ⊆ S φm, pn = δm,n, m, n 1,...,N0 . (12) h i ∈ { } To give a concrete meaning to the above duality product, we are assum- 6 0 0 d ing the existence of a dual pair ( , ) of subspaces of (R ) such that Y0 Y S φ1, . . . , φN and p1, . . . , pN . Such a construction is always feasible 0 ∈ Y 0 ∈ Y (see Proposition 3) with the choice of φ being free. We also understand that there is a whole equivalence class of representations of L = p = p˜ with N N×N N N p˜ = Bp, under the constraint that the matrix B R 0 0 be invertible. ∈

Deﬁnition 3 (Biorthogonal system). A pair (φ, p) with φ = (φ1, . . . , φN0 ) N0 0 N0 ∈ and p = (p1, . . . , pN0 ) ( ) is called a biorthogonal system for the Y ∈ Y N0 0 0 d ﬁnite-dimensional subspace p = span pn n=1 (R ) if any p p admits a unique expansion ofN the form{ } ⊂ Y ⊆ S ∈ N

N X0 p = φn, p pn. (13) h i n=1

The natural norm induced on p by such a system is N 1 N0 ! 2 X 2 p N = φ(p) 2 = φn, p . k k p k k |h i| n=1

d The system is said to be universal if φn (R ) for n = 1,...,N0. ∈ S 6 d 0 In view of the inclusion NL ⊂ L∞,α(R ), the natural choice is to take (Y, Y ) = d d 0 0 L1,−α(R ),L∞,α(R ) . We shall see that this is extendable to (Y, Y ) = (XL, XL).

14 Description Operator Kernel

N0 0 X Riesz map φ p = Rp pn(x)pn(y) N → N Nφ n=1 N0 0 X Riesz map p φ = Rφ φn(x)φn(y) N → N Np n=1 N0 0 d X Projector L∞,α(R ) p ProjN = RpRφ pn(x)φn(y) XL ⊆ → N p n=1 N0 d X Projector L L1,−α(R ) φ ProjN = RφRp φn(x)pn(y) X ⊇ → N φ n=1

Table 1: Complete set of operators associated with the biorthogonal system N0 N0 (φ, p) with p = span pn and φ = span φn . N { }n=1 N { }n=1

The unicity of the representation in Deﬁnition 3 implies that p be a basis of p, while the validity of (13) for p = pn implies that the underlying functionsN be biorthogonal, as expressed by (12). Our next proposition ensures the existence of such systems for any given d basis p under our working hypothesis (R ). Y ⊇ S N0 Proposition 3 (Existence of biorthogonal systems). Let p = span pn n=1 0 d N { } be a N0-dimensional subspace of (R ). Then, there always exists some S d (universal) biorthogonal set of functionals φ1, . . . , φN0 (R ). ∈ S Proof. This result is deduced from the following variant of the Hahn-Banach 0 d theorem with = (R ). V S Theorem 2 ([27, Theorem 3.5]). Let be a linear subspace of a topological Z vector space , and v0 be an element of . If v0 is not in the closure of , V V Z then there exists a continuous linear functional φ on such that φ, v0 = 1 V h i but φ, z = 0 for every z . h i ∈ Z We then proceed by successive exclusion of v0 = pn with φ = φn and = span pm , while the ﬁnite dimensionality of and the linear inde- Z m6=n{ } Z pendence of the pm ensures that pn / = . ∈ Z Z

d Example 2. We recall that the derivative operator D = dx from Example 1 is spline-admissible with α = 0, N0 = 1, and D = span p1 L∞(R) N { } ⊂ where p1(x) = 1. As complementary analysis functional, we may choose any

15 function φ1 L1(R) (or φ1 (R) if we are aiming at a universal solution) R ∈ ∈ S such that φ1(x)dx = φ1, 1 = 1. R h i N c. Example 3. The N0th-order derivative operator L = D 0 : (R) (R) −1 S1 −−→SxN0−1 is spline-admissible with L : ϕ ρ N ϕ where ρ N (x) = sign(x) , 7→ D 0 ∗ D 0 2 (N0−1)! N0 n−1 α = (N0 1), and DN0 = span pn L∞,N0−1(R) with pn(x) = x . − N { }n=1 ⊂ A possible choice of universal biorthogonal system is (φ˜, p˜) with φñ(x) = −x2/2 pñ(x)e (R) and p˜1,..., pÑ0 the normalized Hermite polynomials of ∈ S degree 0 to (N0 1). By definition, the latter form a basis of N —the − ND 0 space of polynomials of degree (N0 1)—that fullfils the (bi)-orthogonality −x2/2 − relation e pñ, p˜m = δ[m n]. h i − N0 Given such a system, we then specify p and φ = span φn as a N N { }n=1 dual pair of N0-dimensional Hilbert spaces equipped with the inner products p, q N = Rφ p , q and φ, ϕ N = Rp φ , ϕ , respectively, where h i p h { } i h i φ h { } i N X0 Rp φ = pn, φ pn { } h i n=1 N X0 Rφ p = φn, p φn. { } h i n=1

Indeed, φn = Rφ pn and pn = Rp φn , which allows us to identify Rφ { } { } 0 0 (resp. Rp) as the Riesz map p φ = p (resp. Rp : φ p = φ). For the proper interpretationN of this→ N (Riesz)N pairing, we recallN → that N theN dual space 0 is an abstract entity that is formed by the linear functionals that Np are continuous on p; strictly speaking, each functional is an equivalence N 0 0 class of tempered distributions. The notation = φ indicates that Np N Np is isometrically isomorphic to φ, meaning that each (abstract) member of 0 N has a unique representative in φ, which then serves as our concrete Np N descriptor. The generic elements of these spaces are denoted by p p with N0 ∈ N p Np = φ(p) 2, where φ(p) = ( φ1, p ,..., φN0 , p ) R and φ φ k k k k h i h i ∈ ∈ N with φ Nφ = p(φ) 2. k k k k 0 Under the working assumptions that φ and p , where is a suitable Schwartz-Banach space, the domainN ⊆ of Y continuityN ⊆ of Y these operatorsY c. 0 c. is extendable to Rp : p and Rφ : φ. The corresponding Y −−→N Y −−→N

16 projection operators are

N X0 Proj g = pn, g φn = RφRp g , (14) Nφ { } h i { } n=1 N X0 Proj f = φn, f pn = RpRφ f (15) Np { } h i { } n=1 for any and 0 with the property that for all g f ProjNφ φ = φ φ and ∈ Y ∈ Yfor all . We also note that{ } ∗ ∈, φ ProjNp p = p p p ProjNφ = ProjNp whichN emphasizes{ } the symmetry∈ of N the construction. We can also rely on the generic duality bound pn, g pn Y0 g Y to get a handle on the continuity properties of these|h operators.i| ≤ k Speciﬁcally,k k k based on (14), we ﬁnd that

1/2 N0 ! X 2 p ProjN g = pn, g φ { } 2 |h i| n=1 1/2 N0 ! X 2 pn 0 g Y (16) ≤ k kY k k n=1 where the leading constant on the right-hand side provides an upper bound c. on the norm of the operator Proj : φ . Likewise, we have Nφ Y −−→N ⊂ Y 1/2 0 c. 0 PN0 2 that Proj : p with Proj Y0→N φn . Np Y −−→N ⊆ Y k Np k p ≤ n=1 k kY The whole setup is summarized in Table 1 for the choice of spaces = d 0 d Y L1,−α(R ) and = L∞,α(R ), which is adopted for the sequel. Y 3.2 Speciﬁcation of a suitable pseudo-inverse operator

Under our hypotheses, the projection operator ProjNp deﬁned by (14) is con- 0 d c. d tinuous = L∞,α(R ) L∞,α(R ). This, in turn, ensures the continuity of Y −−→

−1 M −1 d c. d c. d L = (Id ProjN ) L : L1,−α(R ) L∞,α(R ) L∞,α(R ). φ − p ◦ −−→ −−→ By observing that ∗ , we then readily deduce (Id ProjNp ) = (Id ProjNφ ) −1− − that the adjoint of Lφ is such that

−1∗ −1∗ d c. d L = L (Id ProjN ): L1,−α(R ) L∞,α(R ), (17) φ − φ −−→

17 d iso. d 00 d 0 owing to the property that L1,−α(R ) , L1,−α(R ) = L∞,α(R ) . −1∗ −→ d While (17) ensures that Lφ is well-deﬁned on L1,−α(R ), we also want to make sure that the range of this operator can be restricted to the primary space (the pre-dual of 0 mentioned in the introduction), which calls for some additionalX compatibilityX hypotheses.

Definition 4 (Compatibility of (L, )). Let L be a linear operator and a 0 d Y Y Banach subspace of (R ). We say that the pair (L, ) is compatible if S Y 1. There exists a Schwartz-Banach space (see Definition 1) such that X = 0; Y X 2. the operator L is spline-admissible with order α (see Definition 2);

∗ c. 0 d 3. the adjoint operator L is continuous (R ) and injective; X −−→S 4. there exists a universal biorthogonal system (φ, p) of the null space of −1∗ d c. −1∗ L such that Lφ : L1,−α(R ) where Lφ is specified by (17). This condition is called -stability.−−→X X Conditions 1 and 2 are explicit and hardly constraining. The injectivity of L∗ is equivalent to the intersection between the extended null space of L∗ and being trivial. The most constraining requirement is Condition 4, which needsX to be checked on a case-by-case basis. Interestingly, we shall see that, if the condition holds for one particular biorthogonal system (φ, p) d with φ (R ) (universality property), then it also holds for any other N ⊂ S admissible biorthogonal system (φ˜, p˜) as long as ˜ L, where the pre- Nφ ⊂ X dual space L is characterized in Theorem 3. X −1∗ The operator Lφ in (17) is fundamental to our construction. The key will be to extend its domain to make it surjective over . To that end, we now introduce a suitable pre-Banach space that will thenX be completed to yield our pre-dual space L. X Definition 5 (Pre-Banach space for (L, 0)). Under the compatibity hy- X potheses of Definition 4, we specify the pre-Banach space for (L, 0) as the vector space X

∗ d d L = L ϕ + φ : ϕ (R ), φ φ L1,−α(R ). (18) P { ∈ S ∈ N } ⊆ Proposition 4. Under the compatibity hypotheses of Deﬁnition 4, for any −1∗ d g L, there is a unique pair (ϕ = L g, φ = ProjN g ) (R ) φ ∈ P φ φ { } ∈ S × N such that g = L∗ϕ + φ, where the two underlying operators are deﬁned by (17) and (14), respectively. In particular, this implies the following:

18 −1∗ 1. Null-space property: L φ = 0 for all φ φ L. φ ∈ N ⊂ P ∗ −1∗ ∗ −1∗ ∗ d 2. Left invertibility of L : L L ϕ = L L ϕ = ϕ for all ϕ (R ). φ ∈ S ∗ −1∗ 3. Pseudo-right-invertibility: L L g = (Id Proj ) g for all g L. φ − Nφ { } ∈ P ∗ d 4. L has a direct sum decomposition as L = L (R ) φ. P P S ⊕ N 5. L, equipped with the composite norm P M −1∗ g X = max( L g X , p(Proj g ) 2), (19) k k L k φ k k Nφ { } k | {z } | {z } kϕkX kp(φ)k2

d is a normed subspace of L1,−α(R ). −1∗ −1∗ 6. L is bounded L with L g X g X for all g L. φ P → X k φ k ≤ k k L ∈ P 7. Proj is bounded L φ with Proj g X g X for all g Nφ P → N k Nφ k L ≤ k k L ∈ L. P Before moving to the proof, we observe that every one of the operators d c. , −1∗ d c. , and ∗ −1∗ ProjNφ : L1,−α(R ) φ Lφ : L1,−α(R ) L Lφ : d c. c.−−→N0 d −−→X ◦ L1,−α(R ) (R ) is well-deﬁned on g L because L d −−→ X −−→ S ∈ P P ⊆ L1,−α(R ) by construction. ∗ d Proof. From (18), L = L (R ) + φ. Thanks to the hypotheses pn d P Sd 0 N ∗ d ∈ L L∞,α(R ) = L1,−α(R ) and ψ = L ϕ L1,−α(R ), we have that N ⊆ ∗ ∈ pn, ψ = pn, L ϕ = Lpn, ϕ = 0 or, equivalently, Proj ψ = 0 for all h i h i h i Nφ { } ∗ d . Likewise, due to the biorthogonality of , ψ L (R ) (φ, p) ProjNφ φ = for∈ anyS . In other words, for , the condition { } φ φ φ g L ProjNφ g = g ∈ N ∗ ∈d P { } is equivalent to g φ; that is, L (R ) φ = 0 , which establishes the direct-sum property∈ N (Item 4). S ∩ N { } −1∗ By applying the deﬁnition of Lφ in (17) and by invoking the spline- admissibility of L, we then get that

L−1∗ L∗ϕ + φ = L−1∗(Id Proj ) ψ + φ = L−1∗ψ = L−1∗L∗ϕ = ϕ. φ { } − Nφ { } d for all ϕ (R ) and φ φ. In doing so, we have actually shown that the ∈d S ∈ N ∗ map (R ) φ L :(ϕ, φ) g = L ϕ + φ is invertible and that S × N → P 7→ −1∗ −1∗ ∗ ∗ d L ψ = L ψ for all ψ = L ϕ L (R ) . (20) φ ∈ S

19 The properties in Items 1-3 then simply follow from the observation that for any which, in light of the previous identities, (Id ProjNφ ) φ = 0 φ φ − −{1∗} ∗ −1∗ ∈∗ N d also yields L (L ϕ) = L (L ϕ) = ϕ for all ϕ (R ). φ ∈ S d d. The property that (R ) , ensures that X is a bona ﬁde norm over d S −→X d k·k (R ). This allows us to equip (R ) φ with the composite norm (ϕ, φ) S S ×N d 7→ max( ϕ X , p(φ) 2). We then exploit the bijection between (R ) φ and ∗ k dk k k S ×N L (R ) φ to get the expression of the norm given in (19). In eﬀect, this S ⊕N d shows that the normed space ( (R ), X ) φ is isometrically isomorphic ∗ d S k·k ×N to L = L (R ) L equipped with the XL -norm, which is the desired result.P S ⊕N k·k To determine the corresponding norm inequalities (Items 5 and 6), we apply the direct-sum property to rewrite the norm of g L as ∈ P

g X = max( (Id Proj ) g X , Proj g X ), k k L k − Nφ { }k L k Nφ { }k L | −{z1∗ } kLφ gkX from which we deduce that −1∗ and Lφ g X g XL ProjNφ g XL g XL for any . These twok inequalitiesk ≤ k arek sharpk because{ }k ≤ k(resp.k g L ProjNφ −1∗ ∈ P ∗ d L ) is an isometry over φ (resp. an isometry over L (R ) ). φ N S We note that the projection property in Item 3 is equivalent to

∗ −1∗ ∗ ∗ d L L L ϕ = L ϕ for all ϕ (R ), φ ∈ S −1∗ ∗ which indicates that Lφ is a generalized inverse of L .

4 Direct-sum topology of the native space

0 In this section, we reveal the Banach structure of the native space L and 0 X of its pre-dual L. We like to think of as the largest set of functions X XL f such that Lf X 0 < under the constraint of a ﬁnite-dimensional null k k 0 ∞ space L = f L :L f = 0 = p. We also assume that is a Schwartz-BanachN { ∈ space X with{ the} cases} ofN interest being X

d L2(R ) equipped with the L2 norm and, more generally, • k · k d Lp(R ) equipped with the Lp norm for p [1, ); • k · k ∈ ∞ d d C0(R ), which can be speciﬁed as the closure of (R ) in the ∞- • norm. S k · k

20 4.1 Pre-dual of the native space

Definition 6. The pre-native space L is the completion of the pre-Banach X space L of Definition 5 for the X -norm defined by (19). P k · k L We now identify this space and prove that it is isometrically isomorphic to φ, which requires the use of Cauchy sequences to extend the properties X ×N of L in Proposition 4. We recall that the two normed spaces that underly the P ∗ d specification of L = φ in Proposition 4 are = (L (R ) , W ) with P W ⊕ N W S k · k −1∗ ψ W = L ψ X , k k k φ k −1∗ where L : L = ( φ) and ( φ, N ) with φ N = p(φ) 2. φ P W ⊕ N → X N k · k φ k k φ k k Theorem 3 (Pre-dual of the native space). Under the admissibility and compatibility hypotheses of Definition 4, the completion of ( L, XL ) is the Banach space P k · k

∗ L = L ( ) φ, X X ⊕ N which is itself isometrically isomorphic to φ. Correspondingly, the X−1 ×∗ N bounded operators Proj : L φ and L : L of Proposition 4 Nφ P → N φ P → X have unique continuous extensions c. and −1∗ c. ProjNφ : L φ Lφ : L with the following properties: X −−→N X −−→ X M ∗ 1. Null space of Proj : = L ( ) = g L : Proj g = 0 . Nφ U X { ∈ X Nφ { } }

−1∗ M N0 −1∗ 2. Null space of L : φ = span φn = g L :L g = 0 . φ N { }n=1 { ∈ X φ } 3. Left inverse of L∗: L−1∗L∗v = v for any v . φ ∈ X ∗ −1∗ 4. Pseudo-right-inverse: L L g = (Id Proj ) g for any g L, φ − Nφ { } ∈ X PN0 where Proj : g pn, g φn with the property that pn, u = 0 for Nφ 7→ n=1h i h i all u and pm, φn = δ[m n]. Moreover, we have the hierarchy of continuous∈ U andh dense embeddingsi −

d d. d d. d. 0 d (R ) , L1,−α(R ) , L , (R ). S −→ −→X −→S

Proof. By deﬁnition, we have that L = L which, in view of Theorem 7 in X P Appendix B, is itself decomposable as L = φ = φ (because P W ⊕ N W ⊕ N φ is ﬁnite-dimensional). N

21 (i) = = L∗( ) = u = L∗v : v equipped with the topology inher- itedW fromU . X { ∈ X } X ∗ c. 0 d Since the map L : (R ) is injective, we have an isometric isomor- X −−→S phism between the Banach space ( , X ) and , which is itself a Banach X k · k U space equipped with the norm u U = v X where v is the unique element k k k k in such that u = L∗v. In particular, for any ψ , we can use PropertyX 2 of Proposition 4 (invertibility) to show that∈ W ⊆ U

−1∗ ψ U = L ψ X = ψ W . k k k φ k k k d Moreover, because the spaces ( (R ), X ) and ( , U ) are isometric, S∗ k · k W k · k is dense in : for any u = L v and > 0, there exists some ψ W U ∈ U d ∈ W such that u ψ U . Indeed, the denseness of (R ) in implies the k − k ≤d ∗S ∗ X existence of ϕ (R ) such that v ϕ X = L v L ϕ U so that ∈ S ∗ k − k k − k ≤ it suffices to take ψ = L ϕ . Since is complete and admits as a dense subset, it can be identified∈ W as the completionU of equipped withW the W W = U -norm. k · k k · k (ii) Extension of operators and functionals For clarity, we mark the extended operators mentioned in the theorem with a tilde. Specifically, the application of Theorem 1 with = L and = Z P Y , φ, and R allows us to specify the unique extensions X N −1∗ c. −1∗ L] : L with L] = 1 • φ P −−→X k φ k c. Projg : L φ with Projg = 1 • Nφ P −−→N k Nφ k

pn : L R with pn 1. • e P → ke k ≤ The relevant bounds for the two ﬁrst instances are directly deducible from Properties 6 and 7 in Proposition 4, while the explicit deﬁnitions of these extensions are given in (21) and (23). As for the functionals pn : L R P → for n = 1,...,N0, we observe that

1 N0 ! 2 ∗ X 2 pn, L ϕ + φ = pn, φ max( ϕ X , pm, φ ), |h i| |h i| ≤ k k |h i| m=1

d for any ϕ (R ) and φ φ, which yields the supporting bound ∈ S ∈ N

pn, g g X for all g L. |h i| ≤ k k L ∈ P

22 (iii) Derivation of Properties 1-4 by continuity In line with the argumentation in Item (i), for any u = , we have that ∈ U W −1∗ −1∗ M −1∗ u U = L]u X with L]u = lim L ψi, (21) k k k φ k φ i→∞ φ where (ψi) is any Cauchy sequence in such that u = limi→∞ ψi = . In fact, the underlying isometric isomorphismW ensures that a Cauchy∈ W U −1∗ sequence (ψi) in maps to a corresponding sequence (ϕi = Lφ ψi) that Wd ∗ is Cauchy in ( (R ), X ), and vice versa by taking ψi = L ϕi. In the S k · k −1∗ −1∗ limit, we have that v = limi→∞ ϕi = Lφ limi→∞ ψi = Lφ u and ∗ ∗ { } ∈ X u = limi→∞ ψi = L limi→∞ ϕi = L v . The last characterization also yields that { } ∈ U

∗ pn, u = lim pn, ψi = lim L pn, ϕi = 0 (22) he i i→∞h i i→∞h i for all u , which is consistent with the property that Projg u = 0. The ∈ U Nφ conclusion is that pe(u) = 0 for all u so that the extended projector c. ∈ U Projg : L φ retains the same functional form as before as Nφ X −−→N N X0 Projg g = pn, g φn. (23) Nφ { } he i n=1 Due to the isometric isomorphism between and , it is then also accept- U X able to decompose the extended pseudo-inverse as L]−1∗ = L∗−1(Id Projg ), φ − Nφ where L∗−1 denotes the formal inverse of L∗ from . By plugging in the relevant Cauchy sequences and by invoking (21)U and → (22),X it is then possible to seamlessly transfer Properties 1-3 of Proposition 4 to the completed counterpart of these spaces, which yields Items 1-4. (iv) Embeddings d We simplify the notation by setting = L1,−α(R ) and consider the decom- YM position = p⊥ φ, where p⊥ = ψ : p(ψ) = 0 . To prove that Y Y ⊕ N Y { ∈− Y1∗ c. } p⊥ , we ﬁrst invoke the continuity of Lφ : , which ensures Y ⊆ U −1∗ Y −−→X that v = L ψ for all ψ ⊥ . We then use Property 4 (or the φ ∈ X ∈ Yp injectivity of ∗) to get that ∗ ∗ −1∗ (because L L v = L Lφ ψ = ψ ProjNφ ψ = 0 ∗ { } for all ψ p⊥ ), which shows that ψ L ( ) = . This, together with the ∈ Y c. ∈ X U continuity of L∗ : , ensures the continuity of the inclusion/identity ∗ −1X∗ −−→U c. c. map I = L L : ⊥ , which is equivalent to ⊥ , . ◦ φ Yp −−→X −−→U Yp −→U

23 Consequently, we have that = ( p⊥ φ) , ( φ) = L. More- Y d Y ⊕ N −→ U ⊕ N X over, since L = L1,−α(R ) (see Item 5 in Proposition 4) and L is P ⊆ Y P a dense subspace of L = L by construction, we readily deduce that the X P d d. embedding , L is dense. Likewise, since (R ) , , we get that d d. Y −→X S −→Y∗ c. 0 d (R ) , L by transitivity. Finally, the continuity of L : (R ) S −→X ∗ 0 d X 0 −−→Sd implies that = L ( ) , (R ) which, together with φ , (R ), yields U0 d X −→S N −→S that L , (R ). Here too, the embedding is dense due to the property X −→Sd d. 0 d that (R ) , (R ) (see Propositions 6 and 7 in Appendix A). S −→S (v) Identification of p = p , Proj = Proj and L]−1∗ = L−1∗ en n g Nφ Nφ φ φ So far, we have distinguished the extended operators and functionals from d the original ones whose initial domain was restricted to L1,−α(R ). We now invoke the Schwartz-Banach property of both L (see Item (iv) and d X Definition 1) and L1,−α(R ) to argue that there is a common underlying characterization (see Proposition 2) that is applicable to both instances. Consequently, it is acceptable to write that pñ : g pn, g , which then gives a concrete and rigorous interpretation of the extended7→ h functionalsi and, by the same token, the extended projector (23). Likewise, from now on, −1∗ we denote both the original and extended pseudo-inverse operators by Lφ , under the understanding that their underlying Schwartz kernel is the same.

−1∗ An important observation is that the -stability hypothesis (i.e., Lφ : d c. X L1,−α(R ) ) is only required for the proof of the embeddings (last statement of−−→X the theorem). This is a fundamental point as it ensures that L is a Schwartz-Banach space, while it also yields a concrete interpretation Xof the underlying operators. Last but not least, it guarantees that the actual 0 0 d native space L is a proper Banach subspace of (R ) (by Proposition 1). Implicit inX the statement of Theorem 3 (andS explicit in the proof) are the following fundamental properties of : the primary part of L “perpen- U X dicular” to φ. N Corollary 1. The space = u = L∗v : v in Theorem 3 has the following properties: U { ∈ X } 1. For any u , L−1∗u = L−1∗u and L∗L−1∗u = L∗L−1∗u = u. ∈ U φ ∈ X φ −1∗ 2. is a Banach space equipped with the norm u U = L u X . U k k k φ k 3. = L∗( ) is isometrically isomorphic to : For any u (resp. U X X −1∗ ∈ U for any v ), there exists a unique element v = Lφ u (resp. ∗ ∈ X ∈ X u = L v ) such that u U = v X . ∈ U k k k k 24 4. L∗ : iso. (isometry). X −−−→U 5. L−1∗ = L−1∗ : iso. (isometry). φ U −−−→X 6. For any (p, u) ( p ), p, u = 0. ∈ N × U h i d d 7. is the completion of p⊥ (R ) = ψ (R ): p(ψ) = 0 in the U S { ∈ S } U -norm. k · k Proof. Items 1-5 are re-statements/re-interpretations of the invertibility Prop- erties 3 and 4 in Theorem 3. The key is that (Id Proj ) u = u for all − Nφ { } u , which then makes the presence of this operator redundant. Item 6 results∈ U from the simple manipulation

∗ −1∗ −1∗ p, u = p, L Lφ u = Lp , Lφ u = 0, h i h i h|{z} i =0

0 which is legitimate because u L = φ and p p = φ 0 0 ∈ X U ⊕ N ∈ N N ⊂ L = ( φ) . As for Item 7, we consider the direct-sum decomposition X d U ⊕ N d d (R ) = p⊥ (R ) φ, which is valid whenever φ (R ) (universality assumption).S S We then⊕ N observe that N ⊂ S

d d ( (R ), XL ) = ( p⊥ (R ), U ) ( φ, Nφ ). S k · k S k · k ⊕ N k · k

This equality holds because ϕ XL = ψ U + p(φ) 2 for any ϕ = ψ + φ d k k d k k k k ∈ (R ) L with (ψ, φ) p⊥ (R ) φ ( φ). The denseness of S ⊆ X ∈ S × N ⊆ U × N d d the embedding (R ) , L from Theorem 3 implies that ( (R ), XL ) = S −→X S k · k L = φ. Finally, by invoking Theorem 7 and the property that X U ⊕ N ( φ, N ) = φ (because φ is finite-dimensional), we deduce that N k · k φ N N d d d. ( p⊥ (R ), XL ) = , which is equivalent to p⊥ (R ) , . S k · k U S −→U −1∗ Item 1 in Corollary 1 indicates that the pseudo-inverse Lφ and the canonical adjoint inverse L−1∗ are undistinguishable on . This implies that the topology of does not depend on the choice of biorthogonalU system U (φ, p). By contrast, the effect of the two inverse operators is very different −1∗ on φ: for any φ φ, Lφ φ = 0 by design (Property 1), while q = −1N∗ ∈d N { } L φ L∞,α(R ) is nonzero and, in general, not even included in { } ∈ X unless φ = 0. We end this section by listing the properties of L that we believe to be the most relevant to practice. They are directly deducibleX from Theorem 3, too.

25 1 L

0 0 (u0)=0 X L U

1 0 =L ( 0) XL X Np Rp N Np 0 L R { } p 1 =L⇤( ) L ⇤ U X 1 0 =L ( 0) p(u)=0 U X U L X L = p = span pn ⇤ N N { } 0 = = span NL N { n}

=L⇤( ) XL X N

Figure 1: Schematic representation of the operators and Banach spaces that 0 appear in the deﬁnition of (the native space for L) and its pre-dual L. XL X

Corollary 2. Under the admissibility and compatibility hypotheses of Deﬁ- nition 4, the pre-dual space L has the following properties: X d L is the completion of (R ) in the XL -norm, which is speciﬁed •X −1∗ S k · k as ϕ X = max( L ϕ X , p(ϕ) 2). k k L k φ k k k 0 d Let f (R ). Then, f L if and only if there exists (v, φ) • ∈ S ∈∗ X ∈ ( φ) such that f = L v + φ. Moreover, the decomposition is uniqueX × N with −1∗ and . v = Lφ f φ = ProjNφ f

L is a Schwartz-Banach space. •X 4.2 Native space 0 As indicated by the notation, the native space L is the continuous dual of ∗ X L = φ, where = L ( ). Accordingly, there is a direct correspon- X U ⊕ N U X 0 dence between the properties of L and those of the predual space L in Theorem 3. The whole functionalX picture is summarized in Figure 1. X

Theorem 4 (Native Banach Space). Under the admissibility and compatibility hypotheses of Deﬁnition 4, the continuous dual of L in Theorem 3 is X

26 the native Banach space

0 0 −1 0 = p = L ( ) p XL U ⊕ N φ X ⊕ N −1 0 = L w + p : w , p p , (24) { φ ∈ X ∈ N } 0 which is isometrically isomorphic to p equipped with the composite X × N 0 norm w X 0 + φ(p) 2. In other words, for any f L, there is a unique pair k k 0 k k PN0 ∈ X w = Lf and p = Proj f = φn, f pn p, with the ﬁnite- ∈ X Np { } n=1h i ∈ N N0 dimensional space p = span pn n=1 being the null space of the operator 0 c. 0 N { } −1 0 c. 0 L: L . This is consistent with the operator Lφ : L, which X −−→X −1∗ X −−→X is the adjoint of Lφ in Theorem 3, having the following properties:

1. Eﬀective range: 0 = L−1( 0) =M s = L−1w : w 0 . U φ X { φ ∈ X } −1 0 2. Annihilator: φ = g L : L w, g = 0 for all w . N { ∈ X h φ i ∈ X } 3. Right inverse of L: LL−1w = w for any w 0. φ ∈ X 4. Left pseudo-inverse: L−1Lf = (Id Proj ) f for any f 0 . φ − Np { } ∈ XL Moreover, we have the hierarchy of continuous (and sometime dense) embeddings described by

d 0 d d. 0 d (R ) , L , L∞,α(R ) , (R ). S −→X −→ −→S 0 Finally, if is reﬂexive, then L is reﬂexive as well and we have the dense X d d. 0 X embedding (R ) , . S −→XL Proof. The listed properties are the dual transpositions of the ones in The- 0 0 orem 3. The key is φ = p (see explanation in Section 3.1) and L = 0 0 0N 0 N X ( φ) = = p equipped with the dual composite norm U ⊕ N U ⊕ Nφ U ⊕ N

0 0 f X = ( ProjU f U , ProjN f Np ) 1 k k L k k k k p k k = Proj 0 f U 0 + φ(f) 2. k U k k k For the details, the reader is referred to Appendix B on direct sums and Proposition 8 with (p, q) = ( , 1). The other fundamental ingredient is ∞ iso. iso. the continuity of the adjoint operators L: 0 0, L−1 : 0 0, U −−−→X φ X −−−→U −1 0 c. 0 ∗ 0 c. iso. 0 L : , Proj = Proj : p , , which follows φ X −−→XL Nφ Np XL −−→N −→ XL ∗ iso. −1∗ iso. −1∗ c. from the continuity of L : , L : , L : L , X −−−→U φ U −−−→X φ X −−→X 27 iso. c. in Theorem 3 and Corollary 1. The adjoint ProjNφ : L φ , L relation X −−→N −→∗ is X due to the special form of the underlying kernel ProjNp = ProjNφ (see Table 1). 0 −1 0 (i) Identification of = Lφ ( ) with s U 0 = Ls X 0 Since the mapping betweenU X0 and 0kisk isometrick k and bijective, we have 0 −1 0 X U that = Lφ ( ), with the two spaces being isometrically isomorphic. By recallingU theX definition of the dual norm and invoking the isomorphism between and = L∗( ) with v u = L∗v, we then get that X U X 7→ ∗ s, u U 0×U s, L v U 0×U s U 0 = sup h i = sup h i k k u U v X u∈U\{0} k k v∈X \{0} k k Ls, v X 0×X = sup h i = Ls X 0 . v X k k v∈X \{0} k k (ii) Derivation of Properties 2-4 by duality The underlying principle is that the weak topology (resp. the weak∗ topology) separates the points in (resp. the dual of) a locally convex vector space [27]. Specifically, let ( 0, ) be any dual pair of Banach spaces. Then, for X X 0 any g1, g2 and f1, f2 , ∈ X ∈ X 0 g1 = g2 f, g1 X 0×X = f, g2 X 0×X for all f , ⇔ h i h i ∈ X f1 = f2 f1, g X 0×X = f2, g X 0×X for all g . ⇔ h i h i ∈ X Consequently, the null-space Property 2 in Theorem 3 is equivalent to

−1∗ −1 0 g φ 0 = w, L g X 0×X = L w, g X 0 ×X , w , ∈ N ⇔ h φ i h φ i L L ∀ ∈ X which is the desired result. The same principle applies for the other properties. (iii) Embeddings The application of Theorem 6 to the series of continuous and dense embeddings in Theorem 3 yields

d 0 d 0 d 0 d (R ) , L , L1,−α(R ) = L∞,α(R ) , (R ). S −→X −→ −→S 0 d. 0 d d d. 0 d The denseness of the embedding L , (R ) follows from (R ) , (R ) and Proposition 7. X −→S S −→S

28 We note that Property 2 has the other equivalent formulation

0 = g 0 : φ(g) = 0 = g 0 : Proj g = 0 , U { ∈ XL } { ∈ XL Np { } } which is consistent with the direct-sum property. Hence the combination of Properties 2-4 implies a perfect isometry between 0 and 0 with w = 0 −1 0 U X Ls , s = L w , and s U 0 = Ls X 0 . The consideration of ∈ X φ ∈ U k k k k the direct-sum decomposition f = s + p, where p = Proj f p and Np { } ∈ N s = (f p) = L−1w 0, allows us to identify the norm of 0 as − φ ∈ U XL

f X 0 = s U 0 + φ(p) 2 = Ls X 0 + φ(p) 2 k k L k k k k k k k k = Lf X 0 + φ(f) 2, (25) k k k k where we have made use of the property that L s + p = Ls and φ(s) = 0 0 { } for all (s, p) ( p). This is the basis for a restatement of the primary properties of∈ 0Uin× Theorem N 4 in a form more suitable for practitioners. XL Corollary 3. Under the admissibility and compatibility hypotheses of Deﬁ- 0 0 nition 4, the native space of (L, ), denoted by L, has the following properties: X X 0 is a Banach space that admits the explicit deﬁnition •X L 0 0 d = f ( ): f 0 = sup f, ϕ < (26) L R XL X { ∈ S k k d h i ∞} kϕkXL ≤1: ϕ∈S(R ) or, equivalently,

0 d L = f L∞,α(R ): f X 0 < , X { ∈ k k L ∞} 0 where α is the growth order associated with L. Moreover, f L 0 ∈ X ⇔ f X 0 = Lf X 0 + φ(f) 2. In particular, f Lf X 0 < + . k k L k k k k ∈ XL ⇒ k k ∞ 0 d 0 Let f (R ). Then, f L if and only if there exists (w, p) • 0 ∈ S ∈−1 X ∈ ( p) such that f = L w + p. Moreover, the decomposition is X × N φ unique, with w = Lf and p = Proj f . Np { } 0 Equation (26) provides a rigorous, self-contained deﬁnition of L, but it has the disadvantage of being a bit convoluted. We like to view thisX equation as the justiﬁcation for the two alternative forms

0 n d o L = f L∞,α(R ): f X 0 = Lf X 0 + φ(f) 2 < (27) X ∈ k k L k k k k ∞ 0 n d o L = f L∞,α(R ): Lf X 0 < , (28) X ∈ k k ∞

29 d which are adequate if one implicitly assumes that φ L1,−α(R ) and 0 N ⊂ 0 f / L Lf X 0 + φ(f) 2 = . While Lf is a priori undefined for f / L, one∈ Xcircumvents⇔ k k thek formalk difficulty∞ by adopting the more permissive∈ dual X definition of the underlying semi-norm

0 Lf X 0 , f sup f, L∗ϕ = k k ∈ XL d h i + , otherwise, kϕkX ≤1: ϕ∈S(R ) ∞ d d which is valid for any f L∞,α(R ) in reason of the denseness of (R ) in ∈ ∗ d S X and the admissibility condition L ϕ L1,−α(R ). Under this interpretation, 0 ∈ d (28) is legitimate as well since L , L∞,α(R ). Indeed, the hypothesis of spline-admissibility (DeﬁnitionX 2) requires−→ that the growth-restricted null space of L be ﬁnite-dimensional and spanned by some basis p = (p1, . . . , pN0 ), d while one necessarily has that φ(f) 2 < for all f L∞,α(R ) because d iso. dk0 k ∞ d 00 ∈ φ L1,−α(R ) , L∞,α(R ) = L1,−α(R ) . The slight disadvantage Nis that⊂ (28) does not−→ fully specify the underlying topology.

4.3 Equivalent topologies and biorthogonal systems In order to show that the choice of one biorthogonal system over another— say, (φ˜, p˜) vs. (φ, p)—has no direct incidence on the definition of the underlying native space, we start by extending the range of validity of Theorems 3 ˜ and 4 to the complete set of admissible systems (φ, p˜) with p˜1,..., pÑ0 L, ˜ ˜ ˜ ˜ ∈ N φ1,..., φN0 L, and [φ(˜p1) φ(˜pN0 )] = I (biorthogonality). To that end, we rely∈ on X the existence of··· a primary biorthogonal system such that d φ = span φn (R ) (universality property), which ensures that the N { } ⊂ S initial pre-dual space L is well-defined. X Proposition 5. Let φ˜ = (φñ) with φñ L for n = 1,...,N0 be such that ˜ ˜ N0∈×N X0 the matrix C = [φ(p1) φ(pN0 )] R is invertible. Then, there exists ··· ∈ a unique basis p˜ of L = p˜ such that Theorems 3-4 and Corollaries 1-3 N N remain valid for the biorthogonal system (φ˜, p˜) and define a pair of native ˜0 0 ˜ ∗ and pre-dual spaces L = p˜ and L = φ˜, where = L ( ). This construction thenX specifiesU ⊕ fourN operatorsX withU ⊕ theN followingU properties:X

∗ ˜ c. PN0 ˜ ProjN = ProjN : L ˜ : g φn p˜n, g such that • φ˜ p˜ X −−→Nφ 7→ n=1 h i

φ ˜ : ProjN φ = φ ∀ ∈ Nφ φ˜ { } u : ProjN u = 0. ∀ ∈ U φ˜ { }

30 −1∗ −1∗ d ˜ c. L = L (Id ProjN ): L1,−α(R ) , L such that • φ˜ − φ˜ −→ X −−→X −1∗ φ ˜ :L φ = 0 ∀ ∈ Nφ φ˜ ˜ ∗ −1∗ g L :L L g = (Id ProjN ) g ∀ ∈ X φ˜ − φ˜ { } v :L−1∗L∗v = v. ∀ ∈ X φ˜

0 c. PN0 Proj : ˜ p˜ : f p˜n φ˜n, f such that • Np˜ XL −−→N 7→ n=1 h i

p p˜ : Proj p = p ∀ ∈ N Np˜{ } v 0 : Proj v = 0. ∀ ∈ U Np˜{ }

L−1 : 0 c. ˜0 such that • φ˜ X −−→ XL w 0 : φ˜(L−1 w ) = 0 ∀ ∈ X φ˜ { } w 0 : LL−1w = w ∀ ∈ X φ˜ f 0 :L−1L f = (Id Proj ) f . ∀ ∈ XL φ˜ { } − Np˜ { }

−1 Proof. The new basis p˜ of p = p˜ is given by p˜ = C p, which can easily N N be seen to be biorthogonal to φ˜. Next, we check that the operators Proj Nφ˜ ∗ and (Id ProjN ) are continuous on L = L ( ) p under the hypothesis − φ˜ X X ⊕ N that φ˜n L. Speciﬁcally, for any g L, we have that ∈ X ∈ X

N0 X ˜ ProjN g X p˜n, g φn X (by the triangle inequality) k φ˜ k L ≤ |h i| k k L n=1

N0 X ˜ p˜n X 0 g X φn X = C1 g X , ≤ k k L k k L k k L k k L n=1

PN0 where C1 = pñ 0 φñ X < . The property that Proj g n=1 XL L Nφ˜ k k k k ∞ c. { } ∈ ˜ (by construction) translates into ProjN : L ˜ , L. It is Nφ φ˜ X −−→Nφ −→X also obvious from the definition that Proj is the adjoint of Proj whose Nφ˜ Np˜ PN0 Schwartz kernel is (x, y) pñ(x)φñ(y). 7→ n=1 Likewise, we have that (Id ProjN ) g X (1+C1) g X . By invok- k − φ˜ { }k L ≤ k k L ˜ ing the biorthogonality of (φ, p˜), we readily verify that p˜(g ProjN g ) = − φ˜ { }

31 0 p(g ProjN g ) = 0, which yields (Id ProjN ) g , thereby ⇔ − φ˜ { } − φ˜ { } ∈ U c. −1∗ c. proving that (Id ProjN ): L , L. Since L : − φ˜ X −−→U −→X U −−→X (see Corollary 1), we can therefore chain the two operators, which results −1∗ c. c. in L (Id Proj ): L , thereby proving the conti- Nφ˜ ◦ −1∗ − c. X −−→U −−→X nuity of L : L . Finally, we invoke the continuous embedding φ˜ X −−→X d −1∗ d c. L1,−α(R ) , L (see Theorem 3), which ensures that L : L1,−α(R ) −→X φ˜ −−→ , in accordance with the last compatibility requirement in Deﬁnition 4. X Given that the underlying operators all satisfy the required continuity and annihilation properties, we can then revisit the proofs and constructions ˜ ˜0 in Theorems 3 and 4 to specify the corresponding pair of spaces L and L, 0 X X which inherit the same embedding properties as L and . X XL

An important outcome of the proof of Proposition 5 is that the compatibility condition for a single instance (φ, p) is transferred to all admissible biorthogonal systems (φ˜, p˜). Theorem 5 describes the eﬀect of such a change of biorthogonal system on the underlying norms, while it ensures that the underlying topologies are equivalent.

Theorem 5 (Equivalent direct-sum topologies). Let L be a spline-admissible operator that is compatible with 0 in the sense of Deﬁnition 4. Then, for X any two biorthogonal systems (φ, p) and (φ˜, p˜) with p = p˜ = L and N N N φ, φ˜ N0 , we have that ∈ XL 0 0 L = ˜L and = ˜ as sets; •X X XL XL the norms X and ˜ are equivalent on L; • k · k L k · kXL X 0 the norms X 0 and X˜0 are equivalent on L. • k · k L k · k L X More precisely, with the operator deﬁnitions in Proposition 5 and ˜ f X˜0 = Lf X 0 + φ(f) 2 k k L k k k k −1∗ g ˜ = L g X + p˜(g) 2, k kXL k φ˜ k k k we have the equivalence relations

0 0 (29) f L : Lf X = (Id ProjNp˜) f X˜0 ∀ ∈ X k k k − { }k L 0 f L : A1 f X 0 f X˜0 A2 f X 0 (30) ∀ ∈ X k k L ≤ k k L ≤ k k L p L = p : B1 φ(p) 2 φ˜(p) 2 B2 φ(p) 2 (31) ∀ ∈ N N k k ≤ k k ≤ k k

32 ∗ −1∗ −1∗ u = L ( ): L u X = (Id ProjN ) u X = L u X (32) ∀ ∈ U X k φ˜ k k − φ˜ { }k L k k 0 0 g L : A g X g ˜ A g X (33) ∀ ∈ X 1 k k L ≤ k kXL ≤ 2 k k L 0 0 g L : B p(g) 2 p˜(g) 2 B p(g) 2 (34) ∀ ∈ X 1 k k ≤ k k ≤ 2 k k 0 0 0 0 for some suitable constants A1,A2,B1,B2,A1,A2,B1,B2 > 0.

Note that the fixed parts of the construction are L = p = p˜ and N N N = L∗( ) (see Figure 1), which are associated with (31) and (32), respec- U X tively. On the other hand, we may have that φ = ˜, although what N 6 Nφ distinguishes those two spaces needs to be included in = L∗( ) in the ˜ U X sense that (Id ProjN ) φ , (Id ProjN ) φ for any φ φ and − φ˜ { } − φ { } ∈ U ∈ N φe ˜. In particular, when both φ and φe are biorthogonal to the same ∈ Nφ p = p˜, we have that p(φen φn) = 0 for n = 1,...,N0, so that the condition − φen L is equivalent to ∈ X −1∗ −1∗ L φen φn = L φen φn , (35) φ { − } { − } ∈ X which yields a criterion for admissibility that is convenient since it no longer −1∗ depends on Lφ . Proof. Proposition 5 ensures that underlying spaces and operators are well- defined. (i) Delineation as sets d To avoid circularity, we assume once more that φ = span φn (R ) N { } ⊂− S1 (universality condition). We then consider the generic members f = Lφ w+p −1 and f˜ = L w + p of the underlying spaces with w and p p = p˜ = φ˜ ∈ X ∈ N N span pñ . Since L f˜ f = (w w) = 0, the two functions can only differ { } { − ˜ } − 0 ˜0 by a components p˜ = (f f) p, which proves that L = L (as a set). − ∈ N X X∗ Likewise, on the side of the pre-dual space, we have that g = L v + φ L d ∈ X with (v, φ) ( φ) and φ (R ). Now, if ˜ L, we obviously ∈ X × N N ⊂ S Nφ ⊂ X ∗ ˜ ˜ also have that g˜ = L v + φ L for any φ φ˜ and v , which implies ∈ X d ∈ N ∗ ∈ X that ˜L L. Conversely, since (R ) , ˜L, g = L v + φ ˜L for any X ⊆ X S −→ X ∈ X (v, φ) ( φ), which yields L ˜L. ∈ X × N X ⊆ X (ii) Norm inequalities To get (29), we observe that 0 ˜ (Id ProjNp˜) f X˜0 = L f ProjNp˜ f X + φ f ProjNp˜ f 2 k − { }k L k − { } k k − { } k | {z } | {z } kLfkX 0 =0

33 because Proj f p˜ is annihilated by L and φ˜ f Proj f = 0 by Np˜{ } ∈ N − Np˜{ } construction, irrespective of the choice of (φ˜, p˜). Likewise, the dual relation −1∗ (32) is a direct consequence of the form of the adjoint operator Lφ = L−1∗(Id Proj ) and the orthogonality condition Proj u = 0 for all − Nφ Nφ { } u . ∈ U In order to estimate f X˜0 , we ﬁrst invoke the duality bound k k L ˜ ˜ φn, f φn X f X 0 |h i| ≤ k k L k k L with the role of L and ˜L (resp. φ˜n and φn) being interchangeable. This suggests the estimateX X

1 N0 ! 2 X ˜ 2 f X˜0 = Lf X 0 + φn, f k k L k k |h i| n=1

N0 X ˜ Lf X 0 + φn X f X 0 (by the triangle inequality) ≤ k k k k L k k L n=1

N0 ! X ˜ 1 + φn X f X 0 . (36) ≤ k k L k k L n=1 Likewise, we have that

N ! X0 f X 0 1 + φn X˜ f X˜0 k k L ≤ k k L k k L n=1 which, when combined with (36), yields (30) with

N !−1 N ! X0 X0 A1 = 1 + φn ˜ and A2 = 1 + φ˜n X . (37) k kXL k k L n=1 n=1 We apply a similar technique to derive (33) by considering the generic

34 ∗ element g = L v + φ L with v and φ φ. It leads to ∈ X ∈ X ∈ N 1 N0 ! 2 ∗ X 2 g ˜ = L v + φ ˜ = max( v X , pñ, φ ) k kXL k kXL k k |h i| n=1 N X0 max( v X , pñ X 0 φ X ) ≤ k k k k L k k L n=1 N ! X0 1 + pñ X 0 g X , ≤ k k L k k L n=1

| {z0 } A2 where we have used the property that g XL = max( v X , φ XL ). Likewise, k k0 k k k k the complementary bounding constant A1 is obtained by substituting φn by 0 pn and ˜L by ˜ in the ﬁrst part of (37). X XL As for the null-space component p p, we recall that any admissible ∈ N φ˜ must be such that the cross-product matrix

˜ ˜ N0×N0 C = [φ(p1) φ(pN0 )] R ··· ∈ has full rank for any basis p of p. The biorthogonal basis p˜ is then given by N p˜ = Bp, where B = C−1. The entries of these matrices are denoted by

bm,n = [B]m,n = φm, p˜n (38) h i cm,n = [C]m,n = φ˜m, pn , (39) h i respectively, where the right-hand side of (38) follows from the biorthogonal- PN0 ity of (φ, p). Let us now consider some arbitrary p = φn, p pn p n=1h i ∈ N whose initial norm is φ(p) 2. As we change the system of coordinates, we get k k 2 N0 N0 2 X X φ˜(p) = φn, p φ˜m, pn k k2 h ih i m=1 n=1

N0 N0 ! N0 ! X X 2 X 2 φ˜m, pn φn, p (by Cauchy-Schwarz) ≤ |h i| |h i| m=1 n=1 n=1 N N ! X0 X0 = c2 φ(p) 2. m,n k k2 m=1 n=1

35 Likewise, by interchanging the role of φ and φ˜, we ﬁnd that N N ! X0 X0 φ(p) 2 b2 φ˜(p) 2. k k2 ≤ m,n k k2 m=1 n=1

The combination of these two inequalities yields (31) with B2 = C F (the k k Frobenius norm of the matrix C) and B1 = 1/ B F . Similarly, we establish the norm inequalityk (34)k by constructing the estimates 2 N0 N0 N0 2 X 2 X X 2 2 p˜(u) = p˜n, u = bm,n pm, u B p(u) k k2 h i h i ≤ k kF k k2 n=1 n=1 m=1 2 N0 N0 N0 2 X 2 X X 2 2 p(u) = pn, u = cm,n p˜m, u C p˜(u) . k k2 h i h i ≤ k kF k k2 n=1 n=1 m=1

5 Link with classical results

5.1 Operator-based solution of differential equations −1 The use of the regularized inverse operator Lφ has been proposed for the resolution of stochastic partial differential equations of the form (see [35, 13]) Ls = w s.t. φ(s) = b, 0 d where w (R ) is a realization of a p-admissible white-noise innovation ∈ S N process and b R 0 is a boundary-condition vector that may be deter- ∈ ministic (the typical choice being b = 0) or not. Under the assumption −1∗ d c. d that L : (R ) Lp(R ) with p 1, the solution is then given by φ S −−→ ≥ −1 PN0 s = Lφ w + n=1 bnpn. The connection with the present work is that there are corresponding results available on specific choices of φ that guarantee the −1∗ continuity of Lφ [36, Chap 5]. The scenario most studied in one dimension is L = D with (φ1, p1) = (δ, 1), as it enables the construction of the whole −1 family of Lévy processes [20]. These can be described as s = Dδ w, where w is a white Lévy noise, which is compatible with the classical boundary condition s(0) = δ, s = 0 [36, Section 7.4, pp. 163-166]. Instead of the h i standard integrator D−1, which outputs the primitive of the function, the scheme uses the anti-derivative operator Z x −1 Dδ ϕ (x) = ϕ(y)dy, { } 0

36 −1∗ c. whose adjoint Dδ is continuous (R) (R) = α∈ZL∞,α(R) (the Fréchet space of rapidly decreasingS functions)−−→ R [36, Theorem∩ 5.3, p. 100]. d −1∗ Since Lq(R ) , (R) for all q 1 and Dδ ϕ Lq ϕ L1 , we can readily extend−→R the domain of continuity≤ ofk the{ adjoint}k ≤ pseudo-inverse k k to −1∗ c. d 0 0 Dδ : L1(R) Lq(R ). By taking = Lp(R) = Lq(R) with p 1, this then leads−−→ to the native Banach spacesX ≥

M Lp,D(R) = f : R R : f p,D = Df p + f(0) < , { → k k k k | | ∞} which are Sobolev spaces of degree 1. Theorem 4 ensures that (R) , S −→ Lp,D(R) , L∞(R), which is consistent with the classical embedding prop- −→ erties of Sobolev spaces. In fact, the statement can be reﬁned to Lp,D(R) , −→ Cb(R) for any p 1(see [33]). ≥ 5.2 Total variation and BV While the connection in Section 5.1 is enlightening, it does not cover the case 0 −1∗ (L, ) = D, (R) (total variation) with = C0(R) because D ϕ X M X δ { } does systematically present a discontinuity at the origin when ϕ, 1 = 0, even though it is smooth everywhere else (see [36, Figure 5.1, p.h 91]).i 6 The problem is that δ / C0,D(R). This can be ﬁxed by selecting a more regular ∈ boundary functional (i.e., any φ1 L1(R) with φ1, 1 = 1) which then yields ∈ h i c. a corrected operator that is universal in the sense that D−1∗ : ( ) ( ). φ1 R R It allows us to specify the proper native space S −−→S

M D(R) = f : R R f M,D = Df M + φ1, f < , M { → k k k k |h i| ∞} which extends BV(R) (functions of bounded variations) slightly. In the classical deﬁnition of BV(R), the second term in the norm is replaced by f 1. This is more constraining as it makes the null space trivial by excludingk con-k stant signals. In contrast with Lp,D(R) including the limit scenario p = 1, the continuity of the members of D(R) or BV(R) is guaranteed only al- M most everywhere: in other words, it can happen that the term f(0) is not well-deﬁned, which is the fundamental reason why it needs to| be replaced| by φ1, f . |h i| 5.3 Sobolev/Beppo-Levi spaces with d = 1 N We have seen that the N0th derivative operator L = D 0 is spline-admissible n N0−1 with α = (N0 1) and DN0 = x n=0 (polynomials of degree N0 1). − N −{1∗ } c. − Since we already known that D : L1(R) Lp(R), p = 1 included, δ −−→

37 we can iterate the operator to construct an admissible pseudo-inverse of Dm∗ = ( 1)mDm as − −m∗ −1∗m c. D = D : L1(R) Lp(R). φ δ −−→ (n−1) (n−1) −1 with N0 = m and φn = ( 1) δ . Indeed, since f = D w imposes − δ { } the boundary condition f(0) = 0 and is left-invertible with w = Df, g = m D−1 w is invertible as well and such that g(n)(0) = ( 1)(n−1)δ(n−1), g = δ { } h − i 0 = φn+1, g for n = 0,..., (m 1) By observing that the underlying φn h i xn − are biorthogonal to p˜n(x) = n! , we can then safely deﬁne the corresponding native spaces as    21/2  m−1 (n)   M m X f (0)  Lp,Dm (R) = f : R R : f p,Dm = D f p +   < , → k k k k n! ∞  n=0  which, as expected, are Sobolev spaces of order m, albeit homogeneous extensions of the classical ones for they also includes the polynomials of degree less than m. For p = 2, we recover the typical kind of Beppo-Levi space [2] used to specify smoothing splines; i.e., the classical form of variational polynomial splines, which goes back to the pioneering works of Schoenberg and de Boor [30, 10].

5.4 Reproducing-kernel Hilbert spaces d The best known examples of native spaces on R are RKHS [1, 4, 38, 29]. d They are included in the framework by taking = L2(R ). In that case, −1∗ d c. X d −1 the stability condition Lφ : L1,−α(R ) L2(R ) implies that Lφ : d c. d −−→ −1∗ L2(R ) L∞,α(R ). This means that the canonical inverse operator L −−→ −1∗ −1∗ ∗ d c. d has the unique extension L = Lφ : = L L2(R ) L2(R ) −1∗ −1∗ U −−→ with u 2,L = L u 2 = L u L for any u . This allows us to k k k k k φ k 2 ∈ U deﬁne the pair of self-adjoint operators A = (L−1L−1∗): c. 0 and −1 −1∗ d c. 0 d d d.U −−→U Aφ = (Lφ Lφ ): L2,L(R ) L2,L(R ). Since p⊥ (R ) , , L (by Corollary 1), we have that −−→ S −→U −→X

d −1∗ −1∗ ϕ p(R ) 0 : ϕ 2,L = L ϕ, L ϕ ∀ ∈ S \{ } k k h i = Aϕ, ϕ = Aφϕ, ϕ > 0. (40) h i h i

This expresses the (strict) p-conditional positive deﬁniteness of A (resp. Aφ) which can also be identiﬁed as the inverse (resp. the pseudo-inverse) of (L∗L).

38 In the particular case where p is a basis of the polynomials of degree n0, Condition (40) is equivalent to the notion of n0th-order conditional positive definiteness used in approximation theory [22, 38]. It is a classical hypothesis in the theory of (semi-)RKHS and is also necessary for our construction. Classically, the strict (n0th-order conditional) positive definiteness of A (or of its underlying kernel) it known to be sufficient to yield a (semi-)RKHS. d This is not quite the case here because we also want the embedding (R ) , 0 d d S −→ L2,L(R ) , L∞,α(R ) that controls the growth of the members of the native −→ −1∗ d c. d space. The latter calls for the continuity of Lφ : L1,−α(R ) L2(R ) d c. d −−→ (L2-stability) or, equivalently, of Aφ : L1,−α(R ) L∞,α(R ). −−→ 5.5 Connections with kernel methods and splines In [33], we shall identify a simple condition on the kernel of A = (LL∗)−1 d that ensures that the stability requirement in Definition 4 for = L2(R ) is met. This will enable us to prove that the combination of splineX admissibility in Definition 2 and the classical (conditional-)positivity requirement (40) are 0 0 d necessary and sufficient for the native space of ( , L) with = = L2(R ) dX 0 Xd X d to be a RKHS, with the property that (R ) , L2,L(R ) , Cb,α(R ). We shall further the argument by reformulatingS −→ the primary results−→ of the present paper in terms of kernels, rather than operators. This will provide us with explicit criteria for checking that the compatibility conditions in Definition 4 are met for a broad variety of primary spaces 0. We shall also 0 d X devote a particular attention to the case = (R ), which is central to X M the theory of L-splines [34]. Examples of applications of our native Banach- space formalism, including the resolution of variational inverse problems and the derivation of representer theorems, will be presented in [14].

39 Appendix A: Topological embeddings

The notion of embedding for topological vector spaces comes in four gra- dation: inclusion as a set (symbolized by ), continuous embedding iso. X ⊆ Y ( , ), isometric embedding ( , ), and, finally, continuous and X −→Y d. X −→ Y dense embedding ( , ). X −→Y Definition 7 (Continuous embedding). Let and be two locally convex topological vector spaces where (as a set).X isY said to be continuously X ⊆ Y X embedded in , which is denoted by , , if the inclusion/identity map Y X −→Y I: : x x is continuous. X → Y 7→ In particular, if and are two Banach spaces, then the definition can X Y be restated as: for all x , I x = x with x Y C0 x X for some ∈ X { } ∈ Y k k ≤ k k constant C0 > 0. If, in addition, x X = x Y for all x , then the k k k k iso.∈ X ⊆ Y embedding is isometric, a property that is denoted by , . For instance, a classical result is that any Banach space is isometricallyX −→Y embedded in iso. X its bidual; i.e., , 00. In fact, we have that = 00 (meaning that the two spaces are isometricallyX −→X isomorphic) if and onlyX ifX is reflexive. An example of such embeddings that is relevant toX this paper is

d d iso. d 00 d 0 (R ) , L1,−α(R ) , L1,−α(R ) = (L∞,α(R ) . S −→ −→ So far we have emphasized the property of continuity, but there are in- d d stances such as (R ) , L1,−α(R ) that are more powerful because the S −→ d. embedding also happens to be dense; i.e., , , where can be speciﬁed X −→Y Y as the completion of for Y . X k · k Deﬁnition 8 (Dense embedding). Let be a linear subspace of a locally convex topological vector space . Then,X is said to be dense in if it has the ability to separate distinct elementsY ofX the dual space 0; thatY is, if, for Y any y0 0, ∈ Y 0 0 y , x Y0×Y = 0 for all x y = 0. h i ∈ X ⊆ Y ⇔ In the case where is a Banach space, the denseness of has another Y X equivalent formulation: for any y and > 0, there exists some x ∈ Y ∈ X such that y x Y < , which means that is rich enough to represent any elementk of− withk an arbitrary degree ofX precision. Y

40 Theorem 6 (Dual embedding). Let and be two locally convex topological d. X Y vector spaces such that , , where the embedding is continuous and X −→Y d. dense. Then, 0 , 0. Moreover, if is reflexive, then 0 , 0. Y −→X X iso. Y −→X Likewise, in the case of Banach spaces, , implies that 0 , 0 with bounding constant one (preservation of norm).X −→Y Moreover, if thereY −→X exists d. d. iso. a topological vector space such that , and , , then 0 , 0. S S −→X S −→Y Y −→X Proof. Since , , the linear functionals that are continuous on are also continuousX on−→Y so that 0 0 (as a set). Moreover, the continuityY X X ⊆ Y of the identity/inclusion map I: implies the continuity of its adjoint X → Y I∗ : 0 0, defined as Y → X ∗ 0 0 0 I y , x X 0×X = y , I x Y0×Y = y , x Y0×Y (41) h { } i h { }i h i for all y0 0, x . Finally, the denseness of in ensures that I∗ is the ∈ Y ∈ X X Y correct inclusion map with I∗ y0 = y0, which proves that 0 , 0. { } Y −→X Second part by contradiction: Suppose that 0 is not dense in 0. Then, there 00 00 Y 00 0 X is an x that is not identically zero such that x , y X 00×X 0 = 0 for all 0 ∈ X h 0 i y0 0 0 (contrapositive of the statement in Definition 8). Moreover, ∈ Y ⊆ X due to the reflexivity of , there is a corresponding x0 , such that 00 X 00 ∈ X −→Y B x0 = x0, where B: is the canonical bijective mapping for a reflexive{ } space to its bidual.X → Therefore, X

00 0 0 0 = x0, y X 00×X 0 = B x0 , y X 00×X 0 h 0 i h0 { } i = y , x0 X 0×X = y , x0 Y0×Y , h i h i for all y0 0. Since the topological spaces 0 and form a dual pair, the ∈0 Y Y Y identity y , x0 Y0×Y = 0 implies that x0 = 0, which is a contradiction. h i Banach Isometries: From the deﬁnition of the dual norm and the property 0 0 0 that ϕ X = ϕ Y when ϕ , for any y , we have that k k k k ∈ X ∈ Y ⊆ X

0 y, ϕ y, ϕ y, ϕ 0 y Y0 = sup h i sup h i = sup h i = y X 0 (42) k k ϕ Y ≥ ϕ Y ϕ X k k ϕ∈Y\{0} k k ϕ∈X \{0} k k ϕ∈X \{0} k k where the inequality results from the search space on the right being a X subset of ; hence, 0 , 0. In the second scenario, the two search spaces in (42) can beY replacedX by−→Y the dense subspace , which then yields an equality. S However, this setting also implies that = as both spaces are the unique X Y completion of in the X = Y -norm. S k · k k · k

41 As demonstration of usage, we now prove the following remarkable result in Schwartz’ theory of distributions:

d 0 d Proposition 6. (R ) is continuously and densely embedded in (R ); i.e., d d. 0 d S S (R ) , (R ). S −→S d R Proof. For any φ (R ), the map ϕ φ, ϕ = d φ(x)ϕ(x)dx speciﬁes a ∈ S 7→d h i R d continuous linear functional over (R ), which already shows that (R ) 0 d S S ⊆ (R ) (as a set). To prove that the embedding is continuous, we invoke S d 0 d the continuity of the identity operator I: (R ) (R ), which is obvious from the underlying topology. As for theS denseness→ S property, the relevant annihilator space is

⊥ 0 d 0 d d = ϕ (R ) = (R ): ϕ, φ = 0 for all φ (R ) . S { ∈ S S h i ∈ S } ⊥ d In particular, if ϕ (R ), then ϕ, ϕ = 0 ϕ L2 = 0 ϕ = 0, ∈ S ⊆ S h i ⇔ k k ⇔ which proves that ⊥ = 0 . S { } In practice, it is often easier to prove that an embedding is continuous than establishing its denseness. Fortunately, it is possible to transfer such properties by taking advantage of functional hierarchies.

Proposition 7 (Hierarchy of dense embeddings). Let , , and be three X Y Z locally convex topological vector spaces such that , , (continuous embeddings). Then, we have the following implicationsX −→Y (dense−→Z embeddings).

d. d. d. 1. , and , , X −→Y Y −→Z ⇒ X −→Z d. d. 2. , , . X −→Z ⇒ Y −→Z Proof. Statement 1: as sets. Since the closure of in the topology of is , the closureX ⊆ Y of ⊆ Zin the topology of must alsoY be . Z Z X Z Z Statement 2: The annihilators of and in 0 are X ⊆ Z Y ⊆ Z Z ⊥ 0 = u : u, x Z0×Z = 0 for all x X { ∈ Z h i ∈ X } ⊥ 0 = u : u, y Z0×Z = 0 for all y Y { ∈ Z h i ∈ Y} with the property that ⊥ ⊥ (as a set) from the deﬁnition. Hence, Y ⊆ X ⊥ = 0 ⊥ = 0 . X { } ⇒ Y { }

42 Appendix B: Direct-sum topology

Definition 9. Let and be two subspaces of a linear space . Then, the U V W sum space is + = f = u + v :(u, v) . The sum is called U V { ∈ U × V} ⊂ W direct and is notated if = 0 . U ⊕ V U ∩ V { } If and in the above definition are normed with respective norms U V U and V , then ( u U , v V ) p is a norm for both and . k · k k · k k k k k k k U × V U ⊕ V The choice of the exponent p 1 in the composite norm is flexible since all finite-dimensional norms are equivalent.≥ One also defines the corresponding linear projection operators Proj :( ) and Proj :( ) . U U ⊕ V → U V U ⊕ V → V These are such that, for any (u, v) , ∈ U × V Proj u + v = u, (43) U { } Proj u + v = v. (44) V { } In summary, any f has a unique decomposition as f = u + ∈ U ⊕ V v with Proj f = u and Proj f = v , while f U⊕V = U { } ∈ U V { } ∈ V k k ( ProjU f U , ProjV f V ) p. k k The concept{ }k k is also applicable{ }k k to Banach spaces, which are often specified explicitly as the completion of some normed space. This normed space is called a pre-Banach space when it is not yet completed.

Theorem 7 (Completion of a direct sum space). Let pre pre be the direct U ⊕V sum of two normed spaces ( pre, U ) and ( pre, V ). Then, pre pre = U k·k V k·k U ⊕ V , where = ( pre, U ) and = ( pre, V ) are the Banach spaces U ⊕ V U U k · k V V k · k associated with the U -norm and V -norm, respectively. Moreover, the k · k k · k c. resulting projection operators ProjU : with ProjU = 1 and c. U ⊕ V −−→U k k Proj : with Proj = 1 are the unique continuous extensions V U ⊕ V −−→V k V k of Proj : pre pre and Proj : pre pre , respectively. Upre U ⊕ V → U Vpre U ⊕ V → V Proof. Let pre = pre pre, whose completion with respect to the direct- G U ⊕ V sum norm is the Banach space . We then pick some Cauchy sequence (gi) in G pre that converges to g . Because the projectors Proj : pre pre G ∈ G Upre P → U and Proj : pre pre are contractive maps, the transformed sequences Vpre P → V (ui) = (Proj gi ) and (vi) = (Proj gi ) are Cauchy in pre and pre, Upre { } Vpre { } U V M respectively, and converge to some limits u = limi→∞ ui = pre and M ∈ U U v = limi→∞ vi = pre. Hence, u + v + . Conversely, because of the ∈ V V ∈ U V direct sum property, the sum of any two Cauchy sequences in pre and pre U V is Cauchy in pre converges to limi→∞(ui + vi) = limi→∞ ui + limi→∞ vi = G

43 u + v pre = , which shows that + . Hence, we conclude that ∈ G G U V ⊆ G = + . G U V iso. Since pre , and Proj is a projector, it is bounded pre U −→ U Upre G → U with ProjUpre = 1. By the B.L.T. theorem, it therefore admits a unique k k c. continuous extension ProjU : pre = with ProjU = 1, whose explicit deﬁnition is G G −−→U k k

M ProjU g = lim ProjU gi { } i→∞ pre { } where (gi) is any Cauchy sequence in pre that converges to g . The same c. G ∈ G holds true for Proj : , which is such that V G −−→V M ProjV g = lim ProjV gi = lim vi = v. { } i→∞ pre { } i→∞ We then use these extended operators to show that the sum + is direct. U V Speciﬁcally, by invoking the basic properties of ProjUpre , we get

ProjU g = lim ProjU ProjU gi + ProjV gi { } i→∞ pre { pre { } pre { }} = lim ProjU ui + ProjU vi = lim (ui + 0) = u , i→∞ pre { } pre { } i→∞ ∈ U which is equivalent to Proj u + v = u for any (u, v) . Correspond- U { } ∈ U × V ingly, we also obtain that Proj g = limi→∞(0 + vi) = v = (g u) with V − ∈ V Proj u = 0 and Proj v = v, which proves that = 0 . V { } V { } U ∩ V { }

The ﬁnal element is the identiﬁcation of the dual space which, as expected, also has a direct-sum structure (see [21, Theorem 1.10.13]), albeit with a suitable adaptation of the composite norm.

Proposition 8. Let ( , U ) and ( , V ) be two complementary Banach subspaces of and U k · kthe correspondingV k · k direct-sum space equipped with W U ⊕ V the composite norm ( u U , v V ) p. Then, the continuous dual of is the the direct-sumk Banachk k k spacek k 0 0 equipped with the dual compositeU ⊕ V 0 0 0 U0 ⊕ V p norm u + v U 0⊕V0 = ( u U 0 , v V0 ) q where q = is the conjugate k k k k k k k k p−1 exponent of p 1. ≥ Acknowlegdments The research was partially supported by the Swiss National Science Foun- dation under Grant 200020-162343. The authors are thankful to John-Paul Ward and Shayan Aziznejad for helpful discussions.

44 References

[1] N. Aronszajn. Theory of reproducing kernels. Transactions of the Amer- ican Mathematical Society, 68(3):337–404, 1950.

[2] M. Atteia. Hilbertian Kernels and Spline Functions. Elsevier, 1992.

[3] R. K. Beatson, H.-Q. Bui, and J. Levesley. Embeddings of Beppo– Levi spaces in Hölder–Zygmund spaces, and a new method for radial basis function interpolation error estimates. Journal of Approximation Theory, 137(2):166–178, 2005.

[4] A. Berlinet and C. Thomas-Agnan. Reproducing Kernel Hilbert Spaces in Probability and Statistics, volume 3. Kluwer Academic Boston, 2004.

[5] A. Y. Bezhaev and V. A. Vasilenko. Variational Theory of Splines. Kluwer Academic/Plenum Publishers, New York, 2001.

[6] C. Boyer, A. Chambolle, Y. De Castro, V. Duval, F. De Gournay, and P. Weiss. On representer theorems and convex regularization. arXiv preprint arXiv:1806.09810, 2018.

[7] K. Bredies and M. Carioni. Sparsity of solutions for variational inverse problems with ﬁnite-dimensional data. arXiv preprint arXiv:1809.05045, 2018.

[8] A. Chambolle. An algorithm for total variation minimization and applications. Journal of Mathematical Imaging and Vision, 20:89–97, 2004.

[9] C. de Boor. On “best” interpolation. Journal of Approximation Theory, 16(1):28–42, 1976.

[10] C. de Boor and R. Lynch. On splines and their minimum properties. Journal of Mathematics and Mechanics, 15(6):953–969, 1966.

[11] J. Deny and J.-L. Lions. Les espaces du type de Beppo Levi. Annales de l’Institut Fourier, 5:305–370, 1954.

[12] J. Duchon. Splines minimizing rotation-invariant semi-norms in Sobolev spaces. In W. Schempp and K. Zeller, editors, Constructive Theory of Functions of Several Variables, pages 85–100. Springer-Verlag, Berlin, 1977.

45 [13] J. Fageot, A. Amini, and M. Unser. On the continuity of characteristic functionals and sparse stochastic modeling. The Journal of Fourier Analysis and Applications, 20(6):1179–1211, December 2014.

[14] J. Fageot, M. Unser, and J. Ward. Native Banach spaces: Concrete criteria for existence. Under development.

[15] J. Ferreira and V. Menegatto. Positive deﬁniteness, reproducing kernel Hilbert spaces and beyond. Annals of Functional Analysis, 4(1):64–88, 2013.

[16] A. Flinth and P. Weiss. Exact solutions of inﬁnite dimensional total- variation regularized problems. arXiv preprint arXiv:1708.02157, 2017.

[17] I. Gelfand and N. Y. Vilenkin. Generalized Functions. Vol. 4. Applica- tions of Harmonic Analysis. Academic press, New York, USA, 1964.

[18] H. Gupta, J. Fageot, and M. Unser. Continuous-domain solutions of linear inverse problems with Tikhonov versus generalized TV regularization. IEEE Transactions on Signal Processing, 66(17):4670–4684, September 1, 2018.

[19] L. Hörmander. The analysis of linear partial diﬀerential operators.I. Distribution Theory and Fourier Analysis, volume 256. Springer-Verlag, Berlin, second edition, 1990.

[20] P. Lévy. Théorie de l’Addition des Variables Aléatoires. Gauthier- Villars, Paris, 2nd edition, 1954.

[21] R. E. Megginson. An Introduction to Banach Space Theory. Springer, 1998.

[22] C. A. Micchelli. Interpolation of scattered data: Distance matrices and conditionally positive deﬁnite functions. Constructive Approximation, 2(1):11–22, 1986.

[23] A. Mosamam and J. Kent. Semi-reproducing kernel Hilbert spaces, splines and increment kriging. Journal of Nonparametric Statistics, 22(6):711–722, 2010.

[24] P. Prenter. Splines and Variational Methods. Wiley, New York, 1975.

[25] M. Reed and B. Simon. Methods of Modern Mathematical Physics. Vol. 1: Functional Analysis. Academic Press, 1980.

46 [26] L. I. Rudin, S. Osher, and E. Fatemi. Nonlinear total variation based noise removal algorithms. Physica D, 60(1-4):259–268, 1992.

[27] W. Rudin. Functional Analysis. McGraw-Hill, New York, 2nd edition, 1991. McGraw-Hill Series in Higher Mathematics.

[28] R. Schaback. Native Hilbert spaces for radial basis functions I. In New Developments in Approximation Theory, pages 255–282. Springer, 1999.

[29] R. Schaback. A uniﬁed theory of radial basis functions: Native Hilbert spaces for radial basis functions II. Journal of Computational and Ap- plied Mathematics, 121(1-2):165–177, 2000.

[30] I. Schoenberg. Spline functions and the problem of graduation. Pro- ceedings of the National Academy of Sciences, 52(4):947–950, October 1964.

[31] M. Unser. A representer theorem for deep neural networks. arXiv preprint arXiv:1802.09210, 2018.

[32] M. Unser and T. Blu. Self-similarity: Part I—Splines and operators. IEEE Transactions on Signal Processing, 55(4):1352–1363, April 2007.

[33] M. Unser and J. Fageot. Native Banach spaces: From operators to kernels. Under development.

[34] M. Unser, J. Fageot, and J. P. Ward. Splines are universal solutions of linear inverse problems with generalized-TV regularization. SIAM Review, 59(4):769–793, December 2017.

[35] M. Unser, P. Tafti, and Q. Sun. A uniﬁed formulation of Gaussian versus sparse stochastic processes—Part I: Continuous-domain theory. IEEE Transactions on Information Theory, 60(3):1945–1962, March 2014.

[36] M. Unser and P. D. Tafti. An Introduction to Sparse Stochastic Pro- cesses. Cambridge University Press, 2014.

[37] G. Wahba. Spline Models for Observational Data. Society for Industrial and Applied Mathematics, Philadelphia, PA, 1990.

[38] H. Wendland. Scattered Data Approximations. Cambridge University Press, 2005.