DOCSLIB.ORG

Foundations of Reproducing Kernel Hilbert Spaces Advanced Topics in Machine Learning

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Foundations of Reproducing Kernel Hilbert Spaces Advanced Topics in Machine Learning Foundations of Reproducing Kernel Hilbert Spaces Advanced Topics in Machine Learning D. Sejdinovic, A. Gretton Gatsby Unit March 6, 2012 D. Sejdinovic, A. Gretton (Gatsby Unit) Foundations of RKHS March 6, 2012 1 / 60 Overview 1 Elementary Hilbert space theory Norm. Inner product. Orthogonality Convergence. Complete spaces Linear operators. Riesz representation 2 What is an RKHS? Evaluation functionals view of RKHS Reproducing kernel Inner product between features Positive denite function Moore-Aronszajn Theorem D. Sejdinovic, A. Gretton (Gatsby Unit) Foundations of RKHS March 6, 2012 2 / 60 RKHS: a function space with a very special structure linear / vector RKHS unitary Hilbert normed Banach complete metric D. Sejdinovic, A. Gretton (Gatsby Unit) Foundations of RKHS March 6, 2012 3 / 60 Elementary Hilbert space theory Norm. Inner product. Orthogonality Outline 1 Elementary Hilbert space theory Norm. Inner product. Orthogonality Convergence. Complete spaces Linear operators. Riesz representation 2 What is an RKHS? Evaluation functionals view of RKHS Reproducing kernel Inner product between features Positive denite function Moore-Aronszajn Theorem D. Sejdinovic, A. Gretton (Gatsby Unit) Foundations of RKHS March 6, 2012 4 / 60 In every normed vector space, one can dene a metric induced by the norm: d(f ; g) = f g : k − kF Elementary Hilbert space theory Norm. Inner product. Orthogonality Normed vector space Denition (Norm) Let be a vector space over the eld R of real numbers (or C).A F function : [0; ) is said to be a norm on if k·kF F! 1 F 1 f = 0 if and only if f = 0 (norm separates points), k kF 2 λf = λ f , λ R, f (positive homogeneity), k kF j j k kF 8 2 8 2 F 3 f + g f + g , f ; g (triangle inequality). k kF ≤ k kF k kF 8 2 F D. Sejdinovic, A. Gretton (Gatsby Unit) Foundations of RKHS March 6, 2012 5 / 60 Elementary Hilbert space theory Norm. Inner product. Orthogonality Normed vector space Denition (Norm) Let be a vector space over the eld R of real numbers (or C).A F function : [0; ) is said to be a norm on if k·kF F! 1 F 1 f = 0 if and only if f = 0 (norm separates points), k kF 2 λf = λ f , λ R, f (positive homogeneity), k kF j j k kF 8 2 8 2 F 3 f + g f + g , f ; g (triangle inequality). k kF ≤ k kF k kF 8 2 F In every normed vector space, one can dene a metric induced by the norm: d(f ; g) = f g : k − kF D. Sejdinovic, A. Gretton (Gatsby Unit) Foundations of RKHS March 6, 2012 5 / 60 p = 1: Manhattan p = 2: Euclidean p : maximum norm, x = maxi xi ! 1 k k1 j j 1 p d Pd p = = R : x p = i 1 xi ; p 1 F k k = j j ≥ 1 p b p = = C[a; b]: f p = a f (x) dx ; p 1 F k k ´ j j ≥ Elementary Hilbert space theory Norm. Inner product. Orthogonality Examples of normed linear spaces (R; ), (C; ) |·| |·| D. Sejdinovic, A. Gretton (Gatsby Unit) Foundations of RKHS March 6, 2012 6 / 60 p = 1: Manhattan p = 2: Euclidean p : maximum norm, x = maxi xi ! 1 k k1 j j 1 p b p = = C[a; b]: f p = a f (x) dx ; p 1 F k k ´ j j ≥ Elementary Hilbert space theory Norm. Inner product. Orthogonality Examples of normed linear spaces (R; ), (C; ) |·| |·| 1 p d Pd p = = R : x p = i 1 xi ; p 1 F k k = j j ≥ D. Sejdinovic, A. Gretton (Gatsby Unit) Foundations of RKHS March 6, 2012 6 / 60 1 p b p = = C[a; b]: f p = a f (x) dx ; p 1 F k k ´ j j ≥ Elementary Hilbert space theory Norm. Inner product. Orthogonality Examples of normed linear spaces (R; ), (C; ) |·| |·| 1 p d Pd p = = R : x p = i 1 xi ; p 1 F k k = j j ≥ p = 1: Manhattan p = 2: Euclidean p : maximum norm, x = maxi xi ! 1 k k1 j j D. Sejdinovic, A. Gretton (Gatsby Unit) Foundations of RKHS March 6, 2012 6 / 60 Elementary Hilbert space theory Norm. Inner product. Orthogonality Examples of normed linear spaces (R; ), (C; ) |·| |·| 1 p d Pd p = = R : x p = i 1 xi ; p 1 F k k = j j ≥ p = 1: Manhattan p = 2: Euclidean p : maximum norm, x = maxi xi ! 1 k k1 j j 1 p b p = = C[a; b]: f p = a f (x) dx ; p 1 F k k ´ j j ≥ D. Sejdinovic, A. Gretton (Gatsby Unit) Foundations of RKHS March 6, 2012 6 / 60 In every inner product vector space, one can dene a norm induced by the inner product: 1=2 f = f ; f : k kF h iF Elementary Hilbert space theory Norm. Inner product. Orthogonality Inner product Denition (Inner product) Let be a vector space over . A function is said to R ; F : R be anF inner product on if h· ·i F × F ! F 1 α1f1 + α2f2; g = α1 f1; g + α2 f2; g h iF h iF h iF 2 f ; g = g; f (conjugate symmetry if over C) h iF h iF 3 f ; f 0 and f ; f = 0 if and only if f = 0. h iF ≥ h iF D. Sejdinovic, A. Gretton (Gatsby Unit) Foundations of RKHS March 6, 2012 7 / 60 Elementary Hilbert space theory Norm. Inner product. Orthogonality Inner product Denition (Inner product) Let be a vector space over . A function is said to R ; F : R be anF inner product on if h· ·i F × F ! F 1 α1f1 + α2f2; g = α1 f1; g + α2 f2; g h iF h iF h iF 2 f ; g = g; f (conjugate symmetry if over C) h iF h iF 3 f ; f 0 and f ; f = 0 if and only if f = 0. h iF ≥ h iF In every inner product vector space, one can dene a norm induced by the inner product: 1=2 f = f ; f : k kF h iF D. Sejdinovic, A. Gretton (Gatsby Unit) Foundations of RKHS March 6, 2012 7 / 60 b = C[a; b]: f ; g = f (x)g(x)dx F h i a d×d ´ > = R : A; B = Tr AB F h i Elementary Hilbert space theory Norm. Inner product. Orthogonality Examples of inner product d Pd = R : x; y = i 1 xi yi F h i = D. Sejdinovic, A. Gretton (Gatsby Unit) Foundations of RKHS March 6, 2012 8 / 60 d×d > = R : A; B = Tr AB F h i Elementary Hilbert space theory Norm. Inner product. Orthogonality Examples of inner product d : x y Pd x y = R ; = i=1 i i F h i b = C[a; b]: f ; g = a f (x)g(x)dx F h i ´ D. Sejdinovic, A. Gretton (Gatsby Unit) Foundations of RKHS March 6, 2012 8 / 60 Elementary Hilbert space theory Norm. Inner product. Orthogonality Examples of inner product d : x y Pd x y = R ; = i=1 i i F h i b = C[a; b]: f ; g = f (x)g(x)dx F h i a d×d ´ > = R : A; B = Tr AB F h i D. Sejdinovic, A. Gretton (Gatsby Unit) Foundations of RKHS March 6, 2012 8 / 60 M? is a linear subspace of ; M M? 0 F \ ⊆ f g Elementary Hilbert space theory Norm. Inner product. Orthogonality Angles. Orthogonality Angle θ between f ; g 0 is given by: 2 Fn f g f g cos ; F θ = fh ig k kF k kF Denition We say that f is orthogonal to g and write f g, if f g 0. For ; F = M , the orthogonal complement of M is:? h i ⊂ F M? := g : f g; f M : f 2 F ? 8 2 g D. Sejdinovic, A. Gretton (Gatsby Unit) Foundations of RKHS March 6, 2012 9 / 60 Elementary Hilbert space theory Norm. Inner product. Orthogonality Angles. Orthogonality Angle θ between f ; g 0 is given by: 2 Fn f g f g cos ; F θ = fh ig k kF k kF Denition We say that f is orthogonal to g and write f g, if f g 0. For ; F = M , the orthogonal complement of M is:? h i ⊂ F M? := g : f g; f M : f 2 F ? 8 2 g M? is a linear subspace of ; M M? 0 F \ ⊆ f g D. Sejdinovic, A. Gretton (Gatsby Unit) Foundations of RKHS March 6, 2012 9 / 60 2 2 2 f g = f + g = f + g (Pythagorean theorem) ? ) k k k k k k Elementary Hilbert space theory Norm. Inner product. Orthogonality Key relations in inner product space f ; g f g (Cauchy-Schwarz inequality) j h 2i j ≤ k k2 · k k 2 2 2 f + 2 g = f + g + f g (the parallelogram law) k k k k 2k k 2k − k 4 f ; g = f + g f g (the polarization identity) h i k k − k − k D. Sejdinovic, A. Gretton (Gatsby Unit) Foundations of RKHS March 6, 2012 10 / 60 Elementary Hilbert space theory Norm. Inner product. Orthogonality Key relations in inner product space f ; g f g (Cauchy-Schwarz inequality) j h 2i j ≤ k k2 · k k 2 2 2 f + 2 g = f + g + f g (the parallelogram law) k k k k 2k k 2k − k 4 f ; g = f + g f g (the polarization identity) h i k k − k − k 2 2 2 f g = f + g = f + g (Pythagorean theorem) ? ) k k k k k k D.
Load more

Recommended publications
  • FUNCTIONAL ANALYSIS 1. Banach and Hilbert Spaces in What
    FUNCTIONAL ANALYSIS PIOTR HAJLASZ 1. Banach and Hilbert spaces In what follows K will denote R of C. Definition. A normed space is a pair (X, k · k), where X is a linear space over K and k · k : X → [0, ∞) is a function, called a norm, such that (1) kx + yk ≤ kxk + kyk for all x, y ∈ X; (2) kαxk = |α|kxk for all x ∈ X and α ∈ K; (3) kxk = 0 if and only if x = 0. Since kx − yk ≤ kx − zk + kz − yk for all x, y, z ∈ X, d(x, y) = kx − yk defines a metric in a normed space. In what follows normed paces will always be regarded as metric spaces with respect to the metric d. A normed space is called a Banach space if it is complete with respect to the metric d. Definition. Let X be a linear space over K (=R or C). The inner product (scalar product) is a function h·, ·i : X × X → K such that (1) hx, xi ≥ 0; (2) hx, xi = 0 if and only if x = 0; (3) hαx, yi = αhx, yi; (4) hx1 + x2, yi = hx1, yi + hx2, yi; (5) hx, yi = hy, xi, for all x, x1, x2, y ∈ X and all α ∈ K. As an obvious corollary we obtain hx, y1 + y2i = hx, y1i + hx, y2i, hx, αyi = αhx, yi , Date: February 12, 2009. 1 2 PIOTR HAJLASZ for all x, y1, y2 ∈ X and α ∈ K. For a space with an inner product we define kxk = phx, xi . Lemma 1.1 (Schwarz inequality).
    [Show full text]
  • MAS4107 Linear Algebra 2 Linear Maps And
    Introduction Groups and Fields Vector Spaces Subspaces, Linear . Bases and Coordinates MAS4107 Linear Algebra 2 Linear Maps and . Change of Basis Peter Sin More on Linear Maps University of Florida Linear Endomorphisms email: [email protected]fl.edu Quotient Spaces Spaces of Linear . General Prerequisites Direct Sums Minimal polynomial Familiarity with the notion of mathematical proof and some experience in read- Bilinear Forms ing and writing proofs. Familiarity with standard mathematical notation such as Hermitian Forms summations and notations of set theory. Euclidean and . Self-Adjoint Linear . Linear Algebra Prerequisites Notation Familiarity with the notion of linear independence. Gaussian elimination (reduction by row operations) to solve systems of equations. This is the most important algorithm and it will be assumed and used freely in the classes, for example to find JJ J I II coordinate vectors with respect to basis and to compute the matrix of a linear map, to test for linear dependence, etc. The determinant of a square matrix by cofactors Back and also by row operations. Full Screen Close Quit Introduction 0. Introduction Groups and Fields Vector Spaces These notes include some topics from MAS4105, which you should have seen in one Subspaces, Linear . form or another, but probably presented in a totally different way. They have been Bases and Coordinates written in a terse style, so you should read very slowly and with patience. Please Linear Maps and . feel free to email me with any questions or comments. The notes are in electronic Change of Basis form so sections can be changed very easily to incorporate improvements.
    [Show full text]
  • Using Functional Analysis and Sobolev Spaces to Solve Poisson’S Equation
    USING FUNCTIONAL ANALYSIS AND SOBOLEV SPACES TO SOLVE POISSON'S EQUATION YI WANG Abstract. We study Banach and Hilbert spaces with an eye to- wards defining weak solutions to elliptic PDE. Using Lax-Milgram we prove that weak solutions to Poisson's equation exist under certain conditions. Contents 1. Introduction 1 2. Banach spaces 2 3. Weak topology, weak star topology and reflexivity 6 4. Lower semicontinuity 11 5. Hilbert spaces 13 6. Sobolev spaces 19 References 21 1. Introduction We will discuss the following problem in this paper: let Ω be an open and connected subset in R and f be an L2 function on Ω, is there a solution to Poisson's equation (1) −∆u = f? From elementary partial differential equations class, we know if Ω = R, we can solve Poisson's equation using the fundamental solution to Laplace's equation. However, if we just take Ω to be an open and connected set, the above method is no longer useful. In addition, for arbitrary Ω and f, a C2 solution does not always exist. Therefore, instead of finding a strong solution, i.e., a C2 function which satisfies (1), we integrate (1) against a test function φ (a test function is a Date: September 28, 2016. 1 2 YI WANG smooth function compactly supported in Ω), integrate by parts, and arrive at the equation Z Z 1 (2) rurφ = fφ, 8φ 2 Cc (Ω): Ω Ω So intuitively we want to find a function which satisfies (2) for all test functions and this is the place where Hilbert spaces come into play.
    [Show full text]
  • Chapter IX. Tensors and Multilinear Forms
    Notes c F.P. Greenleaf and S. Marques 2006-2016 LAII-s16-quadforms.tex version 4/25/2016 Chapter IX. Tensors and Multilinear Forms. IX.1. Basic Definitions and Examples. 1.1. Definition. A bilinear form is a map B : V V C that is linear in each entry when the other entry is held fixed, so that × → B(αx, y) = αB(x, y)= B(x, αy) B(x + x ,y) = B(x ,y)+ B(x ,y) for all α F, x V, y V 1 2 1 2 ∈ k ∈ k ∈ B(x, y1 + y2) = B(x, y1)+ B(x, y2) (This of course forces B(x, y)=0 if either input is zero.) We say B is symmetric if B(x, y)= B(y, x), for all x, y and antisymmetric if B(x, y)= B(y, x). Similarly a multilinear form (aka a k-linear form , or a tensor− of rank k) is a map B : V V F that is linear in each entry when the other entries are held fixed. ×···×(0,k) → We write V = V ∗ . V ∗ for the set of k-linear forms. The reason we use V ∗ here rather than V , and⊗ the⊗ rationale for the “tensor product” notation, will gradually become clear. The set V ∗ V ∗ of bilinear forms on V becomes a vector space over F if we define ⊗ 1. Zero element: B(x, y) = 0 for all x, y V ; ∈ 2. Scalar multiple: (αB)(x, y)= αB(x, y), for α F and x, y V ; ∈ ∈ 3. Addition: (B + B )(x, y)= B (x, y)+ B (x, y), for x, y V .
    [Show full text]
  • MATH 421/510 Assignment 3
    MATH 421/510 Assignment 3 Suggested Solutions February 2018 1. Let H be a Hilbert space. (a) Prove the polarization identity: 1 hx; yi = (kx + yk2 − kx − yk2 + ikx + iyk2 − ikx − iyk2): 4 Proof. By direct calculation, kx + yk2 − kx − yk2 = 2hx; yi + 2hy; xi: Similarly, kx + iyk2 − kx − iyk2 = 2hx; iyi − 2hy; ixi = −2ihx; yi + 2ihy; xi: Addition gives kx + yk2−kx − yk2+ikx + iyk2−ikx − iyk2 = 2hx; yi+2hy; xi+2hx; yi−2hy; xi = 4hx; yi: (b) If there is another Hilbert space H0, a linear map from H to H0 is unitary if and only if it is isometric and surjective. Proof. We take the definition from the book that an operator is unitary if and only if it is invertible and hT x; T yi = hx; yi for all x; y 2 H. A unitary operator T is isometric and surjective by definition. On the other hand, assume it is isometric and surjective. We first show that T preserves the inner product: If the scalar field is C, by the polarization identity, 1 hT x; T yi = (kT x + T yk2 − kT x − T yk2 + ikT x + iT yk2 − ikT x − iT yk2) 4 1 = (kx + yk2 − kx − yk2 + ikx + iyk2 − ikx − iyk2) = hx; yi; 4 where in the second equation we used the linearity of T and the assumption that T is isometric. 1 If the scalar field is R, then we use the real version of the polarization identity: 1 hx; yi = (kx + yk2 − kx − yk2) 4 The remaining computation is similar to the complex case.
    [Show full text]
  • A Bit About Hilbert Spaces
    A Bit About Hilbert Spaces David Rosenberg New York University October 29, 2016 David Rosenberg (New York University ) DS-GA 1003 October 29, 2016 1 / 9 Inner Product Space (or “Pre-Hilbert” Spaces) An inner product space (over reals) is a vector space V and an inner product, which is a mapping h·,·i : V × V ! R that has the following properties 8x,y,z 2 V and a,b 2 R: Symmetry: hx,yi = hy,xi Linearity: hax + by,zi = ahx,zi + b hy,zi Postive-definiteness: hx,xi > 0 and hx,xi = 0 () x = 0. David Rosenberg (New York University ) DS-GA 1003 October 29, 2016 2 / 9 Norm from Inner Product For an inner product space, we define a norm as p kxk = hx,xi. Example Rd with standard Euclidean inner product is an inner product space: hx,yi := xT y 8x,y 2 Rd . Norm is p kxk = xT y. David Rosenberg (New York University ) DS-GA 1003 October 29, 2016 3 / 9 What norms can we get from an inner product? Theorem (Parallelogram Law) A norm kvk can be generated by an inner product on V iff 8x,y 2 V 2kxk2 + 2kyk2 = kx + yk2 + kx - yk2, and if it can, the inner product is given by the polarization identity jjxjj2 + jjyjj2 - jjx - yjj2 hx,yi = . 2 Example d `1 norm on R is NOT generated by an inner product. [Exercise] d Is `2 norm on R generated by an inner product? David Rosenberg (New York University ) DS-GA 1003 October 29, 2016 4 / 9 Pythagorean Theroem Definition Two vectors are orthogonal if hx,yi = 0.
    [Show full text]
  • Hilbert Space Methods for Partial Differential Equations
    Hilbert Space Methods for Partial Differential Equations R. E. Showalter Electronic Journal of Differential Equations Monograph 01, 1994. i Preface This book is an outgrowth of a course which we have given almost pe- riodically over the last eight years. It is addressed to beginning graduate students of mathematics, engineering, and the physical sciences. Thus, we have attempted to present it while presupposing a minimal background: the reader is assumed to have some prior acquaintance with the concepts of “lin- ear” and “continuous” and also to believe L2 is complete. An undergraduate mathematics training through Lebesgue integration is an ideal background but we dare not assume it without turning away many of our best students. The formal prerequisite consists of a good advanced calculus course and a motivation to study partial differential equations. A problem is called well-posed if for each set of data there exists exactly one solution and this dependence of the solution on the data is continuous. To make this precise we must indicate the space from which the solution is obtained, the space from which the data may come, and the correspond- ing notion of continuity. Our goal in this book is to show that various types of problems are well-posed. These include boundary value problems for (stationary) elliptic partial differential equations and initial-boundary value problems for (time-dependent) equations of parabolic, hyperbolic, and pseudo-parabolic types. Also, we consider some nonlinear elliptic boundary value problems, variational or uni-lateral problems, and some methods of numerical approximation of solutions. We briefly describe the contents of the various chapters.
    [Show full text]
  • 3. Hilbert Spaces
    3. Hilbert spaces In this section we examine a special type of Banach spaces. We start with some algebraic preliminaries. Definition. Let K be either R or C, and let Let X and Y be vector spaces over K. A map φ : X × Y → K is said to be K-sesquilinear, if • for every x ∈ X, then map φx : Y 3 y 7−→ φ(x, y) ∈ K is linear; • for every y ∈ Y, then map φy : X 3 y 7−→ φ(x, y) ∈ K is conjugate linear, i.e. the map X 3 x 7−→ φy(x) ∈ K is linear. In the case K = R the above properties are equivalent to the fact that φ is bilinear. Remark 3.1. Let X be a vector space over C, and let φ : X × X → C be a C-sesquilinear map. Then φ is completely determined by the map Qφ : X 3 x 7−→ φ(x, x) ∈ C. This can be see by computing, for k ∈ {0, 1, 2, 3} the quantity k k k k k k k Qφ(x + i y) = φ(x + i y, x + i y) = φ(x, x) + φ(x, i y) + φ(i y, x) + φ(i y, i y) = = φ(x, x) + ikφ(x, y) + i−kφ(y, x) + φ(y, y), which then gives 3 1 X (1) φ(x, y) = i−kQ (x + iky), ∀ x, y ∈ X. 4 φ k=0 The map Qφ : X → C is called the quadratic form determined by φ. The identity (1) is referred to as the Polarization Identity.
    [Show full text]
  • Lesson: Bilinear, Quadratic and Hermitian Forms Lesson Developer: Vivek N Sharma College / Department: Department of Mathematics, S.G.T.B
    Bilinear, Quadratic and Hermitian Forms Lesson: Bilinear, Quadratic and Hermitian Forms Lesson Developer: Vivek N Sharma College / Department: Department of Mathematics, S.G.T.B. Khalsa College, University of Delhi Institute of Lifelong Learning, University of Delhi pg. 1 Bilinear, Quadratic and Hermitian Forms Table of Contents 1. Learning Outcomes 2. Introduction 3. Definition of a Bilinear Form on a Vector Space 4. Various Types of Bilinear Forms 5. Matrix Representation of a Bilinear Form on a Vector Space 6. Quadratic Forms on ℝ푛 7. Hermitian Forms on a Vector Space 8. Summary 9. Exercises 10. Glossary and Further Reading 11. Solutions/Hints for Exercises 1. Learning Outcomes: After studying this unit, you will be able to define the concept of a bilinear form on a vector space. explain the equivalence of bilinear forms with matrices. represent a bilinear form on a vector space as a square matrix. define the concept of a quadratic form on ℝ푛 . explain the notion of a Hermitian form on a vector space. Institute of Lifelong Learning, University of Delhi pg. 2 Bilinear, Quadratic and Hermitian Forms 2. Introduction: Bilinear forms occupy a unique place in all of mathematics. The study of linear transformations alone is incapable of handling the notions of orthogonality in geometry, optimization in many variables, Fourier series and so on and so forth. In optimization theory, the relevance of quadratic forms is all the more. The concept of dot product is a particular instance of a bilinear form. Quadratic forms, in particular, play an all important role in deciding the maxima-minima of functions of several variables.
    [Show full text]
  • Notes for Functional Analysis
    Notes for Functional Analysis Wang Zuoqin (typed by Xiyu Zhai) November 13, 2015 1 Lecture 18 1.1 Characterizations of reflexive spaces Recall that a Banach space X is reflexive if the inclusion X ⊂ X∗∗ is a Banach space isomorphism. The following theorem of Kakatani give us a very useful creteria of reflexive spaces: Theorem 1.1. A Banach space X is reflexive iff the closed unit ball B = fx 2 X : kxk ≤ 1g is weakly compact. Proof. First assume X is reflexive. Then B is the closed unit ball in X∗∗. By the Banach- Alaoglu theorem, B is compact with respect to the weak-∗ topology of X∗∗. But by definition, in this case the weak-∗ topology on X∗∗ coincides with the weak topology on X (since both are the weakest topology on X = X∗∗ making elements in X∗ continuous). So B is compact with respect to the weak topology on X. Conversly suppose B is weakly compact. Let ι : X,! X∗∗ be the canonical inclusion map that sends x to evx. Then we've seen that ι preserves the norm, and thus is continuous (with respect to the norm topologies). By PSet 8-1 Problem 3, ∗∗ ι : X(weak) ,! X(weak) is continuous. Since the weak-∗ topology on X∗∗ is weaker than the weak topology on X∗∗ (since X∗∗∗ = (X∗)∗∗ ⊃ X∗. OH MY GOD!) we thus conclude that ∗∗ ι : X(weak) ,! X(weak-∗) is continuous. But by assumption B ⊂ X(weak) is compact, so the image ι(B) is compact in ∗∗ X(weak-∗). In particular, ι(B) is weak-∗ closed.
    [Show full text]
  • The Riesz Representation Theorem
    The Riesz Representation Theorem MA 466 Kurt Bryan Let H be a Hilbert space over lR or Cl, and T a bounded linear functional on H (a bounded operator from H to the field, lR or Cl, over which H is defined). The following is called the Riesz Representation Theorem: Theorem 1 If T is a bounded linear functional on a Hilbert space H then there exists some g 2 H such that for every f 2 H we have T (f) =< f; g > : Moreover, kT k = kgk (here kT k denotes the operator norm of T , while kgk is the Hilbert space norm of g. Proof: Let's assume that H is separable for now. It's not really much harder to prove for any Hilbert space, but the separable case makes nice use of the ideas we developed regarding Fourier analysis. Also, let's just work over lR. Since H is separable we can choose an orthonormal basis ϕj, j ≥ 1, for 2 H. Let T be a bounded linear functionalP and set aj = T (ϕj). Choose f H, n let cj =< f; ϕj >, and define fn = j=1 cjϕj. Since the ϕj form a basis we know that kf − fnk ! 0 as n ! 1. Since T is linear we have Xn T (fn) = ajcj: (1) j=1 Since T is bounded, say with norm kT k < 1, we have jT (f) − T (fn)j ≤ kT kkf − fnk: (2) Because kf − fnk ! 0 as n ! 1 we conclude from equations (1) and (2) that X1 T (f) = lim T (fn) = ajcj: (3) n!1 j=1 In fact, the sequence aj must itself be square-summable.
    [Show full text]
  • Numerical Linear Algebra
    University of Alabama at Birmingham Department of Mathematics Numerical Linear Algebra Lecture Notes for MA 660 (1997{2014) Dr Nikolai Chernov Summer 2014 Contents 0. Review of Linear Algebra 1 0.1 Matrices and vectors . 1 0.2 Product of a matrix and a vector . 1 0.3 Matrix as a linear transformation . 2 0.4 Range and rank of a matrix . 2 0.5 Kernel (nullspace) of a matrix . 2 0.6 Surjective/injective/bijective transformations . 2 0.7 Square matrices and inverse of a matrix . 3 0.8 Upper and lower triangular matrices . 3 0.9 Eigenvalues and eigenvectors . 4 0.10 Eigenvalues of real matrices . 4 0.11 Diagonalizable matrices . 5 0.12 Trace . 5 0.13 Transpose of a matrix . 6 0.14 Conjugate transpose of a matrix . 6 0.15 Convention on the use of conjugate transpose . 6 1. Norms and Inner Products 7 1.1 Norms . 7 1.2 1-norm, 2-norm, and 1-norm . 7 1.3 Unit vectors, normalization . 8 1.4 Unit sphere, unit ball . 8 1.5 Images of unit spheres . 8 1.6 Frobenius norm . 8 1.7 Induced matrix norms . 9 1.8 Computation of kAk1, kAk2, and kAk1 .................... 9 1.9 Inequalities for induced matrix norms . 9 1.10 Inner products . 10 1.11 Standard inner product . 10 1.12 Cauchy-Schwarz inequality . 11 1.13 Induced norms . 12 1.14 Polarization identity . 12 1.15 Orthogonal vectors . 12 1.16 Pythagorean theorem . 12 1.17 Orthogonality and linear independence . 12 1.18 Orthonormal sets of vectors .
    [Show full text]