DOCSLIB.ORG

1 Inner Products and Norms

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
1 Inner Products and Norms ORF 523 Lecture 2 Spring 2016, Princeton University Instructor: A.A. Ahmadi Scribe: G. Hall Tuesday, February 9, 2016 When in doubt on the accuracy of these notes, please cross check with the instructor's notes, on aaa. princeton. edu/ orf523 . Any typos should be emailed to [email protected]. Today, we review basic math concepts that you will need throughout the course. • Inner products and norms • Positive semidefinite matrices • Basic differential calculus 1 Inner products and norms 1.1 Inner products 1.1.1 Definition Definition 1 (Inner product). A function h:; :i : Rn × Rn ! R is an inner product if 1. hx; xi ≥ 0; hx; xi = 0 , x = 0 (positivity) 2. hx; yi = hy; xi (symmetry) 3. hx + y; zi = hx; zi + hy; zi (additivity) 4. hrx; yi = rhx; yi for all r 2 R (homogeneity) Homogeneity in the second argument follows: hx; ryi = hry; xi = rhy; xi = rhx; yi using properties (2) and (4) and again (2) respectively, and hx; y + zi = hy + z; xi = hy; xi + hz; xi = hx; yi + hx; zi using properties (2), (3) and again (2). 1 1.1.2 Examples • The standard inner product is T X n hx; yi = x y = xiyi; x; y 2 R : • The standard inner product between matrices is T X X hX; Y i = Tr(X Y ) = XijYij i j where X; Y 2 Rm×n. Notation: Here, Rm×n is the space of real m × n matrices. Tr(Z) is the trace of a real square P matrix Z, i.e., Tr(Z) = i Zii. Note: The matrix inner product is the same as our original inner product between two vectors of length mn obtained by stacking the columns of the two matrices. • A less classical example in R2 is the following: hx; yi = 5x1y1 + 8x2y2 − 6x1y2 − 6x2y1 Properties (2), (3) and (4) are obvious, positivity is less obvious. It can be seen by writing 2 2 2 2 hx; xi = 5x1 + 8x2 − 12x1x2 = (x1 − 2x2) + (2x1 − 2x2) ≥ 0 hx; xi = 0 , x1 − 2x2 = 0 and 2x1 − 2x2 = 0 , x1 = 0 and x2 = 0: 1.1.3 Properties of inner products Definition 2 (Orthogonality). We say that x and y are orthogonal if hx; yi = 0: Theorem 1 (Cauchy Schwarz). For x; y 2 Rn jhx; yij ≤ jjxjj jjyjj; where jjxjj := phx; xi is the length of x (it is also a norm as we will show later on). 2 Proof: First, assume that jjxjj = jjyjj = 1. jjx − yjj2 ≥ 0 ) hx − y; x − yi = hx; xi + hy; yi − 2hx; yi ≥ 0 ) hx; yi ≤ 1: Now, consider any x; y 2 Rn. If one of the vectors is zero, the inequality is trivially verified. If they are both nonzero, then: x y ; ≤ 1 ) hx; yi ≤ jjxjj · jjyjj: (1) jjxjj jjyjj Since (1) holds 8x; y, replace y with −y: hx; −yi ≤ jjxjj · jj − yjj hx; −yi ≥ −||xjj · jjyjj using properties (1) and (2) respectively. 1.2 Norms 1.2.1 Definition Definition 3 (Norm). A function f : Rn ! R is a norm if 1. f(x) ≥ 0, f(x) = 0 , x = 0 (positivity) 2. f(αx) = jαjf(x); 8α 2 R (homogeneity) 3. f(x + y) ≤ f(x) + f(y) (triangle inequality) Examples: pP 2 • The 2-norm: jjxjj = i xi P • The 1-norm: jjxjj1 = i jxij • The inf-norm: jjxjj1 = maxi jxij P p 1=p • The p-norm: jjxjjp = ( i jxij ) , p ≥ 1 Lemma 1. Take any inner product h:; :i and define f(x) = phx; xi. Then f is a norm. 3 Proof: Positivity follows from the definition. For homogeneity, f(αx) = phαx; αxi = jαjphx; xi We prove triangular inequality by contradiction. If it is not satisfied, then 9x; y s.t. phx + y; x + yi > phx; xi + phy; yi ) hx + y; x + yi > hx; xi + 2phx; xihy; yi + hy; yi ) 2hx; yi > 2phx; xihy; yi which contradicts Cauchy-Schwarz. Note: Not every norm comes from an inner product. 1.2.2 Matrix norms Matrix norms are functions f : Rm×n ! R that satisfy the same properties as vector norms. Let A 2 Rm×n. Here are a few examples of matrix norms: p T qP 2 • The Frobenius norm: jjAjjF = Tr(A A) = i;j Ai;j P • The sum-absolute-value norm: jjAjjsav = i;j jXi;jj • The max-absolute-value norm: jjAjjmav = maxi;j jAi;jj Definition 4 (Operator norm). An operator (or induced) matrix norm is a norm m×n jj:jja;b : R ! R defined as jjAjja;b = max jjAxjja x s.t. jjxjjb ≤ 1; m n where jj:jja is a vector norm on R and jj:jjb is a vector norm on R . Notation: When the same vector norm is used in both spaces, we write jjAjjc = max jjAxjjc s.t. jjxjjc ≤ 1: Examples: 4 p T • jjAjj2 = λmax(A A), where λmax denotes the largest eigenvalue. P • jjAjj1 = maxj i jAijj, i.e., the maximum column sum. P • jjAjj1 = maxi j jAijj, i.e., the maximum row sum. Notice that not all matrix norms are induced norms. An example is the Frobenius norm p given above as jjIjj∗ = 1 for any induced norm, but jjIjjF = n. Lemma 2. Every induced norm is submultiplicative, i.e., jjABjj ≤ jjAjj jjBjj: Proof: We first show that jjAxjj ≤ jjAjj jjxjj. Suppose that this is not the case, then jjAxjj > jjAjj jxjj 1 ) jjAxjj > jjAjj jjxjj x ) A > jjAjj jjxjj x but jjxjj is a vector of unit norm. This contradicts the definition of jjAjj. Now we proceed to prove the claim. jjABjj = max jjABxjj ≤ max jjAjj jjBxjj = jjAjj max jjBxjj = jjAjj jjBjj: jjx||≤1 jjx||≤1 jjx||≤1 Remark: This is only true for induced norms that use the same vector norm in both spaces. In the case where the vector norms are different, submultiplicativity can fail to hold. Consider e.g., the induced norm jj · jj1;2; and the matrices " p p # " # 2=2 2=2 1 0 A = p p and B = : − 2=2 2=2 1 0 In this case, jjABjj1;2 > jjAjj1;2 · jjBjj1;2: Indeed, the image of the unit circle by A (notice that A is a rotation matrix of angle π=4) stays within the unit square, and so jjAjj1;2 ≤ 1. Using similar reasoning, jjBjj1;2 ≤ 1. 5 p p This implies that jjAjj1;2jjBjj1;2 ≤ 1. However, jjABjj1;2 ≥ 2, as jjABxjj1 = 2 for x = (1; 0)T . Example of a norm that is not submultiplicative: jjAjjmav = max jAi;jj i;j This can be seen as any submultiplicative norm satisfies jjA2jj ≤ jjAjj2: In this case, ! ! 1 1 2 2 A = and A2 = 1 1 2 2 2 2 So jjA jjmav = 2 > 1 = jjAjjmav. Remark: Not all submultiplicative norms are induced norms. An example is the Frobenius norm. 1.2.3 Dual norms Definition 5 (Dual norm). Let jj:jj be any norm. Its dual norm is defined as T jjxjj∗ = max x y s.t. jjyjj ≤ 1: You can think of this as the operator norm of xT . The dual norm is indeed a norm. The first two properties are straightforward to prove. The triangle inequality can be shown in the following way: T T T T jjx + zjj∗ = max(x y + z y) ≤ max x y + max z y = jjxjj∗ + jjzjj∗ jjy||≤1 jjy||≤1 jjy||≤1 Examples: 1. jjxjj1∗ = jjxjj1 6 2. jjxjj2∗ = jjxjj2 3. jjxjj∞∗ = jjxjj1. Proofs: • The proof of (1) is left as an exercise. • Proof of (2): We have T jjxjj2∗ = max x y y s.t. jjyjj2 ≤ 1: Cauchy-Schwarz implies that xT y ≤ jjxjj jjyjj ≤ jjxjj x and y = jjxjj achieves this bound. • Proof of (3): We have T jjxjj∞∗ = max x y y s.t. jjyjj1 ≤ 1 So yopt = sign(x) and the optimal value is jjxjj1. 2 Positive semidefinite matrices We denote by Sn×n the set of all symmetric (real) n × n matrices. 2.1 Definition Definition 6. A matrix A 2 Sn×n is • positive semidefinite (psd) (notation: A 0) if T n x Ax ≥ 0; 8x 2 R : • positive definite (pd) (notation: A 0) if T n x Ax > 0; 8x 2 R ; x 6= 0: 7 • negative semidefinite if −A is psd. (Notation: A 0) • negative definite if −A is pd. (Notation: A ≺ 0.) Notation: A 0 means A is psd; A ≥ 0 means that Aij ≥ 0, for all i; j. Remark: Whenever we consider a quadratic form xT Ax, we can assume without loss of generality that the matrix A is symmetric. The reason behind this is that any matrix A can be written as A + AT A − AT A = + 2 2 A+AT A−AT where B := 2 is the symmetric part of A and C := 2 is the anti-symmetric part of A. Notice that xT Cx = 0 for any x 2 Rn. Example: The matrix ! 5 1 M = 1 −2 is indefinite. To see this, consider x = (1; 0)T and x = (0; 1)T : 2.2 Eigenvalues of positive semidefinite matrices Theorem 2. The eigenvalues of a symmetric real-valued matrix A are real. Proof: Let x 2 Cn be a nonzero eigenvector of A and let λ 2 C be the corresponding eigenvalue; i.e., Ax = λx. By multiplying either side of the equality by the conjugate transpose x∗ of eigenvector x, we obtain x∗Ax = λx∗x; (2) We now take the conjugate of both sides, remembering that A 2 Sn×n : x∗AT x = λx¯ ∗x ) x∗Ax = λx¯ ∗x (3) Combining (2) and (3), we get λx∗x = λx¯ ∗x ) x∗x(λ − λ¯) = 0 ) λ = λ,¯ since x 6= 0.
Load more

Recommended publications
  • C H a P T E R § Methods Related to the Norm Al Equations
    ¡ ¢ £ ¤ ¥ ¦ § ¨ © © © © ¨ ©! "# $% &('*),+-)(./+-)0.21434576/),+98,:%;<)>=,'41/? @%3*)BAC:D8E+%=B891GF*),+H;I?H1KJ2.21K8L1GMNAPO45Q5R)S;S+T? =RU ?H1*)>./+EAVOKAPM ;<),5 ?H1G;P8W.XAVO4575R)S;S+Y? =Z8L1*)/[%\Z1*)]AB3*=,'Q;<)B=,'41/? @%3*)RAP8LU F*)BA(;S'*)Z)>@%3/? F*.4U ),1G;^U ?H1*)>./+ APOKAP;_),5a`Rbc`^dfeg`Rbih,jk=>.4U U )Blm;B'*)n1K84+o5R.EUp)B@%3*.K;q? 8L1KA>[r\0:D;_),1Ejp;B'/? A2.4s4s/+t8/.,=,' b ? A7.KFK84? l4)>lu?H1vs,+D.*=S;q? =>)m6/)>=>.43KA<)W;B'*)w=B8E)IxW=K? ),1G;W5R.K;S+Y? yn` `z? AW5Q3*=,'|{}84+-A_) =B8L1*l9? ;I? 891*)>lX;B'*.41Q`7[r~k8K{i)IF*),+NjL;B'*)71K8E+o5R.4U4)>@%3*.G;I? 891KA0.4s4s/+t8.*=,'w5R.BOw6/)Z.,l4)IM @%3*.K;_)7?H17A_8L5R)QAI? ;S3*.K;q? 8L1KA>[p1*l4)>)>lj9;B'*),+-)Q./+-)Q)SF*),1.Es4s4U ? =>.K;q? 8L1KA(?H1Q{R'/? =,'W? ;R? A s/+-)S:Y),+o+D)Blm;_8|;B'*)W3KAB3*.4Urc+HO4U 8,F|AS346KABs*.,=B)Q;_)>=,'41/? @%3*)BAB[&('/? A]=*'*.EsG;_),+}=B8*F*),+tA7? ;PM ),+-.K;q? F*)75R)S;S'K8ElAi{^'/? =,'Z./+-)R)K? ;B'*),+pl9?+D)B=S;BU OR84+k?H5Qs4U ? =K? ;BU O2+-),U .G;<)Bl7;_82;S'*)Q1K84+o5R.EU )>@%3*.G;I? 891KA>[ mWX(fQ - uK In order to solve the linear system 7 when is nonsymmetric, we can solve the equivalent system b b [¥K¦ ¡¢ £N¤ which is Symmetric Positive Definite. This system is known as the system of the normal equations associated with the least-squares problem, [ ­¦ £N¤ minimize §* R¨©7ª§*«9¬ Note that (8.1) is typically used to solve the least-squares problem (8.2) for over- ®°¯± ±²³® determined systems, i.e., when is a rectangular matrix of size , .
    [Show full text]
  • Surjective Isometries on Grassmann Spaces 11
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Repository of the Academy's Library SURJECTIVE ISOMETRIES ON GRASSMANN SPACES FERNANDA BOTELHO, JAMES JAMISON, AND LAJOS MOLNAR´ Abstract. Let H be a complex Hilbert space, n a given positive integer and let Pn(H) be the set of all projections on H with rank n. Under the condition dim H ≥ 4n, we describe the surjective isometries of Pn(H) with respect to the gap metric (the metric induced by the operator norm). 1. Introduction The study of isometries of function spaces or operator algebras is an important research area in functional analysis with history dating back to the 1930's. The most classical results in the area are the Banach-Stone theorem describing the surjective linear isometries between Banach spaces of complex- valued continuous functions on compact Hausdorff spaces and its noncommutative extension for general C∗-algebras due to Kadison. This has the surprising and remarkable consequence that the surjective linear isometries between C∗-algebras are closely related to algebra isomorphisms. More precisely, every such surjective linear isometry is a Jordan *-isomorphism followed by multiplication by a fixed unitary element. We point out that in the aforementioned fundamental results the underlying structures are algebras and the isometries are assumed to be linear. However, descriptions of isometries of non-linear structures also play important roles in several areas. We only mention one example which is closely related with the subject matter of this paper. This is the famous Wigner's theorem that describes the structure of quantum mechanical symmetry transformations and plays a fundamental role in the probabilistic aspects of quantum theory.
    [Show full text]
  • The Nonstandard Theory of Topological Vector Spaces
    TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY Volume 172, October 1972 THE NONSTANDARDTHEORY OF TOPOLOGICAL VECTOR SPACES BY C. WARD HENSON AND L. C. MOORE, JR. ABSTRACT. In this paper the nonstandard theory of topological vector spaces is developed, with three main objectives: (1) creation of the basic nonstandard concepts and tools; (2) use of these tools to give nonstandard treatments of some major standard theorems ; (3) construction of the nonstandard hull of an arbitrary topological vector space, and the beginning of the study of the class of spaces which tesults. Introduction. Let Ml be a set theoretical structure and let *JR be an enlarge- ment of M. Let (E, 0) be a topological vector space in M. §§1 and 2 of this paper are devoted to the elementary nonstandard theory of (F, 0). In particular, in §1 the concept of 0-finiteness for elements of *E is introduced and the nonstandard hull of (E, 0) (relative to *3R) is defined. §2 introduces the concept of 0-bounded- ness for elements of *E. In §5 the elementary nonstandard theory of locally convex spaces is developed by investigating the mapping in *JK which corresponds to a given pairing. In §§6 and 7 we make use of this theory by providing nonstandard treatments of two aspects of the existing standard theory. In §6, Luxemburg's characterization of the pre-nearstandard elements of *E for a normed space (E, p) is extended to Hausdorff locally convex spaces (E, 8). This characterization is used to prove the theorem of Grothendieck which gives a criterion for the completeness of a Hausdorff locally convex space.
    [Show full text]
  • The Column and Row Hilbert Operator Spaces
    The Column and Row Hilbert Operator Spaces Roy M. Araiza Department of Mathematics Purdue University Abstract Given a Hilbert space H we present the construction and some properties of the column and row Hilbert operator spaces Hc and Hr, respectively. The main proof of the survey is showing that CB(Hc; Kc) is completely isometric to B(H ; K ); in other words, the completely bounded maps be- tween the corresponding column Hilbert operator spaces is completely isometric to the bounded operators between the Hilbert spaces. A similar theorem for the row Hilbert operator spaces follows via duality of the operator spaces. In particular there exists a natural duality between the column and row Hilbert operator spaces which we also show. The presentation and proofs are due to Effros and Ruan [1]. 1 The Column Hilbert Operator Space Given a Hilbert space H , we define the column isometry C : H −! B(C; H ) given by ξ 7! C(ξ)α := αξ; α 2 C: Then given any n 2 N; we define the nth-matrix norm on Mn(H ); by (n) kξkc;n = C (ξ) ; ξ 2 Mn(H ): These are indeed operator space matrix norms by Ruan's Representation theorem and thus we define the column Hilbert operator space as n o Hc = H ; k·kc;n : n2N It follows that given ξ 2 Mn(H ) we have that the adjoint operator of C(ξ) is given by ∗ C(ξ) : H −! C; ζ 7! (ζj ξ) : Suppose C(ξ)∗(ζ) = d; c 2 C: Then we see that cd = (cj C(ξ)∗(ζ)) = (cξj ζ) = c(ζj ξ) = (cj (ζj ξ)) : We are also able to compute the matrix norms on rectangular matrices.
    [Show full text]
  • 1 Lecture 09
    Notes for Functional Analysis Wang Zuoqin (typed by Xiyu Zhai) September 29, 2015 1 Lecture 09 1.1 Equicontinuity First let's recall the conception of \equicontinuity" for family of functions that we learned in classical analysis: A family of continuous functions defined on a region Ω, Λ = ffαg ⊂ C(Ω); is called an equicontinuous family if 8 > 0; 9δ > 0 such that for any x1; x2 2 Ω with jx1 − x2j < δ; and for any fα 2 Λ, we have jfα(x1) − fα(x2)j < . This conception of equicontinuity can be easily generalized to maps between topological vector spaces. For simplicity we only consider linear maps between topological vector space , in which case the continuity (and in fact the uniform continuity) at an arbitrary point is reduced to the continuity at 0. Definition 1.1. Let X; Y be topological vector spaces, and Λ be a family of continuous linear operators. We say Λ is an equicontinuous family if for any neighborhood V of 0 in Y , there is a neighborhood U of 0 in X, such that L(U) ⊂ V for all L 2 Λ. Remark 1.2. Equivalently, this means if x1; x2 2 X and x1 − x2 2 U, then L(x1) − L(x2) 2 V for all L 2 Λ. Remark 1.3. If Λ = fLg contains only one continuous linear operator, then it is always equicontinuous. 1 If Λ is an equicontinuous family, then each L 2 Λ is continuous and thus bounded. In fact, this boundedness is uniform: Proposition 1.4. Let X,Y be topological vector spaces, and Λ be an equicontinuous family of linear operators from X to Y .
    [Show full text]
  • Matrices and Linear Algebra
    Chapter 26 Matrices and linear algebra ap,mat Contents 26.1 Matrix algebra........................................... 26.2 26.1.1 Determinant (s,mat,det)................................... 26.2 26.1.2 Eigenvalues and eigenvectors (s,mat,eig).......................... 26.2 26.1.3 Hermitian and symmetric matrices............................. 26.3 26.1.4 Matrix trace (s,mat,trace).................................. 26.3 26.1.5 Inversion formulas (s,mat,mil)............................... 26.3 26.1.6 Kronecker products (s,mat,kron).............................. 26.4 26.2 Positive-definite matrices (s,mat,pd) ................................ 26.4 26.2.1 Properties of positive-definite matrices........................... 26.5 26.2.2 Partial order......................................... 26.5 26.2.3 Diagonal dominance.................................... 26.5 26.2.4 Diagonal majorizers..................................... 26.5 26.2.5 Simultaneous diagonalization................................ 26.6 26.3 Vector norms (s,mat,vnorm) .................................... 26.6 26.3.1 Examples of vector norms.................................. 26.6 26.3.2 Inequalities......................................... 26.7 26.3.3 Properties.......................................... 26.7 26.4 Inner products (s,mat,inprod) ................................... 26.7 26.4.1 Examples.......................................... 26.7 26.4.2 Properties.......................................... 26.8 26.5 Matrix norms (s,mat,mnorm) .................................... 26.8 26.5.1
    [Show full text]
  • Lecture 5 Ch. 5, Norms for Vectors and Matrices
    KTH ROYAL INSTITUTE OF TECHNOLOGY Norms for vectors and matrices — Why? Lecture 5 Problem: Measure size of vector or matrix. What is “small” and what is “large”? Ch. 5, Norms for vectors and matrices Problem: Measure distance between vectors or matrices. When are they “close together” or “far apart”? Emil Björnson/Magnus Jansson/Mats Bengtsson April 27, 2016 Answers are given by norms. Also: Tool to analyze convergence and stability of algorithms. 2 / 21 Vector norm — axiomatic definition Inner product — axiomatic definition Definition: LetV be a vector space over afieldF(R orC). Definition: LetV be a vector space over afieldF(R orC). A function :V R is a vector norm if for all A function , :V V F is an inner product if for all || · || → �· ·� × → x,y V x,y,z V, ∈ ∈ (1) x 0 nonnegative (1) x,x 0 nonnegative || ||≥ � �≥ (1a) x =0iffx= 0 positive (1a) x,x =0iffx=0 positive || || � � (2) cx = c x for allc F homogeneous (2) x+y,z = x,z + y,z additive || || | | || || ∈ � � � � � � (3) x+y x + y triangle inequality (3) cx,y =c x,y for allc F homogeneous || ||≤ || || || || � � � � ∈ (4) x,y = y,x Hermitian property A function not satisfying (1a) is called a vector � � � � seminorm. Interpretation: “Angle” (distance) between vectors. Interpretation: Size/length of vector. 3 / 21 4 / 21 Connections between norm and inner products Examples / Corollary: If , is an inner product, then x =( x,x ) 1 2 � The Euclidean norm (l ) onC n: �· ·� || || � � 2 is a vector norm. 2 2 2 1/2 x 2 = ( x1 + x 2 + + x n ) . Called: Vector norm derived from an inner product.
    [Show full text]
  • Matrix Norms 30.4   
    Matrix Norms 30.4 Introduction A matrix norm is a number defined in terms of the entries of the matrix. The norm is a useful quantity which can give important information about a matrix. ' • be familiar with matrices and their use in $ writing systems of equations • revise material on matrix inverses, be able to find the inverse of a 2 × 2 matrix, and know Prerequisites when no inverse exists Before starting this Section you should ... • revise Gaussian elimination and partial pivoting • be aware of the discussion of ill-conditioned and well-conditioned problems earlier in Section 30.1 #& • calculate norms and condition numbers of % Learning Outcomes small matrices • adjust certain systems of equations with a On completion you should be able to ... view to better conditioning " ! 34 HELM (2008): Workbook 30: Introduction to Numerical Methods ® 1. Matrix norms The norm of a square matrix A is a non-negative real number denoted kAk. There are several different ways of defining a matrix norm, but they all share the following properties: 1. kAk ≥ 0 for any square matrix A. 2. kAk = 0 if and only if the matrix A = 0. 3. kkAk = |k| kAk, for any scalar k. 4. kA + Bk ≤ kAk + kBk. 5. kABk ≤ kAk kBk. The norm of a matrix is a measure of how large its elements are. It is a way of determining the “size” of a matrix that is not necessarily related to how many rows or columns the matrix has. Key Point 6 Matrix Norm The norm of a matrix is a real number which is a measure of the magnitude of the matrix.
    [Show full text]
  • The Banach-Alaoglu Theorem for Topological Vector Spaces
    The Banach-Alaoglu theorem for topological vector spaces Christiaan van den Brink a thesis submitted to the Department of Mathematics at Utrecht University in partial fulfillment of the requirements for the degree of Bachelor in Mathematics Supervisor: Fabian Ziltener date of submission 06-06-2019 Abstract In this thesis we generalize the Banach-Alaoglu theorem to topological vector spaces. the theorem then states that the polar, which lies in the dual space, of a neighbourhood around zero is weak* compact. We give motivation for the non-triviality of this theorem in this more general case. Later on, we show that the polar is sequentially compact if the space is separable. If our space is normed, then we show that the polar of the unit ball is the closed unit ball in the dual space. Finally, we introduce the notion of nets and we use these to prove the main theorem. i ii Acknowledgments A huge thanks goes out to my supervisor Fabian Ziltener for guiding me through the process of writing a bachelor thesis. I would also like to thank my girlfriend, family and my pet who have supported me all the way. iii iv Contents 1 Introduction 1 1.1 Motivation and main result . .1 1.2 Remarks and related works . .2 1.3 Organization of this thesis . .2 2 Introduction to Topological vector spaces 4 2.1 Topological vector spaces . .4 2.1.1 Definition of topological vector space . .4 2.1.2 The topology of a TVS . .6 2.2 Dual spaces . .9 2.2.1 Continuous functionals .
    [Show full text]
  • 5 Norms on Linear Spaces
    5 Norms on Linear Spaces 5.1 Introduction In this chapter we study norms on real or complex linear linear spaces. Definition 5.1. Let F = (F, {0, 1, +, −, ·}) be the real or the complex field. A semi-norm on an F -linear space V is a mapping ν : V −→ R that satisfies the following conditions: (i) ν(x + y) ≤ ν(x)+ ν(y) (the subadditivity property of semi-norms), and (ii) ν(ax)= |a|ν(x), for x, y ∈ V and a ∈ F . By taking a = 0 in the second condition of the definition we have ν(0)=0 for every semi-norm on a real or complex space. A semi-norm can be defined on every linear space. Indeed, if B is a basis of V , B = {vi | i ∈ I}, J is a finite subset of I, and x = i∈I xivi, define νJ (x) as P 0 if x = 0, νJ (x)= ( j∈J |aj | for x ∈ V . We leave to the readerP the verification of the fact that νJ is indeed a seminorm. Theorem 5.2. If V is a real or complex linear space and ν : V −→ R is a semi-norm on V , then ν(x − y) ≥ |ν(x) − ν(y)|, for x, y ∈ V . Proof. We have ν(x) ≤ ν(x − y)+ ν(y), so ν(x) − ν(y) ≤ ν(x − y). (5.1) 160 5 Norms on Linear Spaces Since ν(x − y)= |− 1|ν(y − x) ≥ ν(y) − ν(x) we have −(ν(x) − ν(y)) ≤ ν(x) − ν(y). (5.2) The Inequalities 5.1 and 5.2 give the desired inequality.
    [Show full text]
  • A Degeneracy Framework for Scalable Graph Autoencoders
    Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19) A Degeneracy Framework for Scalable Graph Autoencoders Guillaume Salha1;2 , Romain Hennequin1 , Viet Anh Tran1 and Michalis Vazirgiannis2 1Deezer Research & Development, Paris, France 2Ecole´ Polytechnique, Palaiseau, France [email protected] Abstract In this paper, we focus on the graph extensions of autoen- coders and variational autoencoders. Introduced in the 1980’s In this paper, we present a general framework to [Rumelhart et al., 1986], autoencoders (AE) regained a sig- scale graph autoencoders (AE) and graph varia- nificant popularity in the last decade through neural network tional autoencoders (VAE). This framework lever- frameworks [Baldi, 2012] as efficient tools to learn reduced ages graph degeneracy concepts to train models encoding representations of input data in an unsupervised only from a dense subset of nodes instead of us- way. Furthermore, variational autoencoders (VAE) [Kingma ing the entire graph. Together with a simple yet ef- and Welling, 2013], described as extensions of AE but actu- fective propagation mechanism, our approach sig- ally based on quite different mathematical foundations, also nificantly improves scalability and training speed recently emerged as a successful approach for unsupervised while preserving performance. We evaluate and learning from complex distributions, assuming the input data discuss our method on several variants of existing is the observed part of a larger joint model involving low- graph AE and VAE, providing the first application dimensional latent variables, optimized via variational infer- of these models to large graphs with up to millions ence approximations. [Tschannen et al., 2018] review the of nodes and edges.
    [Show full text]
  • BS II: Elementary Banach Space Theory
    BS II c Gabriel Nagy Banach Spaces II: Elementary Banach Space Theory Notes from the Functional Analysis Course (Fall 07 - Spring 08) In this section we introduce Banach spaces and examine some of their important features. Sub-section B collects the five so-called Principles of Banach space theory, which were already presented earlier. Convention. Throughout this note K will be one of the fields R or C, and all vector spaces are over K. A. Banach Spaces Definition. A normed vector space (X , k . k), which is complete in the norm topology, is called a Banach space. When there is no danger of confusion, the norm will be omitted from the notation. Comment. Banach spaces are particular cases of Frechet spaces, which were defined as locally convex (F)-spaces. Therefore, the results from TVS IV and LCVS III will all be relevant here. In particular, when one checks completeness, the condition that a net (xλ)λ∈Λ ⊂ X is Cauchy, is equivalent to the metric condition: (mc) for every ε > 0, there exists λε ∈ Λ, such that kxλ − xµk < ε, ∀ λ, µ λε. As pointed out in TVS IV, this condition is actually independent of the norm, that is, if one norm k . k satisfies (mc), then so does any norm equivalent to k . k. The following criteria have already been proved in TVS IV. Proposition 1. For a normed vector space (X , k . k), the following are equivalent. (i) X is a Banach space. (ii) Every Cauchy net in X is convergent. (iii) Every Cauchy sequence in X is convergent.
    [Show full text]