Mathematics of Information Hilbert Spaces and Linear Operators
Total Page:16
File Type:pdf, Size:1020Kb
Chair for Mathematical Information Science Verner Vlačić Sternwartstrasse 7 CH-8092 Zürich Mathematics of Information Hilbert spaces and linear operators These notes are based on [1, Chap. 6, Chap. 8], [2, Chap. 1], [3, Chap. 4], [4, App. A] and [5]. 1 Vector spaces Let us consider a field F, which can be, for example, the field of real numbers R or the field of complex numbers C.A vector space over F is a set whose elements are called vectors and in which two operations, addition and multiplication by any of the elements of the field F (referred to as scalars), are defined with some algebraic properties. More precisely, we have Definition 1 (Vector space). A set X together with two operations (+; ·) is a vector space over F if the following properties are satisfied: (i) the first operation, called vector addition: X × X ! X denoted by + satisfies • (x + y) + z = x + (y + z) for all x; y; z 2 X (associativity of addition) • x + y = y + x for all x; y 2 X (commutativity of addition) (ii) there exists an element 0 2 X , called the zero vector, such that x + 0 = x for all x 2 X (iii) for every x 2 X , there exists an element in X , denoted by −x, such that x + (−x) = 0. (iv) the second operation, called scalar multiplication: F × X ! X denoted by · satisfies • 1 · x = x for all x 2 X • α · (β · x) = (αβ) · x for all α; β 2 F and x 2 X (associativity for scalar multiplication) • (α+β)·x = α·x+β ·x for all α; β 2 F and x 2 X (distributivity of scalar multiplication with respect to field addition) • α·(x+y) = α·x+α·y for all α 2 F and x; y 2 X (distributivity of scalar multiplication with respect to vector addition). We refer to X as a real vector space when F = R and as a complex vector space when F = C. Examples. 1. C is both a real and a complex vector space. N 2. The set F , f(x1; x2; : : : ; xN ): xk 2 Fg of all N-tuples forms a vector space over F. 3. The set F[x] of all polynomials with coefficients in F is a vector space over F. 4. The space FZ of all sequences of F is a vector space over F. M×N 5. The set F of all matrices of size M × N with entries in F forms a vector space over F under the laws of matrix addition and scalar multiplication. 6. If X is an arbitrary set and Y an arbitrary vector space over F, the set F(X ; Y) of all functions X!Y is a vector space over F under pointwise addition and multiplication. A subspace of a vector space X over F is a subset of X which is itself a vector space over F (with respect to the same operations). One can easily verify that a subset Y of X is a subspace using the following proposition. Proposition 1.1. Let X be a vector space over F and Y a nonempty subset of X . Then Y is a subspace of X if it contains 0 and if it is stable under linear combinations, that is, α · y + β · z 2 Y for all α; β 2 F and y; z 2 Y. Examples. 1. R and iR are subspaces of the real vector space C. N 2. The set of N-tuples (x1; x2; : : : ; xN−1; 0) with xk 2 R is a subspace of R . p 3. The set ` (Z), p 2 [1; 1], of all complex-valued sequence fukgk2Z which satisfy P+1 p k=−∞ jukj < 1; if p 2 [1; 1); supk2Z jukj < 1; if p = 1; is a subspace of CZ. 4. If X is an arbitrary set, Y a vector space over F, and Z a subspace of Y, then the set F(X ; Z) of functions X!Z is a subspace of F(X ; Y). 5. The space Cn[a; b] of all complex-valued functions with continuous derivatives of order 0 6 k 6 n on the closed, bounded interval [a; b] of the real line is a subspace of F([a; b]; C). 6. If (S; Σ; µ) is a measure space, we let Lp(S; Σ; µ) denote the space of all mea- surable functions mapping S to C whose absolute value raised to the p-power is µ-integrable, that is, Z p p L (S; Σ; µ) = f : S! C measurable: jfj dµ < 1 ; S two elements of Lp(S; Σ; µ) being considered as equivalent if they differ only on p a set whose measure is zero. L (S; Σ; µ) is a subspace of F(S; C). For simplicity, p N we often use the notation L (S) when S is a subset of R , Σ is the Borel σ-algebra over S, and µ the Lebesgue measure on S. In this case, two functions are equivalent when they are equal except possibly on a set of Lebesgue measure 1 N zero (e.g., a finite or countable set of points of R ). 2 Theorem 1 (Intersection of vector spaces). The intersection of any collection of subspaces of a vector space X over F is again a subspace of X . Definition 2. If S and T are linear subspaces of a vector space X with S\T = f0g, then we define the direct sum S ⊕ T by S ⊕ T = fx + y j x 2 S; y 2 T g: 2 Note that the union of two subspaces is in general not a subspace. Take for example X = R . 2 The lines xR and yR, with x = (1; 0) and y = (0; 1), are both subspaces of R , but xR [ yR is a not subspace, given that x + y = (1; 1) 2= xR [ yR. Definition 3 (Subspace spanned by S). Let S be a (possibly infinite) subset of a vector space X over F. The subspace spanned by S is the intersection of all subspaces containing S. It is the smallest subspace containing S. It is denoted by span(S) and may be written the set of all finite combinations of elements of S, that is, ( k ) X span(S) , λ`x` : k 2 N; x` 2 S; λ` 2 F : `=1 Examples. 1. If S is empty, then the subspace spanned by S is f0g. 2. In the real vector space C, we have the following: • the subspace spanned by f1g is R, • the subspace spanned by fig is iR, • the subspace spanned by f1; ig is C. N 3. In F[x], the subspace spanned by f1; x; : : : ; x g is the space FN [x] of all polyno- mials whose degree is less than N. Definition 4 (Linear independence). Let X be a vector space over F. A finite set fx1; x2; : : : ; xN g of vectors of X is said to be linearly independent if for every λ1; λ2; : : : ; λN 2 F, the equality λ1x1 + λ2x2 + ::: + λN xN = 0 2 implies that λ1 = λ2 = ::: = λN = 0. An infinite set S of vectors of X is linearly independent if every finite subset of S is linearly independent. Examples. 1. Any set of vectors containing the zero vector is linearly dependent. 2. The real vector space C, the set f1; ig is linearly independent. 2 N 3. The basic monomials f1; x; x ; : : : ; x g form a linearly independent set of F[x]. 4. The set of trigonometric functions f1; cos(x); sin(x); cos2(x); cos(x) sin(x); sin2(x)g is linearly dependent. 1Note, however, that there exist uncountable sets (e.g., the Cantor set) of Lebesgue measure zero. 3 Definition 5 (Dimension). A vector space X is N-dimensional if there exists N linearly independent vectors in X and any N + 1 vectors in X are linearly dependent. Definition 6 (Finite-dimensional space). A vector space X is finite-dimensional if X is N- dimensional for some integer N. Otherwise, X is infinite dimensional. N Examples. 1. The spaces F and FN [x] are finite-dimensional (their dimension is N). n p 2. The space F(X ; X ), F[x], C [a; b], ` (Z) are infinite-dimensional. 3 Definition 7 (Basis). A basis of a vector space X over F is a set of linearly independent and spanning vectors of X . 3 T T T Examples. 1. The set fekgk=1, where e1 = [1 0 0] , e2 = [0 1 0] , and e3 = [0 0 1] , 3 forms a basis for R . 2 N 2. The set f1; x; x ; : : : ; x g forms a basis for FN [x]. 2 3 3. The set f1; x; x ; x ;:::g forms a basis for F[x]. Theorem 2. Let X be a finite dimensional vector space. Any set of linearly independent vectors can be extended to a basis of X . 2 Inner products and norms From here, we will assume that F = R or F = C. Definition 8 (Norm). Let X be a vector space over F. A norm on X is a function which maps X to R and satisfies the following properties: (i) for all x 2 X , we have kxk > 0 and kxk = 0 implies x = 0 (positivity) (ii) for all x 2 X and for all α 2 F, we have kαxk = jαj kxk (homogeneity) (iii) for all x; y 2 X , we have kx + yk 6 kxk + kyk (triangle inequality). Definition 9. A vector space X over F is a normed vector space (or a pre-Banach space) if it is equipped with a norm k·k.