L2 space of random functions L2 space of random functions Marek Dvoˇr´ak Hilbert Spaces Definition of Hilbert Space Marek Dvoˇr´ak Orthogonality Compact operators Matematicko-fyzik´aln´ıfakulta Univerzity Karlovy Riesz representation theorem Adjoint operators Singular values Hilbert-Schmidt operators Stochastic Modeling in Economics and Finance Space L2 in more detail 11.11.2013 Integral operator Covariance operator Asymptotic properties in L2 Estimation of mean and covariance functions 1/56 Outline L2 space of 1 Introduction to Hilbert Spaces random functions Definition of Hilbert Space Marek Dvoˇr´ak Orthogonality 2 Compact linear operators Hilbert Spaces Definition of Riesz representation theorem Hilbert Space Orthogonality Adjoint operators Compact Singular values operators Riesz Hilbert-Schmidt operators representation theorem 2 Adjoint 3 Space L in more detail operators Singular values Integral operator Hilbert-Schmidt operators Covariance operator 2 Space L2 in Asymptotic properties in L more detail Integral operator 4 Estimation of mean and covariance functions Covariance operator Estimation of mean Asymptotic properties in L2 Estimation of covariance Estimation of 5 Estimating eigenelements of the covariance operator mean and covariance functions 2/56 Outline L2 space of random 1 Introduction to Hilbert Spaces functions Definition of Hilbert Space Marek Dvoˇr´ak Orthogonality Hilbert Spaces 2 Compact linear operators Definition of Hilbert Space Riesz representation theorem Orthogonality Adjoint operators Compact Singular values operators Hilbert-Schmidt operators Riesz representation 2 theorem 3 Space L in more detail Adjoint operators Integral operator Singular values Covariance operator Hilbert-Schmidt 2 operators Asymptotic properties in L Space L2 in more detail 4 Estimation of mean and covariance functions Integral operator Estimation of mean Covariance operator Estimation of covariance Asymptotic properties in L2 5 Estimating eigenelements of the covariance operator Estimation of mean and covariance functions 3/56 Hilbert Space L2 space of random functions Marek Dvoˇr´ak Definition Hilbert Spaces Let H be a complex vector space. An inner product on H is a Definition of C C Hilbert Space function , : H H such that a, b , x, y, z H Orthogonality h· ·i × → ∀ ∈ ∀ ∈ Compact ax + by, z = a x, z + b y, z , operators h i h i h i Riesz x, y = y, x , representation h i h i theorem 2 2 Adjoint x := x, x 0 with equality x =0 iff x = 0. operators k k h i≥ k k Singular values Hilbert-Schmidt A complete inner product space H is called Hilbert space (= operators HS). Space L2 in more detail Integral operator Throughout the talk: separable Hilbert spaces (= spaces with Covariance operator Asymptotic countable orthonormal bases). properties in L2 Estimation of mean and covariance functions 4/56 Properties of the inner product L2 space of random functions Marek Dvoˇr´ak Hilbert Spaces Definition of Hilbert Space Inner product , Orthogonality h· ·i 1 fulfills Cauchy-Schwarz inequality: Compact operators Riesz representation x, y H : x, y x y , theorem ∀ ∈ |h i|≤k k · k k Adjoint operators Singular values 2 is a continuous mapping H H C. Hilbert-Schmidt operators × → Space L2 in more detail Integral operator Covariance operator Asymptotic properties in L2 Estimation of mean and covariance functions 5/56 Examples L2 space of random functions Cn Marek Dvoˇr´ak 1. Standard inner product on for x =(x1,..., xn), y =(y1,..., yn) Hilbert Spaces Definition of Hilbert Space n Orthogonality x, y := xi yi Compact h i operators i=1 Riesz X representation theorem 2 2 Adjoint 2. Space L = L ([0, 1]) of measurable real-valued function f operators 1 2 Singular values defined on [0, 1] satisfying f (t)dt < with the inner Hilbert-Schmidt 0 ∞ operators product Space L2 in R 1 more detail Integral operator f , g = f (t) g(t)dt. Covariance h i 0 operator Z Asymptotic properties in L2 Estimation of mean and covariance functions 6/56 Orthogonality L2 space of random functions Marek Dvoˇr´ak Definition Hilbert Spaces Definition of Hilbert Space x, y H are orthogonal (x y) iff x, y = 0. Orthogonality ∈ ⊥ h i Compact U H is orthogonal iff x, y U, x = y: x, y = 0. operators ⊂ ∀ ∈ 6 h i Riesz representation U, V H are orthogonal (U V ) iff x U, y V : theorem ⊂ ⊥ ∀ ∈ ∀ ∈ Adjoint operators x, y = 0. Singular values h i Hilbert-Schmidt An orthonormal basis U of HS is an orthogonal set of operators Space L2 in elements ui U such that ui = 1. more detail ∈ k k Integral operator Covariance operator Asymptotic properties in L2 Estimation of mean and covariance functions 7/56 Parseval identity L2 space of random functions Lemma (Parseval identity) Marek Dvoˇr´ak In the separable HS H with inner product , and orthonormal Hilbert Spaces h· ·i Definition of basis ej , for every x H holds the following: Hilbert Space { } ∈ Orthogonality ∞ Compact 2 2 operators x, ej = x . Riesz h i k k representation j=1 theorem X Adjoint operators Singular values Hilbert-Schmidt operators Remark Space L2 in more detail This is analogy to the Pythagorean theorem: The sum of the Integral operator Covariance operator squares of the components of a vector with respect to some Asymptotic properties in L2 orthonormal basis is equal to the squared length of the vector. Estimation of mean and covariance functions 8/56 Outline L2 space of random 1 Introduction to Hilbert Spaces functions Definition of Hilbert Space Marek Dvoˇr´ak Orthogonality Hilbert Spaces 2 Compact linear operators Definition of Hilbert Space Riesz representation theorem Orthogonality Adjoint operators Compact Singular values operators Hilbert-Schmidt operators Riesz representation 2 theorem 3 Space L in more detail Adjoint operators Integral operator Singular values Covariance operator Hilbert-Schmidt 2 operators Asymptotic properties in L Space L2 in more detail 4 Estimation of mean and covariance functions Integral operator Estimation of mean Covariance operator Estimation of covariance Asymptotic properties in L2 5 Estimating eigenelements of the covariance operator Estimation of mean and covariance functions 9/56 Space of bounded linear operators L2 space of random functions Marek Dvoˇr´ak Hilbert Spaces Definition of Hilbert Space Let H = H( , ) be a sep. HS with inner product that Orthogonality h· ·i generates the norm and let H be the space of Compact k · k L⊂ operators bounded linear operators on H with the norm Riesz representation theorem Adjoint operators Ψ L := sup Ψ(x) . Singular values k k x∈H: kxk≤1 k k Hilbert-Schmidt operators Space L2 in more detail Integral operator Covariance operator Asymptotic properties in L2 Estimation of mean and covariance functions 10/56 Riesz representation theorem L2 space of random functions Marek Dvoˇr´ak Theorem Hilbert Spaces If Ψ is bounded linear operator on a HS H then there exists Definition of Hilbert Space some g H such that for every f H Orthogonality ∈ ∈ Compact operators Ψ(f )= f , g . Riesz h i representation theorem Adjoint Moreover Ψ = g where the left norm is an operator norm operators k k k k Singular values and g is a HS-norm. Hilbert-Schmidt operators k k Space L2 in more detail Remark Integral operator Covariance Holds for separable and non-separable HS. operator Asymptotic properties in L2 Estimation of mean and covariance functions 11/56 (Self) adjoint operator L2 space of random Definition functions Let Ψ : H1 H2 is bounded linear operator. The adjoint Marek Dvoˇr´ak → operator is defined as follows: For each y H1 the linear Hilbert Spaces ∈ Definition of functional Ψ(x), y is bounded on H1. Hence, according to Hilbert Space h i Orthogonality Riesz, for every y H there exists unique z H such that ∈ 2 ∈ 1 Compact operators Riesz Ψ(x), y = x, z x H1. representation h i h i∀ ∈ theorem Adjoint ∗ ∗ operators Define an operator Ψ : H2 H1 : Ψ (y) := z. Such Singular values ∗ → Hilbert-Schmidt operator Ψ is called adjoint. operators Space L2 in more detail Remark Integral operator Covariance ∗ operator It holds that Ψ(x), y = x, Ψ (y) x H1 and y H2. If Ψ Asymptotic h i h∗ i ∀ ∈ ∈ properties in L2 is acting on H and Ψ Ψ which means that ≡ Estimation of Ψ(x), y = x, Ψ(y) x, y H then Ψ is called self-adjoint. mean and h i h i ∀ ∈ covariance functions 12/56 Preliminary for singular values L2 space of random functions Definition Marek Dvoˇr´ak Positive-definiteness: Ψ(x), x 0. h i≥ Hilbert Spaces Eigenelements: Ψ(vj )= λj vj . Definition of Hilbert Space Orthogonality Lemma Compact operators ∗ Riesz Let Ψ: H1 H2 be a compact operator. Then, Ψ Ψ is a representation → theorem compact positive definite self-adjoint operator on H1. Adjoint operators Singular values Hilbert-Schmidt operators Proof. Space L2 in Ψ∗Ψ(x), x = Ψ(x), Ψ(x) = Ψ(x) 2 0 more detail h i h i k k ≥ ⇒ Integral operator positive-definite. Covariance operator ∗ ∗ Asymptotic Ψ Ψ(x), y = Ψ(x), Ψ(y) = x, Ψ Ψ(y) properties in L2 h i h i h i ⇒ self-adjoint. Estimation of mean and covariance functions 13/56 Singular values L2 space of random functions Definition Marek Dvoˇr´ak We say that operator Ψ is of finite rank if number of nonzero ∗ Hilbert Spaces eigenvalues of the operator Ψ Ψ is finite. Definition of Hilbert Space Orthogonality Definition Compact operators By definition, for j = 1, 2, ... the j-th singular value or j-th Riesz representation theorem s-number of the operator Ψ is the number Adjoint operators Singular values Hilbert-Schmidt ∗ operators sj (Ψ) := λj (Ψ Ψ) Space L2 in q more detail Integral operator where Covariance ∗ ∗ operator λ1(Ψ Ψ) λ2(Ψ Ψ) ... Asymptotic ≥ ≥ 2 properties in L is non-increasing sequence of eigenvalues of the operator Ψ∗Ψ.