DOCSLIB.ORG

Linear Algebra Chapter 5: Norms, Inner Products and Orthogonality

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Linear Algebra Chapter 5: Norms, Inner Products and Orthogonality MATH 532: Linear Algebra Chapter 5: Norms, Inner Products and Orthogonality Greg Fasshauer Department of Applied Mathematics Illinois Institute of Technology Spring 2015 [email protected] MATH 532 1 Outline 1 Vector Norms 2 Matrix Norms 3 Inner Product Spaces 4 Orthogonal Vectors 5 Gram–Schmidt Orthogonalization & QR Factorization 6 Unitary and Orthogonal Matrices 7 Orthogonal Reduction 8 Complementary Subspaces 9 Orthogonal Decomposition 10 Singular Value Decomposition 11 Orthogonal Projections [email protected] MATH 532 2 Vector Norms Vector[0] Norms 1 Vector Norms 2 Matrix Norms Definition 3 Inner Product Spaces Let x; y 2 Rn (Cn). Then 4 Orthogonal Vectors n T X 5 Gram–Schmidt Orthogonalization & QRx Factorizationy = xi yi 2 R i=1 6 Unitary and Orthogonal Matrices n X ∗ = ¯ 2 7 Orthogonal Reduction x y xi yi C i=1 8 Complementary Subspaces is called the standard inner product for Rn (Cn). 9 Orthogonal Decomposition 10 Singular Value Decomposition 11 Orthogonal Projections [email protected] MATH 532 4 Vector Norms Definition Let V be a vector space. A function k · k : V! R≥0 is called a norm provided for any x; y 2 V and α 2 R 1 kxk ≥ 0 and kxk = 0 if and only if x = 0, 2 kαxk = jαj kxk, 3 kx + yk ≤ kxk + kyk. Remark The inequality in (3) is known as the triangle inequality. [email protected] MATH 532 5 Vector Norms Remark Any inner product h·; ·i induces a norm via (more later) p kxk = hx; xi: We will show that the standard inner product induces the Euclidean norm (cf. length of a vector). Remark Inner products let us define angles via xT y cos θ = : kxkkyk In particular, x; y are orthogonal if and only if xT y = 0. [email protected] MATH 532 6 Vector Norms Example Let x 2 Rn and consider the Euclidean norm p T kxk2 = x x n !1=2 X 2 = xi : i=1 We show that k · k2 is a norm. We do this for the real case, but the complex case goes analogously. 1 Clearly, kxk2 ≥ 0. Also, 2 kxk2 = 0 () kxk2 = 0 n X 2 () xi = 0 () xi = 0; i = 1;:::; n; i=1 () x = 0: [email protected] MATH 532 7 Vector Norms Example (cont.) 2 We have n !1=2 n !1=2 X 2 X 2 kαxk2 = (αxi ) = jαj xi = jαj kxk2: i=1 i=1 3 To establish (3) we need Lemma Let x; y 2 Rn. Then T jx yj ≤ kxk2kyk2: (Cauchy–Schwarz–Bunyakovsky) Moreover, equality holds if and only if y = αx with xT y α = : 2 kxk2 [email protected] MATH 532 8 Vector Norms Motivation for Proof of Cauchy–Schwarz–Bunyakovsky As already alluded to above, the angle θ between two vectors a and b is related to the inner product by aT b cos θ = : kakkbk Using trigonometry as in the figure, the projection of a onto b is then b aT b b aT b kak cos θ = kak = b: kbk kakkbk kbk kbk2 Now, we let y = a and x = b, so that the projection of y onto x is given by xT y αx; where α = : kxk2 [email protected] MATH 532 9 Vector Norms Proof of Cauchy–Schwarz–Bunyakovsky 2 We know that ky − αxk2 ≥ 0 since it’s (the square of) a norm. Therefore, 2 T 0 ≤ ky − αxk2 = (y − αx) (y − αx) = y T y − 2αxT y + α2xT x 2 xT y xT y = y T y − 2 xT y + xT x kxk2 kxk4 |{z} 2 =kxk2 2 xT y = k k2 − : y 2 2 kxk2 This implies 2 T 2 2 x y ≤ kxk2kyk2; and the Cauchy–Schwarz–Bunyakovsky inequality follows by taking square roots. [email protected] MATH 532 10 Vector Norms Proof (cont.) Now we look at the equality claim. T “=)”: Let’s assume that x y = kxk2kyk2. But then the first part of the proof shows that ky − αxk2 = 0 so that y = αx. “(=”: Let’s assume y = αx. Then T T 2 x y = x (αx) = jαjkxk2 2 kxk2kyk2 = kxk2kαxk2 = jαjkxk2; so that we have equality. [email protected] MATH 532 11 Vector Norms Example (cont.) 3 Now we can show that k · k2 satisfies the triangle inequality: 2 T kx + yk2 = (x + y) (x + y) = xT x +xT y + y T x + y T y |{z} |{z} |{z} =kxk2 = T 2 2 x y =kyk2 2 T 2 = kxk2 + 2x y + kyk2 2 T 2 ≤ kxk2 + 2jx yj + kyk2 CSB 2 2 ≤ kxk2 + 2kxk2kyk2 + kyk2 2 = (kxk2 + kyk2) : Now we just need to take square roots to have the triangle inequality. [email protected] MATH 532 12 Vector Norms Lemma Let x; y 2 Rn. Then we have the backward triangle inequality j kxk − kyk j ≤ kx − yk: Proof We write tri.ineq. kxk = kx − y + yk ≤ kx − yk + kyk: But this implies kxk − kyk ≤ kx − yk: [email protected] MATH 532 13 Vector Norms Proof (cont.) Switch the roles of x and y to get kyk − kxk ≤ ky − xk () − (kxk − kyk) ≤ kx − yk: Together with the previous inequality we have jkxk − kykj ≤ kx − yk: [email protected] MATH 532 14 Vector Norms Other common norms `1-norm (or taxi-cab norm, Manhattan norm): n X kxk1 = jxi j i=1 `1-norm (or maximum norm, Chebyshev norm): kxk1 = max jxi j 1≤i≤n `p-norm: n !1=p X p kxkp = jxi j i=1 Remark In the homework you will use Hölder’s and Minkowski’s inequalities to show that the p-norm is a norm. [email protected] MATH 532 15 Vector Norms Remark We now show that kxk1 = lim kxkp: p!1 Let’s use tildes to mark all components of x that are maximal, i.e.. x~1 = x~2 = ::: = x~k = max jxi j: 1≤i≤n The remaining components are then x~k+1;:::; x~n. This implies that x~ i < 1; for i = k + 1;:::; n: x~1 [email protected] MATH 532 16 Vector Norms Remark (cont.) Now n !1=p X p kxkp = jx~i j i=1 0 11=p B x~ p x~ pC ~ B k+1 n C = jx1j Bk + + ::: + C : @ x~1 x~1 A | {z } |{z} <1 <1 Since the terms inside the parentheses — except for k — go to 0 for p ! 1, (·)1=p ! 1 for p ! 1. And so kxkp ! jx~1j = max jxi j = kxk1: 1≤i≤n [email protected] MATH 532 17 Vector Norms 2 Figure: Unit “balls” in R for the `1, `2 and `1 norms. Note that B1 ⊆ B2 ⊆ B1 since, e.g., p ! p ! p ! p 2 p 2 q 2 2 p2 = 2; p2 = 1 + 1 = 1; p2 = ; 2 2 2 2 2 2 2 1 2 2 2 1 p ! p ! p ! 2 2 2 so that p2 ≥ p2 ≥ p2 : 2 2 2 2 1 2 2 2 1 In fact, we have in general (similar to HW) n kxk1 ≥ kxk2 ≥ kxk1; for any x 2 R : [email protected] MATH 532 18 Vector Norms Norm equivalence Definition Two norms k · k and k · k0 on a vector space V are called equivalent if there exist constants α; β such that kxk α ≤ ≤ β for all x(6= 0) 2 V: kxk0 Example k · k1 andp k · k2 are equivalent since from above kxk1 ≥ kxk2 and also kxk1 ≤ nkxk2 (see HW) so that kxk p α = 1 ≤ 1 ≤ n = β: kxk2 Remark In fact, all norms on finite-dimensional vector spaces are equivalent. [email protected] MATH 532 19 Matrix Norms Matrix[0] norms are special norms — they will satisfy one additional property1 Vector Norms. This property should help us measure kABk for two matrices A; B of appropriate2 Matrix Norms sizes. We3 lookInner Product at the Spaces simplest matrix norm, the Frobenius norm, defined for ; A 2 Rm n: 4 Orthogonal Vectors 1=2 0 m n 1 m !1=2 5 Gram–Schmidt OrthogonalizationX & QRX Factorization2 X 2 kAkF = @ jaij j A = kAi∗k2 6 Unitary and Orthogonal Matricesi=1 j=1 i=1 0 11=2 7 Orthogonal Reduction n q X 2 T = @ kA∗j k2A = trace(A A); 8 Complementary Subspaces j=1 9 Orthogonal Decomposition i.e., the Frobenius norm is just a 2-norm for the vector that contains all elements10 Singular Value of the Decomposition matrix. 11 Orthogonal Projections [email protected] MATH 532 21 Matrix Norms Now m 2 X 2 kAxk2 = jAi∗xj i=1 m CSB X 2 2 ≤ kAi∗k2 kxk2 i=1 | {z } 2 =kAkF so that kAxk2 ≤ kAkF kxk2: We can generalize this to matrices, i.e., we have kABkF ≤ kAkF kBkF ; which motivates us to require this submultiplicativity for any matrix norm. [email protected] MATH 532 22 Matrix Norms Definition A matrix norm is a function k · k from the set of all real (or complex) matrices of finite size into R≥0 that satisfies 1 kAk ≥ 0 and kAk = 0 if and only if A = O (a matrix of all zeros). 2 kαAk = jαjkAk for all α 2 R. 3 kA + Bk ≤ kAk + kBk (requires A; B to be of same size). 4 kABk ≤ kAkkBk (requires A; B to have appropriate sizes). Remark This definition is usually too general. In addition to the Frobenius norm, most useful matrix norms are induced by a vector norm. [email protected] MATH 532 23 Matrix Norms Induced matrix norms Theorem m n Let k · k(m) and k · k(n) be vector norms on R and R , respectively, and let A be an m × n matrix.
Load more

Recommended publications
  • A New Description of Space and Time Using Clifford Multivectors
    A new description of space and time using Clifford multivectors James M. Chappell† , Nicolangelo Iannella† , Azhar Iqbal† , Mark Chappell‡ , Derek Abbott† †School of Electrical and Electronic Engineering, University of Adelaide, South Australia 5005, Australia ‡Griffith Institute, Griffith University, Queensland 4122, Australia Abstract Following the development of the special theory of relativity in 1905, Minkowski pro- posed a unified space and time structure consisting of three space dimensions and one time dimension, with relativistic effects then being natural consequences of this space- time geometry. In this paper, we illustrate how Clifford’s geometric algebra that utilizes multivectors to represent spacetime, provides an elegant mathematical framework for the study of relativistic phenomena. We show, with several examples, how the application of geometric algebra leads to the correct relativistic description of the physical phenomena being considered. This approach not only provides a compact mathematical representa- tion to tackle such phenomena, but also suggests some novel insights into the nature of time. Keywords: Geometric algebra, Clifford space, Spacetime, Multivectors, Algebraic framework 1. Introduction The physical world, based on early investigations, was deemed to possess three inde- pendent freedoms of translation, referred to as the three dimensions of space. This naive conclusion is also supported by more sophisticated analysis such as the existence of only five regular polyhedra and the inverse square force laws. If we lived in a world with four spatial dimensions, for example, we would be able to construct six regular solids, and in arXiv:1205.5195v2 [math-ph] 11 Oct 2012 five dimensions and above we would find only three [1].
    [Show full text]
  • Orthogonal Projection, Low Rank Approximation, and Orthogonal Bases
    Week 11 Orthogonal Projection, Low Rank Approximation, and Orthogonal Bases 11.1 Opening Remarks 11.1.1 Low Rank Approximation * View at edX 463 Week 11. Orthogonal Projection, Low Rank Approximation, and Orthogonal Bases 464 11.1.2 Outline 11.1. Opening Remarks..................................... 463 11.1.1. Low Rank Approximation............................. 463 11.1.2. Outline....................................... 464 11.1.3. What You Will Learn................................ 465 11.2. Projecting a Vector onto a Subspace........................... 466 11.2.1. Component in the Direction of ............................ 466 11.2.2. An Application: Rank-1 Approximation...................... 470 11.2.3. Projection onto a Subspace............................. 474 11.2.4. An Application: Rank-2 Approximation...................... 476 11.2.5. An Application: Rank-k Approximation...................... 478 11.3. Orthonormal Bases.................................... 481 11.3.1. The Unit Basis Vectors, Again........................... 481 11.3.2. Orthonormal Vectors................................ 482 11.3.3. Orthogonal Bases.................................. 485 11.3.4. Orthogonal Bases (Alternative Explanation).................... 488 11.3.5. The QR Factorization................................ 492 11.3.6. Solving the Linear Least-Squares Problem via QR Factorization......... 493 11.3.7. The QR Factorization (Again)........................... 494 11.4. Change of Basis...................................... 498 11.4.1. The Unit Basis Vectors,
    [Show full text]
  • Eigenvalues and Eigenvectors
    Jim Lambers MAT 605 Fall Semester 2015-16 Lecture 14 and 15 Notes These notes correspond to Sections 4.4 and 4.5 in the text. Eigenvalues and Eigenvectors In order to compute the matrix exponential eAt for a given matrix A, it is helpful to know the eigenvalues and eigenvectors of A. Definitions and Properties Let A be an n × n matrix. A nonzero vector x is called an eigenvector of A if there exists a scalar λ such that Ax = λx: The scalar λ is called an eigenvalue of A, and we say that x is an eigenvector of A corresponding to λ. We see that an eigenvector of A is a vector for which matrix-vector multiplication with A is equivalent to scalar multiplication by λ. Because x is nonzero, it follows that if x is an eigenvector of A, then the matrix A − λI is singular, where λ is the corresponding eigenvalue. Therefore, λ satisfies the equation det(A − λI) = 0: The expression det(A−λI) is a polynomial of degree n in λ, and therefore is called the characteristic polynomial of A (eigenvalues are sometimes called characteristic values). It follows from the fact that the eigenvalues of A are the roots of the characteristic polynomial that A has n eigenvalues, which can repeat, and can also be complex, even if A is real. However, if A is real, any complex eigenvalues must occur in complex-conjugate pairs. The set of eigenvalues of A is called the spectrum of A, and denoted by λ(A). This terminology explains why the magnitude of the largest eigenvalues is called the spectral radius of A.
    [Show full text]
  • Algebra of Linear Transformations and Matrices Math 130 Linear Algebra
    Then the two compositions are 0 −1 1 0 0 1 BA = = 1 0 0 −1 1 0 Algebra of linear transformations and 1 0 0 −1 0 −1 AB = = matrices 0 −1 1 0 −1 0 Math 130 Linear Algebra D Joyce, Fall 2013 The products aren't the same. You can perform these on physical objects. Take We've looked at the operations of addition and a book. First rotate it 90◦ then flip it over. Start scalar multiplication on linear transformations and again but flip first then rotate 90◦. The book ends used them to define addition and scalar multipli- up in different orientations. cation on matrices. For a given basis β on V and another basis γ on W , we have an isomorphism Matrix multiplication is associative. Al- γ ' φβ : Hom(V; W ) ! Mm×n of vector spaces which though it's not commutative, it is associative. assigns to a linear transformation T : V ! W its That's because it corresponds to composition of γ standard matrix [T ]β. functions, and that's associative. Given any three We also have matrix multiplication which corre- functions f, g, and h, we'll show (f ◦ g) ◦ h = sponds to composition of linear transformations. If f ◦ (g ◦ h) by showing the two sides have the same A is the standard matrix for a transformation S, values for all x. and B is the standard matrix for a transformation T , then we defined multiplication of matrices so ((f ◦ g) ◦ h)(x) = (f ◦ g)(h(x)) = f(g(h(x))) that the product AB is be the standard matrix for S ◦ T .
    [Show full text]
  • Lecture 4: April 8, 2021 1 Orthogonality and Orthonormality
    Mathematical Toolkit Spring 2021 Lecture 4: April 8, 2021 Lecturer: Avrim Blum (notes based on notes from Madhur Tulsiani) 1 Orthogonality and orthonormality Definition 1.1 Two vectors u, v in an inner product space are said to be orthogonal if hu, vi = 0. A set of vectors S ⊆ V is said to consist of mutually orthogonal vectors if hu, vi = 0 for all u 6= v, u, v 2 S. A set of S ⊆ V is said to be orthonormal if hu, vi = 0 for all u 6= v, u, v 2 S and kuk = 1 for all u 2 S. Proposition 1.2 A set S ⊆ V n f0V g consisting of mutually orthogonal vectors is linearly inde- pendent. Proposition 1.3 (Gram-Schmidt orthogonalization) Given a finite set fv1,..., vng of linearly independent vectors, there exists a set of orthonormal vectors fw1,..., wng such that Span (fw1,..., wng) = Span (fv1,..., vng) . Proof: By induction. The case with one vector is trivial. Given the statement for k vectors and orthonormal fw1,..., wkg such that Span (fw1,..., wkg) = Span (fv1,..., vkg) , define k u + u = v − hw , v i · w and w = k 1 . k+1 k+1 ∑ i k+1 i k+1 k k i=1 uk+1 We can now check that the set fw1,..., wk+1g satisfies the required conditions. Unit length is clear, so let’s check orthogonality: k uk+1, wj = vk+1, wj − ∑ hwi, vk+1i · wi, wj = vk+1, wj − wj, vk+1 = 0. i=1 Corollary 1.4 Every finite dimensional inner product space has an orthonormal basis.
    [Show full text]
  • Math 217: Multilinearity of Determinants Professor Karen Smith (C)2015 UM Math Dept Licensed Under a Creative Commons By-NC-SA 4.0 International License
    Math 217: Multilinearity of Determinants Professor Karen Smith (c)2015 UM Math Dept licensed under a Creative Commons By-NC-SA 4.0 International License. A. Let V −!T V be a linear transformation where V has dimension n. 1. What is meant by the determinant of T ? Why is this well-defined? Solution note: The determinant of T is the determinant of the B-matrix of T , for any basis B of V . Since all B-matrices of T are similar, and similar matrices have the same determinant, this is well-defined—it doesn't depend on which basis we pick. 2. Define the rank of T . Solution note: The rank of T is the dimension of the image. 3. Explain why T is an isomorphism if and only if det T is not zero. Solution note: T is an isomorphism if and only if [T ]B is invertible (for any choice of basis B), which happens if and only if det T 6= 0. 3 4. Now let V = R and let T be rotation around the axis L (a line through the origin) by an 21 0 0 3 3 angle θ. Find a basis for R in which the matrix of ρ is 40 cosθ −sinθ5 : Use this to 0 sinθ cosθ compute the determinant of T . Is T othogonal? Solution note: Let v be any vector spanning L and let u1; u2 be an orthonormal basis ? for V = L . Rotation fixes ~v, which means the B-matrix in the basis (v; u1; u2) has 213 first column 405.
    [Show full text]
  • Basic Multivector Calculus, Proceedings of FAN 2008, Hiroshima, Japan, 23+24 October 2008
    E. Hitzer, Basic Multivector Calculus, Proceedings of FAN 2008, Hiroshima, Japan, 23+24 October 2008. Basic Multivector Calculus Eckhard Hitzer, Department of Applied Physics, Faculty of Engineering, University of Fukui Abstract: We begin with introducing the generalization of real, complex, and quaternion numbers to hypercomplex numbers, also known as Clifford numbers, or multivectors of geometric algebra. Multivectors encode everything from vectors, rotations, scaling transformations, improper transformations (reflections, inversions), geometric objects (like lines and spheres), spinors, and tensors, and the like. Multivector calculus allows to define functions mapping multivectors to multivectors, differentiation, integration, function norms, multivector Fourier transformations and wavelet transformations, filtering, windowing, etc. We give a basic introduction into this general mathematical language, which has fascinating applications in physics, engineering, and computer science. more didactic approach for newcomers to geometric algebra. 1. Introduction G(I) is the full geometric algebra over all vectors in the “Now faith is being sure of what we hope for and certain of what we do not see. This is what the ancients were n-dimensional unit pseudoscalar I e1 e2 ... en . commended for. By faith we understand that the universe was formed at God’s command, so that what is seen was not made 1 A G (I) is the n-dimensional vector sub-space of out of what was visible.” [7] n The German 19th century mathematician H. Grassmann had grade-1 elements in G(I) spanned by e1,e2 ,...,en . For the clear vision, that his “extension theory (now developed to geometric calculus) … forms the keystone of the entire 1 vectors a,b,c An G (I) and scalars , ,, ; structure of mathematics.”[6] The algebraic “grammar” of this universal form of calculus is geometric algebra (or Clifford G(I) has the fundamental properties of algebra).
    [Show full text]
  • Orthogonality Handout
    3.8 (SUPPLEMENT) | ORTHOGONALITY OF EIGENFUNCTIONS We now develop some properties of eigenfunctions, to be used in Chapter 9 for Fourier Series and Partial Differential Equations. 1. Definition of Orthogonality R b We say functions f(x) and g(x) are orthogonal on a < x < b if a f(x)g(x) dx = 0 . [Motivation: Let's approximate the integral with a Riemann sum, as follows. Take a large integer N, put h = (b − a)=N and partition the interval a < x < b by defining x1 = a + h; x2 = a + 2h; : : : ; xN = a + Nh = b. Then Z b f(x)g(x) dx ≈ f(x1)g(x1)h + ··· + f(xN )g(xN )h a = (uN · vN )h where uN = (f(x1); : : : ; f(xN )) and vN = (g(x1); : : : ; g(xN )) are vectors containing the values of f and g. The vectors uN and vN are said to be orthogonal (or perpendicular) if their dot product equals zero (uN ·vN = 0), and so when we let N ! 1 in the above formula it makes R b sense to say the functions f and g are orthogonal when the integral a f(x)g(x) dx equals zero.] R π 1 2 π Example. sin x and cos x are orthogonal on −π < x < π, since −π sin x cos x dx = 2 sin x −π = 0. 2. Integration Lemma Suppose functions Xn(x) and Xm(x) satisfy the differential equations 00 Xn + λnXn = 0; a < x < b; 00 Xm + λmXm = 0; a < x < b; for some numbers λn; λm. Then Z b 0 0 b (λn − λm) Xn(x)Xm(x) dx = [Xn(x)Xm(x) − Xn(x)Xm(x)]a: a Proof.
    [Show full text]
  • MATH 304 Linear Algebra Lecture 24: Scalar Product. Vectors: Geometric Approach
    MATH 304 Linear Algebra Lecture 24: Scalar product. Vectors: geometric approach B A B′ A′ A vector is represented by a directed segment. • Directed segment is drawn as an arrow. • Different arrows represent the same vector if • they are of the same length and direction. Vectors: geometric approach v B A v B′ A′ −→AB denotes the vector represented by the arrow with tip at B and tail at A. −→AA is called the zero vector and denoted 0. Vectors: geometric approach v B − A v B′ A′ If v = −→AB then −→BA is called the negative vector of v and denoted v. − Vector addition Given vectors a and b, their sum a + b is defined by the rule −→AB + −→BC = −→AC. That is, choose points A, B, C so that −→AB = a and −→BC = b. Then a + b = −→AC. B b a C a b + B′ b A a C ′ a + b A′ The difference of the two vectors is defined as a b = a + ( b). − − b a b − a Properties of vector addition: a + b = b + a (commutative law) (a + b) + c = a + (b + c) (associative law) a + 0 = 0 + a = a a + ( a) = ( a) + a = 0 − − Let −→AB = a. Then a + 0 = −→AB + −→BB = −→AB = a, a + ( a) = −→AB + −→BA = −→AA = 0. − Let −→AB = a, −→BC = b, and −→CD = c. Then (a + b) + c = (−→AB + −→BC) + −→CD = −→AC + −→CD = −→AD, a + (b + c) = −→AB + (−→BC + −→CD) = −→AB + −→BD = −→AD. Parallelogram rule Let −→AB = a, −→BC = b, −−→AB′ = b, and −−→B′C ′ = a. Then a + b = −→AC, b + a = −−→AC ′.
    [Show full text]
  • Inner Products and Orthogonality
    Advanced Linear Algebra – Week 5 Inner Products and Orthogonality This week we will learn about: • Inner products (and the dot product again), • The norm induced by the inner product, • The Cauchy–Schwarz and triangle inequalities, and • Orthogonality. Extra reading and watching: • Sections 1.3.4 and 1.4.1 in the textbook • Lecture videos 17, 18, 19, 20, 21, and 22 on YouTube • Inner product space at Wikipedia • Cauchy–Schwarz inequality at Wikipedia • Gram–Schmidt process at Wikipedia Extra textbook problems: ? 1.3.3, 1.3.4, 1.4.1 ?? 1.3.9, 1.3.10, 1.3.12, 1.3.13, 1.4.2, 1.4.5(a,d) ??? 1.3.11, 1.3.14, 1.3.15, 1.3.25, 1.4.16 A 1.3.18 1 Advanced Linear Algebra – Week 5 2 There are many times when we would like to be able to talk about the angle between vectors in a vector space V, and in particular orthogonality of vectors, just like we did in Rn in the previous course. This requires us to have a generalization of the dot product to arbitrary vector spaces. Definition 5.1 — Inner Product Suppose that F = R or F = C, and V is a vector space over F. Then an inner product on V is a function h·, ·i : V × V → F such that the following three properties hold for all c ∈ F and all v, w, x ∈ V: a) hv, wi = hw, vi (conjugate symmetry) b) hv, w + cxi = hv, wi + chv, xi (linearity in 2nd entry) c) hv, vi ≥ 0, with equality if and only if v = 0.
    [Show full text]
  • Eigenvalues, Eigenvectors and the Similarity Transformation
    Eigenvalues, Eigenvectors and the Similarity Transformation Eigenvalues and the associated eigenvectors are ‘special’ properties of square matrices. While the eigenvalues parameterize the dynamical properties of the system (timescales, resonance properties, amplification factors, etc) the eigenvectors define the vector coordinates of the normal modes of the system. Each eigenvector is associated with a particular eigenvalue. The general state of the system can be expressed as a linear combination of eigenvectors. The beauty of eigenvectors is that (for square symmetric matrices) they can be made orthogonal (decoupled from one another). The normal modes can be handled independently and an orthogonal expansion of the system is possible. The decoupling is also apparent in the ability of the eigenvectors to diagonalize the original matrix, A, with the eigenvalues lying on the diagonal of the new matrix, . In analogy to the inertia tensor in mechanics, the eigenvectors form the principle axes of the solid object and a similarity transformation rotates the coordinate system into alignment with the principle axes. Motion along the principle axes is decoupled. The matrix mechanics is closely related to the more general singular value decomposition. We will use the basis sets of orthogonal eigenvectors generated by SVD for orbit control problems. Here we develop eigenvector theory since it is more familiar to most readers. Square matrices have an eigenvalue/eigenvector equation with solutions that are the eigenvectors xand the associated eigenvalues : Ax = x The special property of an eigenvector is that it transforms into a scaled version of itself under the operation of A. Note that the eigenvector equation is non-linear in both the eigenvalue () and the eigenvector (x).
    [Show full text]
  • A Guided Tour to the Plane-Based Geometric Algebra PGA
    A Guided Tour to the Plane-Based Geometric Algebra PGA Leo Dorst University of Amsterdam Version 1.15{ July 6, 2020 Planes are the primitive elements for the constructions of objects and oper- ators in Euclidean geometry. Triangulated meshes are built from them, and reflections in multiple planes are a mathematically pure way to construct Euclidean motions. A geometric algebra based on planes is therefore a natural choice to unify objects and operators for Euclidean geometry. The usual claims of `com- pleteness' of the GA approach leads us to hope that it might contain, in a single framework, all representations ever designed for Euclidean geometry - including normal vectors, directions as points at infinity, Pl¨ucker coordinates for lines, quaternions as 3D rotations around the origin, and dual quaternions for rigid body motions; and even spinors. This text provides a guided tour to this algebra of planes PGA. It indeed shows how all such computationally efficient methods are incorporated and related. We will see how the PGA elements naturally group into blocks of four coordinates in an implementation, and how this more complete under- standing of the embedding suggests some handy choices to avoid extraneous computations. In the unified PGA framework, one never switches between efficient representations for subtasks, and this obviously saves any time spent on data conversions. Relative to other treatments of PGA, this text is rather light on the mathematics. Where you see careful derivations, they involve the aspects of orientation and magnitude. These features have been neglected by authors focussing on the mathematical beauty of the projective nature of the algebra.
    [Show full text]