Linear Algebra Review

Total Page:16

File Type:pdf, Size:1020Kb

Linear Algebra Review Linear algebra review Dec 10, 2014 1. Solving system of linear equations: Given a system of linear equation Ax = b, where A is an m n matrix. × (a) We first observe the augmented matrix and perform elementary row operations to find the reduced echelon form: [A b] [R c]. | −→ | If there are some rows in [R c] of the form | [0 0 d], ··· where d =0,thenthesystemoflinearequationshasnosolutionandisinconsis- tent; otherwise$ we only need to observe the coefficient matrix A: If rank A = n, then there is a unique solution (no free variables); If rank A<n, then there are infinitely many solutions (still has free variables). Note rank A can never be greater than either m or n. (b) To find the general solutions to the system of linear equations, we still need to use the augmented matrix [R c], the concrete steps are as follows: | Recover a “new” system of linear equations from [R c], solve basic variables using free variables, write them in the parametric vector| form. (c) If A is square matrix (number of the equations equals the number of variables), then Cramer’s rule is an efficient method to determine the existence as well as to calculate the exact solutions of a system of linear equations. There will be formulas involving determinant for the solution. 2. Homogeneous system of linear equations Ax = 0 There are only two possible types of solutions for homogeneous system: If rank A = n (column vectors of A are linearly independent), only trivial solution. If rank A<n, infinitely many solutions. The set of all solutions forms a vector space, called Null space of A. 1 3. Linear dependence and independence (a) Definition: Ax = 0 only has trivial solution or not (b) Criterion: rank A = n (independent) or not (dependent). 4. Subspace, basis, dimension (a) Definitions and examples (e.g. linear combination of a set of vectors) (b) Some distinguished examples: Row(A), Col(A), Null(A), Eigλ(A), W ⊥ (c) Basis and dimension (d) Relations: T Row(A)=Col(A ), Null(A)=Row(A)⊥. 5. Invertible matrix, elementary matrix (a) Definition and Criterion, 2 2example × (b) Calculate the inverse: Elementary row operations, adjoint matrix (c) Correspondence between elementary matrices and elementary row operations; in- verse of the elementary matrices 6. Determinant (a) Cofactor expansions (b) Properties under elementary row operations (c) Applications: Geometry of determinant, Cramer’s rule, adjoint matrix. 7. Linear transformation (a) Definition and examples, T : Rn Rm maps x to Ax. −→ (b) Distinguished examples: Orthogonal projections, differential operators (c) One-to-one (rank A =n)andonto(rankA =m). 8. Change of basis (a) Change of the coordinate from basis to standard basis: B Let = b ,...,b , B { 1 n} and let P =[b1 bn]. B |···| Then x = P [x] . B B 2 (b) Change of the coordinate from basis to basis : B C [x] = P [x] , C C←B B where 1 P = P − P . C←B C B (c) The linear transformation under change of basis The general formula is on Page 289 of the book. A special case, let T : Rn Rn −→ x Ax %−→ and = b1,...,bn , then B { } 1 [T ] = P − AP . B B B 1 This result can be applied to a diagonalizable matrix A = PDP− , where P is invertible and D is diagonal. If we change the standard basis to a basis consisting of columns of P (hence P = P ), then [T ] = D. B B B 9. Eigenvalues and Eigenspaces (a) Definitions (b) Characteristic polynomials 10. Diagonalization (a) Definition (b) Examples: Matrices with mutually distinct eigenvalues. (c) Criterion: For each eigenvalue λ, mult(λ)=dimEig (A)=n rank(A λI ) λ − − n (d) Find invertible matrix P (formed by eigenvectors as its columns) and D (eigen- 1 values as the diagonal entries, correspondingly) such that A = PDP− . 11. Inner product and orthogonality (a) Properties, norm, normalize, some inequalities (b) Orthogonal (orthonormal) sets, orthogonal (orthonormal) basis (c) Decomposition of a vector under orthogonal (orthonormal) basis (d) Orthogonal matrix (e) Orthogonal complement 3 12. Gram-Schmidt process, QR decomposition The formula is on page 355, by applying Gram-Schmidt, a basis is converted to an orthogonal basis. 13. Orthogonal projection Let ProjW y = PW y to denote the orthogonal projection of a vector y to a subspace m W R . Let x1,...,xn be a basis of W and A =[x1 xn]. ⊂ { } |···| T 1 T (a) Use projection matrix to find ProjW y = PW y, where PW =(A A)− A . (b) Apply Gram-Schmidt on the basis x ,...,x to obtain an orthogonal basis { 1 n} u1,...,un , then ProjW y can be write explicitly under the orthogonal (orthonor- mal){ basis: } y u y u Proj y = · 1 u + + · n u . W u u 1 ··· u u n 1 · 1 n · n (c) There is a unique decomposition: for y Rm, ∈ y =ProjW y + z, where z W ⊥. ∈ 14. Inconsistent system of linear equations, least squares (a) Least-squares solution xˆ of Ax = b is a vector such that b Axˆ b Ax . ( − (≤( − ( The solution is actually ProjCol Ab, that is, T 1 T xˆ =(A A)− A b, Note here we require columns of A are linearly independent. (b) Line of least square is a special case of (a), see page 369 for the detail. 4.
Recommended publications
  • Graph Equivalence Classes for Spectral Projector-Based Graph Fourier Transforms Joya A
    1 Graph Equivalence Classes for Spectral Projector-Based Graph Fourier Transforms Joya A. Deri, Member, IEEE, and José M. F. Moura, Fellow, IEEE Abstract—We define and discuss the utility of two equiv- Consider a graph G = G(A) with adjacency matrix alence graph classes over which a spectral projector-based A 2 CN×N with k ≤ N distinct eigenvalues and Jordan graph Fourier transform is equivalent: isomorphic equiv- decomposition A = VJV −1. The associated Jordan alence classes and Jordan equivalence classes. Isomorphic equivalence classes show that the transform is equivalent subspaces of A are Jij, i = 1; : : : k, j = 1; : : : ; gi, up to a permutation on the node labels. Jordan equivalence where gi is the geometric multiplicity of eigenvalue 휆i, classes permit identical transforms over graphs of noniden- or the dimension of the kernel of A − 휆iI. The signal tical topologies and allow a basis-invariant characterization space S can be uniquely decomposed by the Jordan of total variation orderings of the spectral components. subspaces (see [13], [14] and Section II). For a graph Methods to exploit these classes to reduce computation time of the transform as well as limitations are discussed. signal s 2 S, the graph Fourier transform (GFT) of [12] is defined as Index Terms—Jordan decomposition, generalized k gi eigenspaces, directed graphs, graph equivalence classes, M M graph isomorphism, signal processing on graphs, networks F : S! Jij i=1 j=1 s ! (s ;:::; s ;:::; s ;:::; s ) ; (1) b11 b1g1 bk1 bkgk I. INTRODUCTION where sij is the (oblique) projection of s onto the Jordan subspace Jij parallel to SnJij.
    [Show full text]
  • EUCLIDEAN DISTANCE MATRIX COMPLETION PROBLEMS June 6
    EUCLIDEAN DISTANCE MATRIX COMPLETION PROBLEMS HAW-REN FANG∗ AND DIANNE P. O’LEARY† June 6, 2010 Abstract. A Euclidean distance matrix is one in which the (i, j) entry specifies the squared distance between particle i and particle j. Given a partially-specified symmetric matrix A with zero diagonal, the Euclidean distance matrix completion problem (EDMCP) is to determine the unspecified entries to make A a Euclidean distance matrix. We survey three different approaches to solving the EDMCP. We advocate expressing the EDMCP as a nonconvex optimization problem using the particle positions as variables and solving using a modified Newton or quasi-Newton method. To avoid local minima, we develop a randomized initial- ization technique that involves a nonlinear version of the classical multidimensional scaling, and a dimensionality relaxation scheme with optional weighting. Our experiments show that the method easily solves the artificial problems introduced by Mor´e and Wu. It also solves the 12 much more difficult protein fragment problems introduced by Hen- drickson, and the 6 larger protein problems introduced by Grooms, Lewis, and Trosset. Key words. distance geometry, Euclidean distance matrices, global optimization, dimensional- ity relaxation, modified Cholesky factorizations, molecular conformation AMS subject classifications. 49M15, 65K05, 90C26, 92E10 1. Introduction. Given the distances between each pair of n particles in Rr, n r, it is easy to determine the relative positions of the particles. In many applications,≥ though, we are given only some of the distances and we would like to determine the missing distances and thus the particle positions. We focus in this paper on algorithms to solve this distance completion problem.
    [Show full text]
  • Calculus Terminology
    AP Calculus BC Calculus Terminology Absolute Convergence Asymptote Continued Sum Absolute Maximum Average Rate of Change Continuous Function Absolute Minimum Average Value of a Function Continuously Differentiable Function Absolutely Convergent Axis of Rotation Converge Acceleration Boundary Value Problem Converge Absolutely Alternating Series Bounded Function Converge Conditionally Alternating Series Remainder Bounded Sequence Convergence Tests Alternating Series Test Bounds of Integration Convergent Sequence Analytic Methods Calculus Convergent Series Annulus Cartesian Form Critical Number Antiderivative of a Function Cavalieri’s Principle Critical Point Approximation by Differentials Center of Mass Formula Critical Value Arc Length of a Curve Centroid Curly d Area below a Curve Chain Rule Curve Area between Curves Comparison Test Curve Sketching Area of an Ellipse Concave Cusp Area of a Parabolic Segment Concave Down Cylindrical Shell Method Area under a Curve Concave Up Decreasing Function Area Using Parametric Equations Conditional Convergence Definite Integral Area Using Polar Coordinates Constant Term Definite Integral Rules Degenerate Divergent Series Function Operations Del Operator e Fundamental Theorem of Calculus Deleted Neighborhood Ellipsoid GLB Derivative End Behavior Global Maximum Derivative of a Power Series Essential Discontinuity Global Minimum Derivative Rules Explicit Differentiation Golden Spiral Difference Quotient Explicit Function Graphic Methods Differentiable Exponential Decay Greatest Lower Bound Differential
    [Show full text]
  • History of Algebra and Its Implications for Teaching
    Maggio: History of Algebra and its Implications for Teaching History of Algebra and its Implications for Teaching Jaime Maggio Fall 2020 MA 398 Senior Seminar Mentor: Dr.Loth Published by DigitalCommons@SHU, 2021 1 Academic Festival, Event 31 [2021] Abstract Algebra can be described as a branch of mathematics concerned with finding the values of unknown quantities (letters and other general sym- bols) defined by the equations that they satisfy. Algebraic problems have survived in mathematical writings of the Egyptians and Babylonians. The ancient Greeks also contributed to the development of algebraic concepts. In this paper, we will discuss historically famous mathematicians from all over the world along with their key mathematical contributions. Mathe- matical proofs of ancient and modern discoveries will be presented. We will then consider the impacts of incorporating history into the teaching of mathematics courses as an educational technique. 1 https://digitalcommons.sacredheart.edu/acadfest/2021/all/31 2 Maggio: History of Algebra and its Implications for Teaching 1 Introduction In order to understand the way algebra is the way it is today, it is important to understand how it came about starting with its ancient origins. In a mod- ern sense, algebra can be described as a branch of mathematics concerned with finding the values of unknown quantities defined by the equations that they sat- isfy. Algebraic problems have survived in mathematical writings of the Egyp- tians and Babylonians. The ancient Greeks also contributed to the development of algebraic concepts, but these concepts had a heavier focus on geometry [1]. The combination of all of the discoveries of these great mathematicians shaped the way algebra is taught today.
    [Show full text]
  • Cramer's Rule 1 Cramer's Rule
    Cramer's rule 1 Cramer's rule In linear algebra, Cramer's rule is an explicit formula for the solution of a system of linear equations with as many equations as unknowns, valid whenever the system has a unique solution. It expresses the solution in terms of the determinants of the (square) coefficient matrix and of matrices obtained from it by replacing one column by the vector of right hand sides of the equations. It is named after Gabriel Cramer (1704–1752), who published the rule for an arbitrary number of unknowns in 1750[1], although Colin Maclaurin also published special cases of the rule in 1748[2] (and probably knew of it as early as 1729).[3][4][5] General case Consider a system of n linear equations for n unknowns, represented in matrix multiplication form as follows: where the n by n matrix has a nonzero determinant, and the vector is the column vector of the variables. Then the theorem states that in this case the system has a unique solution, whose individual values for the unknowns are given by: where is the matrix formed by replacing the ith column of by the column vector . The rule holds for systems of equations with coefficients and unknowns in any field, not just in the real numbers. It has recently been shown that Cramer's rule can be implemented in O(n3) time,[6] which is comparable to more common methods of solving systems of linear equations, such as Gaussian elimination. Proof The proof for Cramer's rule uses just two properties of determinants: linearity with respect to any given column (taking for that column a linear combination of column vectors produces as determinant the corresponding linear combination of their determinants), and the fact that the determinant is zero whenever two columns are equal (the determinant is alternating in the columns).
    [Show full text]
  • Dimensionality Reduction
    CHAPTER 7 Dimensionality Reduction We saw in Chapter 6 that high-dimensional data has some peculiar characteristics, some of which are counterintuitive. For example, in high dimensions the center of the space is devoid of points, with most of the points being scattered along the surface of the space or in the corners. There is also an apparent proliferation of orthogonal axes. As a consequence high-dimensional data can cause problems for data mining and analysis, although in some cases high-dimensionality can help, for example, for nonlinear classification. Nevertheless, it is important to check whether the dimensionality can be reduced while preserving the essential properties of the full data matrix. This can aid data visualization as well as data mining. In this chapter we study methods that allow us to obtain optimal lower-dimensional projections of the data. 7.1 BACKGROUND Let the data D consist of n points over d attributes, that is, it is an n × d matrix, given as ⎛ ⎞ X X ··· Xd ⎜ 1 2 ⎟ ⎜ x x ··· x ⎟ ⎜x1 11 12 1d ⎟ ⎜ x x ··· x ⎟ D =⎜x2 21 22 2d ⎟ ⎜ . ⎟ ⎝ . .. ⎠ xn xn1 xn2 ··· xnd T Each point xi = (xi1,xi2,...,xid) is a vector in the ambient d-dimensional vector space spanned by the d standard basis vectors e1,e2,...,ed ,whereei corresponds to the ith attribute Xi . Recall that the standard basis is an orthonormal basis for the data T space, that is, the basis vectors are pairwise orthogonal, ei ej = 0, and have unit length ei = 1. T As such, given any other set of d orthonormal vectors u1,u2,...,ud ,withui uj = 0 T and ui = 1(orui ui = 1), we can re-express each point x as the linear combination x = a1u1 + a2u2 +···+adud (7.1) 183 184 Dimensionality Reduction T where the vector a = (a1,a2,...,ad ) represents the coordinates of x in the new basis.
    [Show full text]
  • Linear Algebra
    Linear Algebra July 28, 2006 1 Introduction These notes are intended for use in the warm-up camp for incoming Berkeley Statistics graduate students. Welcome to Cal! We assume that you have taken a linear algebra course before and that most of the material in these notes will be a review of what you already know. If you have never taken such a course before, you are strongly encouraged to do so by taking math 110 (or the honors version of it), or by covering material presented and/or mentioned here on your own. If some of the material is unfamiliar, do not be intimidated! We hope you find these notes helpful! If not, you can consult the references listed at the end, or any other textbooks of your choice for more information or another style of presentation (most of the proofs on linear algebra part have been adopted from Strang, the proof of F-test from Montgomery et al, and the proof of bivariate normal density from Bickel and Doksum). Go Bears! 1 2 Vector Spaces A set V is a vector space over R and its elements are called vectors if there are 2 opera- tions defined on it: 1. Vector addition, that assigns to each pair of vectors v ; v V another vector w V 1 2 2 2 (we write v1 + v2 = w) 2. Scalar multiplication, that assigns to each vector v V and each scalar r R another 2 2 vector w V (we write rv = w) 2 that satisfy the following 8 conditions v ; v ; v V and r ; r R: 8 1 2 3 2 8 1 2 2 1.
    [Show full text]
  • The Evolution of Equation-Solving: Linear, Quadratic, and Cubic
    California State University, San Bernardino CSUSB ScholarWorks Theses Digitization Project John M. Pfau Library 2006 The evolution of equation-solving: Linear, quadratic, and cubic Annabelle Louise Porter Follow this and additional works at: https://scholarworks.lib.csusb.edu/etd-project Part of the Mathematics Commons Recommended Citation Porter, Annabelle Louise, "The evolution of equation-solving: Linear, quadratic, and cubic" (2006). Theses Digitization Project. 3069. https://scholarworks.lib.csusb.edu/etd-project/3069 This Thesis is brought to you for free and open access by the John M. Pfau Library at CSUSB ScholarWorks. It has been accepted for inclusion in Theses Digitization Project by an authorized administrator of CSUSB ScholarWorks. For more information, please contact [email protected]. THE EVOLUTION OF EQUATION-SOLVING LINEAR, QUADRATIC, AND CUBIC A Project Presented to the Faculty of California State University, San Bernardino In Partial Fulfillment of the Requirements for the Degre Master of Arts in Teaching: Mathematics by Annabelle Louise Porter June 2006 THE EVOLUTION OF EQUATION-SOLVING: LINEAR, QUADRATIC, AND CUBIC A Project Presented to the Faculty of California State University, San Bernardino by Annabelle Louise Porter June 2006 Approved by: Shawnee McMurran, Committee Chair Date Laura Wallace, Committee Member , (Committee Member Peter Williams, Chair Davida Fischman Department of Mathematics MAT Coordinator Department of Mathematics ABSTRACT Algebra and algebraic thinking have been cornerstones of problem solving in many different cultures over time. Since ancient times, algebra has been used and developed in cultures around the world, and has undergone quite a bit of transformation. This paper is intended as a professional developmental tool to help secondary algebra teachers understand the concepts underlying the algorithms we use, how these algorithms developed, and why they work.
    [Show full text]
  • Linear Algebra with Exercises B
    Linear Algebra with Exercises B Fall 2017 Kyoto University Ivan Ip These notes summarize the definitions, theorems and some examples discussed in class. Please refer to the class notes and reference books for proofs and more in-depth discussions. Contents 1 Abstract Vector Spaces 1 1.1 Vector Spaces . .1 1.2 Subspaces . .3 1.3 Linearly Independent Sets . .4 1.4 Bases . .5 1.5 Dimensions . .7 1.6 Intersections, Sums and Direct Sums . .9 2 Linear Transformations and Matrices 11 2.1 Linear Transformations . 11 2.2 Injection, Surjection and Isomorphism . 13 2.3 Rank . 14 2.4 Change of Basis . 15 3 Euclidean Space 17 3.1 Inner Product . 17 3.2 Orthogonal Basis . 20 3.3 Orthogonal Projection . 21 i 3.4 Orthogonal Matrix . 24 3.5 Gram-Schmidt Process . 25 3.6 Least Square Approximation . 28 4 Eigenvectors and Eigenvalues 31 4.1 Eigenvectors . 31 4.2 Determinants . 33 4.3 Characteristic polynomial . 36 4.4 Similarity . 38 5 Diagonalization 41 5.1 Diagonalization . 41 5.2 Symmetric Matrices . 44 5.3 Minimal Polynomials . 46 5.4 Jordan Canonical Form . 48 5.5 Positive definite matrix (Optional) . 52 5.6 Singular Value Decomposition (Optional) . 54 A Complex Matrix 59 ii Introduction Real life problems are hard. Linear Algebra is easy (in the mathematical sense). We make linear approximations to real life problems, and reduce the problems to systems of linear equations where we can then use the techniques from Linear Algebra to solve for approximate solutions. Linear Algebra also gives new insights and tools to the original problems.
    [Show full text]
  • Mathematics for Earth Science
    Mathematics for Earth Science The module covers concepts such as: • Maths refresher • Fractions, Percentage and Ratios • Unit conversions • Calculating large and small numbers • Logarithms • Trigonometry • Linear relationships Mathematics for Earth Science Contents 1. Terms and Operations 2. Fractions 3. Converting decimals and fractions 4. Percentages 5. Ratios 6. Algebra Refresh 7. Power Operations 8. Scientific Notation 9. Units and Conversions 10. Logarithms 11. Trigonometry 12. Graphs and linear relationships 13. Answers 1. Terms and Operations Glossary , 2 , 3 & 17 are TERMS x4 + 2 + 3 = 17 is an EQUATION 17 is the SUM of + 2 + 3 4 4 4 is an EXPONENT + 2 + 3 = 17 4 3 is a CONSTANT 2 is a COEFFICIENT is a VARIABLE + is an OPERATOR +2 + 3 is an EXPRESSION 4 Equation: A mathematical sentence containing an equal sign. The equal sign demands that the expressions on either side are balanced and equal. Expression: An algebraic expression involves numbers, operation signs, brackets/parenthesis and variables that substitute numbers but does not include an equal sign. Operator: The operation (+ , ,× ,÷) which separates the terms. Term: Parts of an expression− separated by operators which could be a number, variable or product of numbers and variables. Eg. 2 , 3 & 17 Variable: A letter which represents an unknown number. Most common is , but can be any symbol. Constant: Terms that contain only numbers that always have the same value. Coefficient: A number that is partnered with a variable. The term 2 is a coefficient with variable. Between the coefficient and variable is a multiplication. Coefficients of 1 are not shown. Exponent: A value or base that is multiplied by itself a certain number of times.
    [Show full text]
  • Algebra Vocabulary List (Definitions for Middle School Teachers)
    Algebra Vocabulary List (Definitions for Middle School Teachers) A Absolute Value Function – The absolute value of a real number x, x is ⎧ xifx≥ 0 x = ⎨ ⎩−<xifx 0 http://www.math.tamu.edu/~stecher/171/F02/absoluteValueFunction.pdf Algebra Lab Gear – a set of manipulatives that are designed to represent polynomial expressions. The set includes representations for positive/negative 1, 5, 25, x, 5x, y, 5y, xy, x2, y2, x3, y3, x2y, xy2. The manipulatives can be used to model addition, subtraction, multiplication, division, and factoring of polynomials. They can also be used to model how to solve linear equations. o For more info: http://www.stlcc.cc.mo.us/mcdocs/dept/math/homl/manip.htm http://www.awl.ca/school/math/mr/alg/ss/series/algsrtxt.html http://www.picciotto.org/math-ed/manipulatives/lab-gear.html Algebra Tiles – a set of manipulatives that are designed for modeling algebraic expressions visually. Each tile is a geometric model of a term. The set includes representations for positive/negative 1, x, and x2. The manipulatives can be used to model addition, subtraction, multiplication, division, and factoring of polynomials. They can also be used to model how to solve linear equations. o For more info: http://math.buffalostate.edu/~it/Materials/MatLinks/tilelinks.html http://plato.acadiau.ca/courses/educ/reid/Virtual-manipulatives/tiles/tiles.html Algebraic Expression – a number, variable, or combination of the two connected by some mathematical operation like addition, subtraction, multiplication, division, exponents and/or roots. o For more info: http://www.wtamu.edu/academic/anns/mps/math/mathlab/beg_algebra/beg_alg_tut11 _simp.htm http://www.math.com/school/subject2/lessons/S2U1L1DP.html Area Under the Curve – suppose the curve y=f(x) lies above the x axis for all x in [a, b].
    [Show full text]
  • Polynomial Coefficient Enumeration
    Polynomial Coefficient Enumeration Tewodros Amdeberhan∗ Richard P. Stanley† February 03, 2008 Abstract Let f(x1, . , xk) be a polynomial over a field K. This paper con- siders such questions as the enumeration of the number of nonzero coefficients of ∗ f or of the number of coefficients equal to α ∈ K . For instance, if K = Fq then a matrix formula is obtained for the number of coefficients of f n that are equal to ∗ α ∈ Fq, as a function of n. Many additional results are obtained related to such areas as lattice path enumeration and the enumeration of integer points in convex polytopes. 1 Introduction. Given a polynomial f ∈ Z[x1, . , xn], how many coefficients of f are nonzero? For α ∈ Z and p prime, how many coefficients are congruent to α modulo p? In this paper we will investigate these and related questions. First let us review some known results that will suggest various generalizations. The archetypal result for understanding the coefficients of a polynomial modulo p is Lucas’ theorem [7]: if n, k are positive integers with p-ary ex- P i P i pansions n = i≥0 aip and k = i≥0 bip , then n a a ≡ 0 1 ··· (mod p). k b0 b1 ∗Department of Mathematics, Massachusetts Institute of Technology, Cambridge, MA 02139 Email: [email protected] †Department of Mathematics, Massachusetts Institute of Technology, Cambridge, MA 02139 Email: [email protected] 1 Thus, for instance, it immediately follows that the number of odd coefficients of the polynomial (1 + x)n is equal to 2s(n), where s(n) denotes the number of 1’s in the 2-ary (binary) expansion of n.
    [Show full text]