DOCSLIB.ORG

Matrix Notation and Operations

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Matrix Notation and Operations D.G. Bonett (6/2018) Matrix Notation and Operations Matrix Notation An r × c matrix is a rectangular array of elements with r rows and c columns. An r × c matrix is said to be of order r × c. A matrix is usually denoted by a capital letter printed in a boldface font (e.g., A, B, X). The elements of the matrix are represented by lower case letters with a double subscript (e.g., 푎푗푘, 푏푗푘, 푥푗푘). For instance, in the matrix X, 푥13 is the element in the first row and the third column. A 4 × 3 matrix X is shown below. 푥11 푥12 푥13 푥 푥 푥 X = [ 21 22 23] 푥31 푥32 푥33 푥41 푥42 푥43 A matrix with a single row is called a row vector and a matrix with a single column is called a column vector. Vectors are usually represented by lower case letters printed in a boldface font (e.g., a, b, x). The elements of the vector are represented by lower case letters with a single subscript (e.g., 푎푗, 푏푗, 푥푗). A 3 × 1 column vector y and a 1 × 4 row vector h are shown below. 푦1 y = [푦2] 푦3 h = [ℎ1 ℎ2 ℎ3 ℎ4 ]. It is sometimes necessary to refer to a particular row or column of a matrix. These row or column vectors are represented by a subscripted lower case letter in a boldface font. For th instance, the j row vector in the above 4 × 3 matrix X would be noted as 퐱푗. A square matrix has the same number of rows as columns. A square matrix where the diagonal elements (the elements where the two subscripts are equal) are nonzero and the off-diagonal elements are zero is called a diagonal matrix. A 3 × 3 diagonal matrix D is shown below. 1 D.G. Bonett (6/2018) 푑11 0 0 D = [ 0 푑22 0 ] 0 0 푑33 The identity matrix is a special type of diagonal matrix where all diagonal elements are equal to 1. The identity matrix is usually represented as I or In where n is the order of the identity matrix. A square matrix where the jkth element is equal to the kjth element is called a symmetric matrix. A symmetric 3 × 3 matrix is shown below. 14 5 2 S = [ 5 20 8 ] 2 8 11 A one vector is a row or column vector in which every element is equal to 1 and is represented as the number one printed in a boldface font. A 1 × 3 one vector is shown below. 1 = [1 1 1] Matrix Operations The transpose of a matrix X is represented as X' (or XT). The transpose of a matrix is obtained by interchanging the rows and columns of the matrix. For instance, if 4 6 X = [7 1] 3 9 then 4 7 3 X' = [ ] . 6 1 9 Note that the jkth element in X is equal to the kjth element in X'. Most vectors in statistical formulas are assumed to be column vectors. Row vectors, when needed, are obtained by taking the transpose of a column vector. 2 D.G. Bonett (6/2018) If two matrices A and B are of the same order, the two matrices are then conformable for th addition or subtraction, and A + B is a matrix with element 푎푗푘 + 푏푗푘 in the j row and the kth column, as illustrated below for the sum of two 3 × 2 matrices. 푎11 푎12 푎13 푏 푏 푏 A + B = [ ] + [ 11 12 13] 푎21 푎22 푎23 푏21 푏22 푏23 푎11 + 푏11 푎12 + 푏12 푎13 + 푏13 = [ ] 푎21 + 푎11 푎22 + 푏22 푎23 + 푏23 th th Likewise, A – B is a matrix with element 푎푗푘 – 푏푗푘 in the j row and the k column, as illustrated below for the difference of two 3 × 2 matrices. 푎 푎 푎 푏 푏 푏 A – B = [ 11 12 13] – [ 11 12 13] 푎21 푎22 푎23 푏21 푏22 푏23 푎 − 푏 푎 − 푏 푎 − 푏 = [ 11 11 12 12 13 13] 푎21 − 푎11 푎22 − 푏22 푎23 − 푏23 To multiply a matrix by a scalar (i.e., a single number), simply multiply each element in the matrix by the scalar. Scalars are represented by italicized lower case letters in a non- boldface font. To illustrate, if b = 2 and 4 7 3 A = [ ] 6 1 9 then 8 14 6 bA = [ ]. 12 2 18 Some statistical formulas involve the subtraction of a scalar from an n x 1 vector. The result is obtained by first multiplying the scalar by an n x 1 one vector and then taking the difference of the two vectors. For instance, if y is a 3 × 1 vector and m is a scalar, then y – m is 푦1 1 푦1 − 푚 y – m1 = [푦2] − 푚 [1] = [푦2 − 푚] 푦3 1 푦3 − 푚 3 D.G. Bonett (6/2018) The dot product of a n × 1 vector a with a n × 1 vector b is a'b = 푎1푏1 + 푎2푏2 + … + 푎푛푏푛 Note that a'b = b'a. For instance, if a' = [4 3 2] and b' = [6 1 4], then a'b = 4(6) + 3(1) + 2(4) = 35. Two n × 1 vectors, a and b, are said to be orthogonal if a'b = 0. For instance, if a' = [.5 .5 -1] and b' = [1 -1 0], then a and b are orthogonal because 퐚'b = (.5)(1) + (.5)(-1) + (-1)(0) = 0. Two matrices A and B can be multiplied if they are conformable for multiplication. To compute the matrix product AB, the number of columns of A must equal the number of rows of B. In general, if A is r × n and B is n × c, then the matrix product AB is an r × c th matrix. The jk element in the r × c product matrix is equal to the dot product 퐚푗퐛푘 where th th 퐚푗 is the j row vector of matrix A and 퐛푘 is the k column vector of matrix B. For instance, the matrices A and B shown below are conformable for computing the product AB because A is 2 × 3 and B is 3 × 4 so that the product will be a 2 × 4 matrix. 1 2 1 4 4 7 3 A = [ ] B = [5 4 3 1] 6 1 9 4 2 3 2 Each of the 2 × 4 = 8 elements of the AB matrix is a dot product. For instance, the element in row 1 and column 1 of the product AB is 1 [4 7 3] [5] = 4(1) + 7(5) + 3(4) = 51 4 and the element in row 2 and column 3 of AB is 1 [6 1 9] [3] = 6(1) + 1(3) + 9(3) = 36. 3 After computing all 8 dot products, the following result is obtained. 51 42 34 29 AB = [ ] 47 34 36 43 4 D.G. Bonett (6/2018) Unlike scalar multiplication where ab = ba, the matrix product AB does not in general equal BA. Regarding the matrix product AB, we can say that B is pre-multiplied by A or that A is post-multiplied by B. The product of matrix A with itself is denoted as A2. The transpose of a matrix product is equal to the product of the transposed matrices in reverse order. Specifically, (AB)' = B'A'. The product of three matrices ABC requires A and B to be conformable for multiplication and also requires B and C to be conformable for multiplication. The product ABC can be obtained by first computing AB and then post-multiplying the result by C, or by first computing BC and then pre-multiplying the result by A. If A is a square matrix, then the matrix inverse of A is represented as A-1. If the inverse of A exists, then AA-1 = I. This result is a generalization of scalar arithmetic where x(1/x) = 1, assuming x 0 so that the inverse of x exists. Computing a matrix inverse is tedious, and the amount of computational effort increases as the size of the matrix increases, but inverting a 2 × 2 matrix is not difficult. The inverse of a 2 × 2 matrix A is 푎 −푎 -1 22 12 A = (1/d)[ ] −푎21 푎11 where d = 푎11푎22 − 푎12푎21 is called the determinant of A. The matrix inverse does not exist unless the determinant is nonzero. Inverting a diagonal matrix D of any order is simple. The inverse of D is equal to a th diagonal matrix where the j diagonal element is equal to 1/푑푗 . The trace of a square n × n matrix A, denoted as tr(A), is defined as the sum of its n diagonal elements. tr(A) = 푎11 + 푎22 + … + 푎푛푛 5 D.G. Bonett (6/2018) For instance if 14 5 2 V = [ 9 20 8 ] 7 8 11 then tr(V) = 14 + 20 + 11 = 45. The Kronecker product of two matrices, an m × n matrix A and a p × q matrix B, is defined to be the mp × nq matrix 푎11퐁 푎12퐁 ⋯ 푎1푛퐁 푎 퐁 푎 퐁 … 푎 퐁 ⨂ [ 21 22 2푛 ] A B = ⋮ ⋮ ⋮ 푎푚1퐁 푎푚2퐁 ⋯ 푎푚푛퐁 which is obtained by replacing each element 푎푗푘 with the p × q matrix 푎푗푘B. For example, if 1 2 A = [ ] and b = [1 2 3] 3 −1 then 1 2 3 2 4 6 A ⨂ b = [ ]. 3 6 9 −1 −2 −3 The Kronecker product of an identity matrix and another matrix has the following simple form 퐁 ퟎ ⋯ ퟎ ퟎ 퐁 … ퟎ I ⨂ B = [ ] ⋮ ⋮ ⋮ ퟎ ퟎ ⋯ 퐁 where ퟎ is a matrix of zeros that has the same order as B.
Load more

Recommended publications
  • Common Course Outline MATH 257 Linear Algebra 4 Credits
    Common Course Outline MATH 257 Linear Algebra 4 Credits The Community College of Baltimore County Description MATH 257 – Linear Algebra is one of the suggested elective courses for students majoring in Mathematics, Computer Science or Engineering. Included are geometric vectors, matrices, systems of linear equations, vector spaces, linear transformations, determinants, eigenvectors and inner product spaces. 4 Credits: 5 lecture hours Prerequisite: MATH 251 with a grade of “C” or better Overall Course Objectives Upon successfully completing the course, students will be able to: 1. perform matrix operations; 2. use Gaussian Elimination, Cramer’s Rule, and the inverse of the coefficient matrix to solve systems of Linear Equations; 3. find the inverse of a matrix by Gaussian Elimination or using the adjoint matrix; 4. compute the determinant of a matrix using cofactor expansion or elementary row operations; 5. apply Gaussian Elimination to solve problems concerning Markov Chains; 6. verify that a structure is a vector space by checking the axioms; 7. verify that a subset is a subspace and that a set of vectors is a basis; 8. compute the dimensions of subspaces; 9. compute the matrix representation of a linear transformation; 10. apply notions of linear transformations to discuss rotations and reflections of two dimensional space; 11. compute eigenvalues and find their corresponding eigenvectors; 12. diagonalize a matrix using eigenvalues; 13. apply properties of vectors and dot product to prove results in geometry; 14. apply notions of vectors, dot product and matrices to construct a best fitting curve; 15. construct a solution to real world problems using problem methods individually and in groups; 16.
    [Show full text]
  • Parametrizations of K-Nonnegative Matrices
    Parametrizations of k-Nonnegative Matrices Anna Brosowsky, Neeraja Kulkarni, Alex Mason, Joe Suk, Ewin Tang∗ October 2, 2017 Abstract Totally nonnegative (positive) matrices are matrices whose minors are all nonnegative (positive). We generalize the notion of total nonnegativity, as follows. A k-nonnegative (resp. k-positive) matrix has all minors of size k or less nonnegative (resp. positive). We give a generating set for the semigroup of k-nonnegative matrices, as well as relations for certain special cases, i.e. the k = n − 1 and k = n − 2 unitriangular cases. In the above two cases, we find that the set of k-nonnegative matrices can be partitioned into cells, analogous to the Bruhat cells of totally nonnegative matrices, based on their factorizations into generators. We will show that these cells, like the Bruhat cells, are homeomorphic to open balls, and we prove some results about the topological structure of the closure of these cells, and in fact, in the latter case, the cells form a Bruhat-like CW complex. We also give a family of minimal k-positivity tests which form sub-cluster algebras of the total positivity test cluster algebra. We describe ways to jump between these tests, and give an alternate description of some tests as double wiring diagrams. 1 Introduction A totally nonnegative (respectively totally positive) matrix is a matrix whose minors are all nonnegative (respectively positive). Total positivity and nonnegativity are well-studied phenomena and arise in areas such as planar networks, combinatorics, dynamics, statistics and probability. The study of total positivity and total nonnegativity admit many varied applications, some of which are explored in “Totally Nonnegative Matrices” by Fallat and Johnson [5].
    [Show full text]
  • DERIVATIONS and PROJECTIONS on JORDAN TRIPLES an Introduction to Nonassociative Algebra, Continuous Cohomology, and Quantum Functional Analysis
    DERIVATIONS AND PROJECTIONS ON JORDAN TRIPLES An introduction to nonassociative algebra, continuous cohomology, and quantum functional analysis Bernard Russo July 29, 2014 This paper is an elaborated version of the material presented by the author in a three hour minicourse at V International Course of Mathematical Analysis in Andalusia, at Almeria, Spain September 12-16, 2011. The author wishes to thank the scientific committee for the opportunity to present the course and to the organizing committee for their hospitality. The author also personally thanks Antonio Peralta for his collegiality and encouragement. The minicourse on which this paper is based had its genesis in a series of talks the author had given to undergraduates at Fullerton College in California. I thank my former student Dana Clahane for his initiative in running the remarkable undergraduate research program at Fullerton College of which the seminar series is a part. With their knowledge only of the product rule for differentiation as a starting point, these enthusiastic students were introduced to some aspects of the esoteric subject of non associative algebra, including triple systems as well as algebras. Slides of these talks and of the minicourse lectures, as well as other related material, can be found at the author's website (www.math.uci.edu/∼brusso). Conversely, these undergraduate talks were motivated by the author's past and recent joint works on derivations of Jordan triples ([116],[117],[200]), which are among the many results discussed here. Part I (Derivations) is devoted to an exposition of the properties of derivations on various algebras and triple systems in finite and infinite dimensions, the primary questions addressed being whether the derivation is automatically continuous and to what extent it is an inner derivation.
    [Show full text]
  • Diagonalizing a Matrix
    Diagonalizing a Matrix Definition 1. We say that two square matrices A and B are similar provided there exists an invertible matrix P so that . 2. We say a matrix A is diagonalizable if it is similar to a diagonal matrix. Example 1. The matrices and are similar matrices since . We conclude that is diagonalizable. 2. The matrices and are similar matrices since . After we have developed some additional theory, we will be able to conclude that the matrices and are not diagonalizable. Theorem Suppose A, B and C are square matrices. (1) A is similar to A. (2) If A is similar to B, then B is similar to A. (3) If A is similar to B and if B is similar to C, then A is similar to C. Proof of (3) Since A is similar to B, there exists an invertible matrix P so that . Also, since B is similar to C, there exists an invertible matrix R so that . Now, and so A is similar to C. Thus, “A is similar to B” is an equivalence relation. Theorem If A is similar to B, then A and B have the same eigenvalues. Proof Since A is similar to B, there exists an invertible matrix P so that . Now, Since A and B have the same characteristic equation, they have the same eigenvalues. > Example Find the eigenvalues for . Solution Since is similar to the diagonal matrix , they have the same eigenvalues. Because the eigenvalues of an upper (or lower) triangular matrix are the entries on the main diagonal, we see that the eigenvalues for , and, hence, are .
    [Show full text]
  • 21. Orthonormal Bases
    21. Orthonormal Bases The canonical/standard basis 011 001 001 B C B C B C B0C B1C B0C e1 = B.C ; e2 = B.C ; : : : ; en = B.C B.C B.C B.C @.A @.A @.A 0 0 1 has many useful properties. • Each of the standard basis vectors has unit length: q p T jjeijj = ei ei = ei ei = 1: • The standard basis vectors are orthogonal (in other words, at right angles or perpendicular). T ei ej = ei ej = 0 when i 6= j This is summarized by ( 1 i = j eT e = δ = ; i j ij 0 i 6= j where δij is the Kronecker delta. Notice that the Kronecker delta gives the entries of the identity matrix. Given column vectors v and w, we have seen that the dot product v w is the same as the matrix multiplication vT w. This is the inner product on n T R . We can also form the outer product vw , which gives a square matrix. 1 The outer product on the standard basis vectors is interesting. Set T Π1 = e1e1 011 B C B0C = B.C 1 0 ::: 0 B.C @.A 0 01 0 ::: 01 B C B0 0 ::: 0C = B. .C B. .C @. .A 0 0 ::: 0 . T Πn = enen 001 B C B0C = B.C 0 0 ::: 1 B.C @.A 1 00 0 ::: 01 B C B0 0 ::: 0C = B. .C B. .C @. .A 0 0 ::: 1 In short, Πi is the diagonal square matrix with a 1 in the ith diagonal position and zeros everywhere else.
    [Show full text]
  • Partitioned (Or Block) Matrices This Version: 29 Nov 2018
    Partitioned (or Block) Matrices This version: 29 Nov 2018 Intermediate Econometrics / Forecasting Class Notes Instructor: Anthony Tay It is frequently convenient to partition matrices into smaller sub-matrices. e.g. 2 3 2 1 3 2 3 2 1 3 4 1 1 0 7 4 1 1 0 7 A B (2×2) (2×3) 3 1 1 0 0 = 3 1 1 0 0 = C I 1 3 0 1 0 1 3 0 1 0 (3×2) (3×3) 2 0 0 0 1 2 0 0 0 1 The same matrix can be partitioned in several different ways. For instance, we can write the previous matrix as 2 3 2 1 3 2 3 2 1 3 4 1 1 0 7 4 1 1 0 7 a b0 (1×1) (1×4) 3 1 1 0 0 = 3 1 1 0 0 = c D 1 3 0 1 0 1 3 0 1 0 (4×1) (4×4) 2 0 0 0 1 2 0 0 0 1 One reason partitioning is useful is that we can do matrix addition and multiplication with blocks, as though the blocks are elements, as long as the blocks are conformable for the operations. For instance: A B D E A + D B + E (2×2) (2×3) (2×2) (2×3) (2×2) (2×3) + = C I C F 2C I + F (3×2) (3×3) (3×2) (3×3) (3×2) (3×3) A B d E Ad + BF AE + BG (2×2) (2×3) (2×1) (2×3) (2×1) (2×3) = C I F G Cd + F CE + G (3×2) (3×3) (3×1) (3×3) (3×1) (3×3) | {z } | {z } | {z } (5×5) (5×4) (5×4) 1 Intermediate Econometrics / Forecasting 2 Examples (1) Let 1 2 1 1 2 1 c 1 4 2 3 4 2 3 h i A = = = a a a and c = c 1 2 3 2 3 0 1 3 0 1 c 0 1 3 0 1 3 3 c1 h i then Ac = a1 a2 a3 c2 = c1a1 + c2a2 + c3a3 c3 The product Ac produces a linear combination of the columns of A.
    [Show full text]
  • Analysis of Functions of a Single Variable a Detailed Development
    ANALYSIS OF FUNCTIONS OF A SINGLE VARIABLE A DETAILED DEVELOPMENT LAWRENCE W. BAGGETT University of Colorado OCTOBER 29, 2006 2 For Christy My Light i PREFACE I have written this book primarily for serious and talented mathematics scholars , seniors or first-year graduate students, who by the time they finish their schooling should have had the opportunity to study in some detail the great discoveries of our subject. What did we know and how and when did we know it? I hope this book is useful toward that goal, especially when it comes to the great achievements of that part of mathematics known as analysis. I have tried to write a complete and thorough account of the elementary theories of functions of a single real variable and functions of a single complex variable. Separating these two subjects does not at all jive with their development historically, and to me it seems unnecessary and potentially confusing to do so. On the other hand, functions of several variables seems to me to be a very different kettle of fish, so I have decided to limit this book by concentrating on one variable at a time. Everyone is taught (told) in school that the area of a circle is given by the formula A = πr2: We are also told that the product of two negatives is a positive, that you cant trisect an angle, and that the square root of 2 is irrational. Students of natural sciences learn that eiπ = 1 and that sin2 + cos2 = 1: More sophisticated students are taught the Fundamental− Theorem of calculus and the Fundamental Theorem of Algebra.
    [Show full text]
  • Triple Product Formula and the Subconvexity Bound of Triple Product L-Function in Level Aspect
    TRIPLE PRODUCT FORMULA AND THE SUBCONVEXITY BOUND OF TRIPLE PRODUCT L-FUNCTION IN LEVEL ASPECT YUEKE HU Abstract. In this paper we derived a nice general formula for the local integrals of triple product formula whenever one of the representations has sufficiently higher level than the other two. As an application we generalized Venkatesh and Woodbury’s work on the subconvexity bound of triple product L-function in level aspect, allowing joint ramifications, higher ramifications, general unitary central characters and general special values of local epsilon factors. 1. introduction 1.1. Triple product formula. Let F be a number field. Let ⇡i, i = 1, 2, 3 be three irreducible unitary cuspidal automorphic representations, such that the product of their central characters is trivial: (1.1) w⇡i = 1. Yi Let ⇧=⇡ ⇡ ⇡ . Then one can define the triple product L-function L(⇧, s) associated to them. 1 ⌦ 2 ⌦ 3 It was first studied in [6] by Garrett in classical languages, where explicit integral representation was given. In particular the triple product L-function has analytic continuation and functional equation. Later on Shapiro and Rallis in [19] reformulated his work in adelic languages. In this paper we shall study the following integral representing the special value of triple product L function (see Section 2.2 for more details): − ⇣2(2)L(⇧, 1/2) (1.2) f (g) f (g) f (g)dg 2 = F I0( f , f , f ), | 1 2 3 | ⇧, , v 1,v 2,v 3,v 8L( Ad 1) v ZAD (FZ) D (A) ⇤ \ ⇤ Y D D Here fi ⇡i for a specific quaternion algebra D, and ⇡i is the image of ⇡i under Jacquet-Langlands 2 0 correspondence.
    [Show full text]
  • Variables in Mathematics Education
    Variables in Mathematics Education Susanna S. Epp DePaul University, Department of Mathematical Sciences, Chicago, IL 60614, USA http://www.springer.com/lncs Abstract. This paper suggests that consistently referring to variables as placeholders is an effective countermeasure for addressing a number of the difficulties students’ encounter in learning mathematics. The sug- gestion is supported by examples discussing ways in which variables are used to express unknown quantities, define functions and express other universal statements, and serve as generic elements in mathematical dis- course. In addition, making greater use of the term “dummy variable” and phrasing statements both with and without variables may help stu- dents avoid mistakes that result from misinterpreting the scope of a bound variable. Keywords: variable, bound variable, mathematics education, placeholder. 1 Introduction Variables are of critical importance in mathematics. For instance, Felix Klein wrote in 1908 that “one may well declare that real mathematics begins with operations with letters,”[3] and Alfred Tarski wrote in 1941 that “the invention of variables constitutes a turning point in the history of mathematics.”[5] In 1911, A. N. Whitehead expressly linked the concepts of variables and quantification to their expressions in informal English when he wrote: “The ideas of ‘any’ and ‘some’ are introduced to algebra by the use of letters. it was not till within the last few years that it has been realized how fundamental any and some are to the very nature of mathematics.”[6] There is a question, however, about how to describe the use of variables in mathematics instruction and even what word to use for them.
    [Show full text]
  • Eigenvalues and Eigenvectors
    Jim Lambers MAT 605 Fall Semester 2015-16 Lecture 14 and 15 Notes These notes correspond to Sections 4.4 and 4.5 in the text. Eigenvalues and Eigenvectors In order to compute the matrix exponential eAt for a given matrix A, it is helpful to know the eigenvalues and eigenvectors of A. Definitions and Properties Let A be an n × n matrix. A nonzero vector x is called an eigenvector of A if there exists a scalar λ such that Ax = λx: The scalar λ is called an eigenvalue of A, and we say that x is an eigenvector of A corresponding to λ. We see that an eigenvector of A is a vector for which matrix-vector multiplication with A is equivalent to scalar multiplication by λ. Because x is nonzero, it follows that if x is an eigenvector of A, then the matrix A − λI is singular, where λ is the corresponding eigenvalue. Therefore, λ satisfies the equation det(A − λI) = 0: The expression det(A−λI) is a polynomial of degree n in λ, and therefore is called the characteristic polynomial of A (eigenvalues are sometimes called characteristic values). It follows from the fact that the eigenvalues of A are the roots of the characteristic polynomial that A has n eigenvalues, which can repeat, and can also be complex, even if A is real. However, if A is real, any complex eigenvalues must occur in complex-conjugate pairs. The set of eigenvalues of A is called the spectrum of A, and denoted by λ(A). This terminology explains why the magnitude of the largest eigenvalues is called the spectral radius of A.
    [Show full text]
  • Lecture 13: Simple Linear Regression in Matrix Format
    11:55 Wednesday 14th October, 2015 See updates and corrections at http://www.stat.cmu.edu/~cshalizi/mreg/ Lecture 13: Simple Linear Regression in Matrix Format 36-401, Section B, Fall 2015 13 October 2015 Contents 1 Least Squares in Matrix Form 2 1.1 The Basic Matrices . .2 1.2 Mean Squared Error . .3 1.3 Minimizing the MSE . .4 2 Fitted Values and Residuals 5 2.1 Residuals . .7 2.2 Expectations and Covariances . .7 3 Sampling Distribution of Estimators 8 4 Derivatives with Respect to Vectors 9 4.1 Second Derivatives . 11 4.2 Maxima and Minima . 11 5 Expectations and Variances with Vectors and Matrices 12 6 Further Reading 13 1 2 So far, we have not used any notions, or notation, that goes beyond basic algebra and calculus (and probability). This has forced us to do a fair amount of book-keeping, as it were by hand. This is just about tolerable for the simple linear model, with one predictor variable. It will get intolerable if we have multiple predictor variables. Fortunately, a little application of linear algebra will let us abstract away from a lot of the book-keeping details, and make multiple linear regression hardly more complicated than the simple version1. These notes will not remind you of how matrix algebra works. However, they will review some results about calculus with matrices, and about expectations and variances with vectors and matrices. Throughout, bold-faced letters will denote matrices, as a as opposed to a scalar a. 1 Least Squares in Matrix Form Our data consists of n paired observations of the predictor variable X and the response variable Y , i.e., (x1; y1);::: (xn; yn).
    [Show full text]
  • Algebra of Linear Transformations and Matrices Math 130 Linear Algebra
    Then the two compositions are 0 −1 1 0 0 1 BA = = 1 0 0 −1 1 0 Algebra of linear transformations and 1 0 0 −1 0 −1 AB = = matrices 0 −1 1 0 −1 0 Math 130 Linear Algebra D Joyce, Fall 2013 The products aren't the same. You can perform these on physical objects. Take We've looked at the operations of addition and a book. First rotate it 90◦ then flip it over. Start scalar multiplication on linear transformations and again but flip first then rotate 90◦. The book ends used them to define addition and scalar multipli- up in different orientations. cation on matrices. For a given basis β on V and another basis γ on W , we have an isomorphism Matrix multiplication is associative. Al- γ ' φβ : Hom(V; W ) ! Mm×n of vector spaces which though it's not commutative, it is associative. assigns to a linear transformation T : V ! W its That's because it corresponds to composition of γ standard matrix [T ]β. functions, and that's associative. Given any three We also have matrix multiplication which corre- functions f, g, and h, we'll show (f ◦ g) ◦ h = sponds to composition of linear transformations. If f ◦ (g ◦ h) by showing the two sides have the same A is the standard matrix for a transformation S, values for all x. and B is the standard matrix for a transformation T , then we defined multiplication of matrices so ((f ◦ g) ◦ h)(x) = (f ◦ g)(h(x)) = f(g(h(x))) that the product AB is be the standard matrix for S ◦ T .
    [Show full text]