Symmetric Matrices and Quadratic Forms Quadratic form • Suppose 풙 is a column vector in ℝ푛, and 퐴 is a symmetric 푛 × 푛 matrix. • The term 풙푇퐴풙 is called a quadratic form. • The result of the quadratic form is a scalar. (1 × 푛)(푛 × 푛)(푛 × 1) • The quadratic form is also called a quadratic function 푄 풙 = 풙푇퐴풙. • The quadratic function’s input is the vector 푥 and the output is a scalar. 2 Quadratic form • Suppose 푥 is a vector in ℝ3, the quadratic form is: 푎11 푎12 푎13 푥1 푇 • 푄 풙 = 풙 퐴풙 = 푥1 푥2 푥3 푎21 푎22 푎23 푥2 푎31 푎32 푎33 푥3 2 2 2 • 푄 풙 = 푎11푥1 + 푎22푥2 + 푎33푥3 + ⋯ 푎12 + 푎21 푥1푥2 + 푎13 + 푎31 푥1푥3 + 푎23 + 푎32 푥2푥3 • Since 퐴 is symmetric 푎푖푗 = 푎푗푖, so: 2 2 2 • 푄 풙 = 푎11푥1 + 푎22푥2 + 푎33푥3 + 2푎12푥1푥2 + 2푎13푥1푥3 + 2푎23푥2푥3 3 Quadratic form • Example: find the quadratic polynomial for the following symmetric matrices: 1 −1 0 1 0 퐴 = , 퐵 = −1 2 1 0 2 0 1 −1 푇 1 0 푥1 2 2 • 푄 풙 = 풙 퐴풙 = 푥1 푥2 = 푥1 + 2푥2 0 2 푥2 1 −1 0 푥1 푇 2 2 2 • 푄 풙 = 풙 퐵풙 = 푥1 푥2 푥3 −1 2 1 푥2 = 푥1 + 2푥2 − 푥3 − 0 1 −1 푥3 2푥1푥2 + 2푥2푥3 4 Motivation for quadratic forms • Example: Consider the function 2 2 푄 푥 = 8푥1 − 4푥1푥2 + 5푥2 Determine whether Q(0,0) is the global minimum. • Solution we can rewrite following equation as quadratic form 8 −2 푄 푥 = 푥푇퐴푥 푤ℎ푒푟푒 퐴 = −2 5 The matrix A is symmetric by construction. Eigen vectors of A are 1 2 1 1 푣1 = , 푣2 = 5 −1 5 2 With associated eigenvalues 휆1 = 9 푎푛푑휆2 = 4 Example Cont’ • Let 푥 = 푐1푣1 + 푐2푣2, Now we have 푇 2 2 2 2 푄 푥 = 푥 퐴푥 = 휆1푐1 + 휆2푐2 = 9푐1 + 4푐2 Therefore Q(x)>0 and Q(0,0) is the global minimum. Quadratic form • Example: find the symmetric matrix for the following quadratic polynomials: 2 2 푄1 풙 = 푥1 + 푥2 + 2푥1푥2 2 2 푄2 풙 = 푥1 + 푥2 + 2푥1푥2 − 4푥1푥3 + 2 2푥2푥3 푇 1 1 푥1 • 푄1 풙 = 풙 퐴풙 = 푥1 푥2 1 1 푥2 1 1 −2 푥1 푇 • 푄2 풙 = 풙 퐵풙 = 푥1 푥2 푥3 1 1 2 푥2 −2 2 0 푥3 7 Geometric interpretation in ℝ2 • For diagonal matrix 퐴, 푄(풙) is an ellipse, or hyperbola, or intersection of two lines, or a point. Motivation: This form happens for diagonal matrices and maxima and minima appear along the eigenvectors and a and b are the eigenvalues 8 Geometric interpretation in ℝ2 • For non-diagonal matrix 퐴, 푄(풙) is a rotated geometry. Motivation: This form happens for nondiagonal matrices and maxima and minima appear along the eigenvectors (but not aligned) . For aligning them we can use change of variables as explained in next slides 9 Change of variable • We can convert the rotated ellipse or hyperbola to its standard form. • Recall that we can diagonalize symmetric matrices. • If 퐴 is a symmetric matrix, it can be diagonalized as 퐷 = 푃푇퐴푃, where 푃 is the orthogonal matrix of eigenvectors of 퐴. • Suppose 풙 = 푃풚, then 푄 풙 = 풙푇퐴풙 = 푃풚 푇퐴 푃풚 = 풚푇푃푇퐴푃풚 = 풚푇퐷풚 Motivation: Alignment is required for change of variable. This is diagonalization of symmetric matrices 10 Change of Variable Example • Example: Make a change of variable that transforms the quadratic 2 2 form 푄 푥 = 푥1 − 8푥1푥2 − 5푥2 into a quadratic form with no cross- product term. • Solution if we write Q(x) as quadratic form, matrix A is The first step is to orthogonally diagonalize A. Its eigenvalues and with associated unit eigenvectors are Example Cont’ • Then 퐴 = 푃퐷푃−1 and 퐷 = 푃−1퐴푃 = 푃푇퐴푃. A suitable change of variable is Then.