Math 3191 Applied Linear Algebra Lecture 2: Row Reduction/Vector Equations

Home , Reduction (mathematics)

Stephen Billups

University of Colorado at Denver

Math 3191Applied Linear Algebra – p.1/25 Announcements

Please discard pages 15–25 from Lecture 1. (These slides have been redone to be consistent with the textbook).

Math 3191Applied Linear Algebra – p.2/25 Outline

Review Row Reduction Vector Equations

Math 3191Applied Linear Algebra – p.3/25 Review

Last time, we introduced elementary row operations. Key Fact: Elementary row operations do not alter the solution set. Tonight: we look out how to use elementary row operations to solve a linear system. Key Concepts: Reduced echelon form. (RREF) Row reduction.

Math 3191Applied Linear Algebra – p.4/25 Gauss-Jordan Elimination

Apply a sequence of elementary row operations to get the matrix into a form that is trivial to solve.

Example: A series of elementary row operations yields the following transformation

2 1 1 1 3 3 2 1 0 0 1 3 2 −3 −1 −8 =⇒ 0 1 0 4 6 7 6 7 6 7 6 7 4 −1 2 2 3 5 4 0 0 1 −2 5

The righthand matrix corresponds to the system of equations:

x1 = 1

x2 = 4

x3 = −2

Trivial to solve!!

The above matrix is in a special form called the reduced row echelon form.

Math 3191Applied Linear Algebra – p.5/25 Reduced Row Echelon Form:

Zero rows are at the bottom. The leftmost nonzero entry of each nonzero row, (called its pivot) equals 1. Each pivot is further to the right than the pivot in the row above it. Each pivot is the only nonzero entry in its column.

Examples:

2 1 2 0 0 3 2 1 0 2 2 3 0 0 1 0 0 1 −1 3 6 7 6 7 6 7 6 7 4 0 0 0 1 5 4 0 0 0 0 5

Math 3191Applied Linear Algebra – p.6/25 Examples–NOT RREF

1 2 1 0 0 0 0 0  0 0 1 0   0 1 0 3  0 0 0 1 0 0 1 −2         Nonzero in pivot col. Zero row not at bottom.

1 0 2 2  0 2 −1 3  0 0 0 0     Pivot =6 1

Math 3191Applied Linear Algebra – p.7/25 Exercise

Which of the following are in RREF? 1 0 7 1 0 3 0 1 −8 A. 0 1 −2 . C.   .   0 0 0 0 0 1      0 0 0      1 0 −2 1 2   B. . 1 0 0 1 " 0 1 −1 3 1 # D.  0 0 1 −1  . 0 0 0 0    

Math 3191Applied Linear Algebra – p.8/25 Interpretting RREF (for Augmented Matrix)

Inconsistent system: If there is a pivot in the right-hand side, the system is inconsistent (i.e., it has no solution).

1 2 0 0 x = −2x =⇒ 1 2 " 0 0 0 1 # 0 = 1 Impossible! NOTE: You can usually detect inconsistency before reaching RREF:

2 4 6 2 4 3 =⇒ (Inconsistent) " 3 6 8 # " 0 0 −1 #

Math 3191Applied Linear Algebra – p.9/25 Free Variables

Non-pivot columns correspond to free variables.

Free variables can take on any values, and each choice of values yields a solution.

The (general) solution to the linear system is written in terms of the free variables.

Example:

2 1 2 0 3 0 7 3 0 0 1 0 0 1 6 7 6 7 6 0 0 0 0 1 2 7 6 7 6 7 4 0 0 0 0 0 0 5

Free variables = x2 and x4. Two ways to write the solution:

x1 = −2s − 3t + 7 x1 = −2x2 − 3x4 + 7 x2 = s x3 = 1 or x3 = 1 x5 = 2 x4 = t x2, x4 free x5 = 2

Math 3191Applied Linear Algebra – p.10/25 Exercise

What is the general solution of the system whose augmented matrix is as follows?

1 2 0 1  0 0 1 −1  . 0 0 0 0    

Math 3191Applied Linear Algebra – p.11/25 Uniqueness of RREF

Theorem 1: Each matrix is row equivalent to one and only one reduced row echelon matrix. The order of row operations doesn’t matter–you always get the same RREF. The book also talks about row echelon form (REF). Look over this deﬁnition on your own. Row echelon form is not unique.

Math 3191Applied Linear Algebra – p.12/25 0 0 −2 0 7 12 originally row 1 0 0 5 0 −17 −29 row 3 plus (-1) times the pivot row 0 0 −4 0 15 26 row 4 plus (-4) times the pivot row

Row Reduction Algorithm

− 2 0 0 2 0 7 12 3 2 4 −10 6 12 28 6 7 Example: 6 7 6 2 4 −5 6 −5 −1 7 6 7 6 7 4 8 16 −44 24 63 138 5

Step 1: Locate the pivot column (the leftmost column that does not consist entirely of zeros). (Ex: Column 1).

Step 2: Choose a row with a nonzero entry in the pivot column. Interchange this row with the top row. (For consistency, choose the topmost row with a nonzero entry. (Ex: row 2 ).

Step 3: Create zeros in the pivot column below the pivot by adding multiples of the pivot row to the remaining rows:

− 2 2 4 10 6 12 28 3 Originally row 2 6 7 6 7 6 7 6 7 6 7 4 5

Math 3191Applied Linear Algebra – p.13/25 0 0 5 0 −17 −29 row 3 plus (-1) times the pivot row 0 0 −4 0 15 26 row 4 plus (-4) times the pivot row

Row Reduction Algorithm

− 2 0 0 2 0 7 12 3 2 4 −10 6 12 28 6 7 Example: 6 7 6 2 4 −5 6 −5 −1 7 6 7 6 7 4 8 16 −44 24 63 138 5

Step 1: Locate the pivot column (the leftmost column that does not consist entirely of zeros). (Ex: Column 1).

Step 2: Choose a row with a nonzero entry in the pivot column. Interchange this row with the top row. (For consistency, choose the topmost row with a nonzero entry. (Ex: row 2 ).

Step 3: Create zeros in the pivot column below the pivot by adding multiples of the pivot row to the remaining rows:

− 2 2 4 10 6 12 28 3 Originally row 2 0 0 −2 0 7 12 originally row 1 6 7 6 7 6 7 6 7 6 7 4 5

Math 3191Applied Linear Algebra – p.13/25 0 0 −4 0 15 26 row 4 plus (-4) times the pivot row

Row Reduction Algorithm

− 2 0 0 2 0 7 12 3 2 4 −10 6 12 28 6 7 Example: 6 7 6 2 4 −5 6 −5 −1 7 6 7 6 7 4 8 16 −44 24 63 138 5

Step 1: Locate the pivot column (the leftmost column that does not consist entirely of zeros). (Ex: Column 1).

Step 2: Choose a row with a nonzero entry in the pivot column. Interchange this row with the top row. (For consistency, choose the topmost row with a nonzero entry. (Ex: row 2 ).

Step 3: Create zeros in the pivot column below the pivot by adding multiples of the pivot row to the remaining rows:

− 2 2 4 10 6 12 28 3 Originally row 2 0 0 −2 0 7 12 originally row 1 6 7 6 7 6 0 0 5 0 −17 −29 7 row 3 plus (-1) times the pivot row 6 7 6 7 4 5

Math 3191Applied Linear Algebra – p.13/25 Row Reduction Algorithm

− 2 0 0 2 0 7 12 3 2 4 −10 6 12 28 6 7 Example: 6 7 6 2 4 −5 6 −5 −1 7 6 7 6 7 4 8 16 −44 24 63 138 5

Step 1: Locate the pivot column (the leftmost column that does not consist entirely of zeros). (Ex: Column 1).

Step 2: Choose a row with a nonzero entry in the pivot column. Interchange this row with the top row. (For consistency, choose the topmost row with a nonzero entry. (Ex: row 2 ).

Step 3: Create zeros in the pivot column below the pivot by adding multiples of the pivot row to the remaining rows:

− 2 2 4 10 6 12 28 3 Originally row 2 0 0 −2 0 7 12 originally row 1 6 7 6 7 6 0 0 5 0 −17 −29 7 row 3 plus (-1) times the pivot row 6 7 6 7 4 0 0 −4 0 15 26 5 row 4 plus (-4) times the pivot row

Math 3191Applied Linear Algebra – p.13/25 0 0 0 0 1/2 1 row 3 plus 5/2 times the pivot row 0 0 0 0 1 2 row 4 plus (-2) times the pivot row

Row Reduction Alg. (cont).

Step 4: Cover up top row and repeat Steps 1-3 on the remaining part of the matrix.

− 2 2 4 10 6 12 28 3 0 0 −2 0 7 12 6 7 6 7 6 0 0 5 0 −17 −29 7 6 7 6 7 4 0 0 −4 0 15 26 5

− 2 2 4 10 6 12 28 3 cover (ignore) this row 0 0 −2 0 7 12 pivot column = third column 6 7 6 7 6 7 6 7 6 7 4 5

Math 3191Applied Linear Algebra – p.14/25 0 0 0 0 1 2 row 4 plus (-2) times the pivot row

Row Reduction Alg. (cont).

Step 4: Cover up top row and repeat Steps 1-3 on the remaining part of the matrix.

− 2 2 4 10 6 12 28 3 0 0 −2 0 7 12 6 7 6 7 6 0 0 5 0 −17 −29 7 6 7 6 7 4 0 0 −4 0 15 26 5

− 2 2 4 10 6 12 28 3 cover (ignore) this row 0 0 −2 0 7 12 pivot column = third column 6 7 6 7 6 0 0 0 0 1/2 1 7 row 3 plus 5/2 times the pivot row 6 7 6 7 4 5

Math 3191Applied Linear Algebra – p.14/25 Row Reduction Alg. (cont).

Step 4: Cover up top row and repeat Steps 1-3 on the remaining part of the matrix.

− 2 2 4 10 6 12 28 3 0 0 −2 0 7 12 6 7 6 7 6 0 0 5 0 −17 −29 7 6 7 6 7 4 0 0 −4 0 15 26 5

− 2 2 4 10 6 12 28 3 cover (ignore) this row 0 0 −2 0 7 12 pivot column = third column 6 7 6 7 6 0 0 0 0 1/2 1 7 row 3 plus 5/2 times the pivot row 6 7 6 7 4 0 0 0 0 1 2 5 row 4 plus (-2) times the pivot row

Math 3191Applied Linear Algebra – p.14/25 0 0 0 0 0 0 row 4 plus (-2) times the pivot row

NOTES:

The matrix is now in row echelon form but not reduced row echelon form.

Two ways to proceed from here: Use back substitution (more later). Continue on to get RREF.

Row Reduction Alg. (cont).

One more time: − 2 2 4 10 6 12 28 3 0 0 −2 0 7 12 6 7 6 7 6 0 0 0 0 1/2 1 7 6 7 6 7 4 0 0 0 0 1 2 5

− 2 2 4 10 6 12 28 3 cover (ignore) this row 0 0 −2 0 7 12 cover this row 6 7 6 7 6 0 0 0 0 1/2 1 7 pivot columm = col 5. 6 7 6 7 4 5

Math 3191Applied Linear Algebra – p.15/25 NOTES:

The matrix is now in row echelon form but not reduced row echelon form.

Two ways to proceed from here: Use back substitution (more later). Continue on to get RREF.

Row Reduction Alg. (cont).

One more time: − 2 2 4 10 6 12 28 3 0 0 −2 0 7 12 6 7 6 7 6 0 0 0 0 1/2 1 7 6 7 6 7 4 0 0 0 0 1 2 5

− 2 2 4 10 6 12 28 3 cover (ignore) this row 0 0 −2 0 7 12 cover this row 6 7 6 7 6 0 0 0 0 1/2 1 7 pivot columm = col 5. 6 7 6 7 4 0 0 0 0 0 0 5 row 4 plus (-2) times the pivot row

Math 3191Applied Linear Algebra – p.15/25 Row Reduction Alg. (cont).

One more time: − 2 2 4 10 6 12 28 3 0 0 −2 0 7 12 6 7 6 7 6 0 0 0 0 1/2 1 7 6 7 6 7 4 0 0 0 0 1 2 5

− 2 2 4 10 6 12 28 3 cover (ignore) this row 0 0 −2 0 7 12 cover this row 6 7 6 7 6 0 0 0 0 1/2 1 7 pivot columm = col 5. 6 7 6 7 4 0 0 0 0 0 0 5 row 4 plus (-2) times the pivot row

NOTES:

The matrix is now in row echelon form but not reduced row echelon form.

Two ways to proceed from here: Use back substitution (more later). Continue on to get RREF. Math 3191Applied Linear Algebra – p.15/25 2 4 −10 6 0 4 row 1 plus (-12) times row 3 0 0 −2 0 0 −2 row 2 plus (-7) times row 3

Create zeros above the pivot by adding multiples of the 2 2 4 0 6 0 14 3 row 1 plus 10 times row 2 pivot row. 0 0 1 0 0 1 scaled by -1/2 6 7 6 7 Step 6: Cover up 6 0 0 0 0 1 2 7 6 7 pivot row and below, 6 7 4 0 0 0 0 0 0 5 and repeat Step 5. Last step: scale row 1 by 1/2.

REF → RREF

2 4 −10 6 12 28 2 3 0 0 −2 0 7 12 6 7 Working Bottom to Top 6 7 6 0 0 0 0 1/2 1 7 6 7 6 7 4 0 0 0 0 0 0 5

Step 5. Scale last 2 3 nonzero row to 6 7 6 7 make the pivot entry 6 0 0 0 0 1 2 7 scaled (multiplied) by 2 6 7 equal to 1. 6 7 4 0 0 0 0 0 0 5

Math 3191Applied Linear Algebra – p.16/25 2 2 4 0 6 0 14 3 row 1 plus 10 times row 2 0 0 1 0 0 1 scaled by -1/2 6 7 6 7 Step 6: Cover up 6 0 0 0 0 1 2 7 6 7 pivot row and below, 6 7 4 0 0 0 0 0 0 5 and repeat Step 5. Last step: scale row 1 by 1/2.

REF → RREF

2 4 −10 6 12 28 2 3 0 0 −2 0 7 12 6 7 Working Bottom to Top 6 7 6 0 0 0 0 1/2 1 7 6 7 6 7 4 0 0 0 0 0 0 5

Math 3191Applied Linear Algebra – p.16/25 2 4 0 6 0 14 row 1 plus 10 times row 2

Last step: scale row 1 by 1/2.

REF → RREF

2 4 −10 6 12 28 2 3 0 0 −2 0 7 12 6 7 Working Bottom to Top 6 7 6 0 0 0 0 1/2 1 7 6 7 6 7 4 0 0 0 0 0 0 5

− Step 5. Scale last 2 2 4 10 6 0 4 3 row 1 plus (-12) times row 3 nonzero row to 0 0 −2 0 0 −2 row 2 plus (-7) times row 3 6 7 6 7 make the pivot entry 6 0 0 0 0 1 2 7 scaled (multiplied) by 2 equal to 1. Create 6 7 6 0 0 0 0 0 0 7 zeros above the 4 5 pivot by adding multiples of the 2 3 pivot row. 0 0 1 0 0 1 scaled by -1/2 6 7 6 7 Step 6: Cover up 6 0 0 0 0 1 2 7 6 7 pivot row and below, 6 7 4 0 0 0 0 0 0 5 and repeat Step 5.

Math 3191Applied Linear Algebra – p.16/25 REF → RREF

2 4 −10 6 12 28 2 3 0 0 −2 0 7 12 6 7 Working Bottom to Top 6 7 6 0 0 0 0 1/2 1 7 6 7 6 7 4 0 0 0 0 0 0 5

− Step 5. Scale last 2 2 4 10 6 0 4 3 row 1 plus (-12) times row 3 nonzero row to 0 0 −2 0 0 −2 row 2 plus (-7) times row 3 6 7 6 7 make the pivot entry 6 0 0 0 0 1 2 7 scaled (multiplied) by 2 equal to 1. Create 6 7 6 0 0 0 0 0 0 7 zeros above the 4 5 pivot by adding multiples of the 2 2 4 0 6 0 14 3 row 1 plus 10 times row 2 pivot row. 0 0 1 0 0 1 scaled by -1/2 6 7 6 7 Step 6: Cover up 6 0 0 0 0 1 2 7 6 7 pivot row and below, 6 7 4 0 0 0 0 0 0 5 and repeat Step 5. Last step: scale row 1 by 1/2.

Math 3191Applied Linear Algebra – p.16/25 RREF–Finally

1 2 0 3 0 7 scaled by 1/2  0 0 1 0 0 1  0 0 0 0 1 2    0 0 0 0 0 0      From RREF, we can “read off” the solution:

x1 = −2x2 − 3x4 + 7, x2 and x4 are free x3 = 1 x5 = 2

Math 3191Applied Linear Algebra – p.17/25 Geometry of Linear Systems

In 2 variables, each equation corresponds to a line. With two equations, there are three possibilities:

1. Intersecting lines (unique solution).

1 0 a RREF: 2 3 4 0 1 b 5

2. Parallel lines (no solution). 1 0 a RREF: 2 3 4 0 0 b 5 3. Coincident lines (infinitely many solutions).

1 0 a RREF: 2 3 4 0 0 0 5

Math 3191Applied Linear Algebra – p.18/25 Geometry (cont)

In 3 variables, each equation corresponds to a plane. There are numerous possibilities:

Math 3191Applied Linear Algebra – p.19/25 Answer: NO. There must be a free variable. So if there is a solution, there are inﬁnitely many solutions.

Existence and Uniqueness Theorem

Theorem 2: A linear system is consistent if and only if the rightmost column of the augmented matrix is not a pivot column. If the RREF has a row of the form [0 0 . . . 0b] with b nonzero, then the system is inconsistent. If not, the system is consistent. If there is a free variable, then there are inﬁnitely many solutions. If not, there is a unique solution.

Question: Suppose an augmented matrix has 3 rows and 5 columns. Can it have a unique solution?

Math 3191Applied Linear Algebra – p.20/25 Existence and Uniqueness Theorem

Question: Suppose an augmented matrix has 3 rows and 5 columns. Can it have a unique solution? Answer: NO. There must be a free variable. So if there is a solution, there are inﬁnitely many solutions.

Math 3191Applied Linear Algebra – p.20/25 Back Substitution

− 2 2 4 10 6 12 28 3 0 0 −2 0 7 12 6 7 From row echelon form: 6 7 6 0 0 0 0 1/2 1 7 6 7 6 7 4 0 0 0 0 0 0 5

Use last nonzero row (row 3) to solve for last non-free variable (x5):

1/2x5 = 1 =⇒ x5 = 2.

Plug this value into row above (row 2):

−2x3 + 7( 2 ) = 12 =⇒ x3 = 1. x |{z}5

Plug x3 and x5 into next row up:

2x1 + 4x2 − 10( 1 ) + 6x4 + 12( 2 ) = 28 =⇒ x1 = −2x2 − 3x4 + 7. x x |{z}3 |{z}5

Math 3191Applied Linear Algebra – p.21/25 Section 1.3: Vector Equations

Key Concepts:

Euclidean vectors (vectors in IRn). Arithmetic operations on vectors. Linear Combinations. Vector Equations. Span.

Math 3191Applied Linear Algebra – p.22/25 Vectors in IRn

Textbook deﬁnition: A matrix with only one column is called a column vector or simply a vector.

3 e.g.  1  . 4     Note: Eventually, we will deﬁne vector spaces, and call other things vectors, but for now, think of vectors as column vectors. The set of all vectors with n real components is denoted by IRn. (IR denotes the set of real numbers).

Math 3191Applied Linear Algebra – p.23/25 Notation

Space saver notation: To save space, we can represent a vector as just an ordered list of numbers: e.g. u = (1, 2, 3), instead of the more cumbersome 1 2 3 column notation u = 2 . 6 7 6 7 4 3 5 Note the use of parentheses instead of square brackets. Thus,

1 2 3 (1, 2, 3) = 2 =6 1 2 3 . 6 7 h i 6 7 4 3 5

n Subscripts: If u is a vector in IR , we denote the ith component of u by ui. For example, suppose u = (5, 4, 2). Then u1 = 5, u2 = 4, and u3 = 2.

Math 3191Applied Linear Algebra – p.24/25 Operations on Vectors

n Vector addition: Given two vectors u, v ∈ IR , their sum u + v is obtained by adding corresponding entries of u and v. That is

2 u1 + v1 3 u2 + v2 6 7 u + v = 6 7 . 6 . 7 6 . 7 6 7 6 7 4 un + vn 5

Example: u = (1, 2, 3), v = (4, 5, 6). u + v = (5, 7, 9). n Scalar multiplication: Given a vector u ∈ IR and a scalar (real number) c, the scalar multiple cu is obtained by multiplying each entry of u by c. That is

2 cu1 3 cu2 6 7 cu = 6 . 7 . 6 . 7 6 . 7 6 7 6 7 4 cun 5

Example: u = (1, 2, 3), c = −2. cu = (−2, −4, −6). Math 3191Applied Linear Algebra – p.25/25