Gaussian Elimination

Andrew Stacey

November 26, 2013

Meta-Theorem

Almost every linear algebra problema can be solved using Gaus- sian Elimination. aAt least as far as Matte 3 is concerned.

The general strategy for solving a linear algebra problem in Matte 3 is the following:

1. Find the matrix.

2. Run Gaussian Elimination on that matrix.

3. Interpret the result.

1 Find the Matrix

Sometimes, it’s obvious: Exam Question: Kont 2011, Problem 4

Let ⎡ 2 −3 6 2 5 ⎤ ⎢−2 3 −3 −3 −4⎥ 퐴 = ⎢ ⎥ ⎢ 4 −6 9 5 9 ⎥ ⎣−2 3 3 −4 1 ⎦

a) Find a basis for the null space, Null(퐴), and a basis for the row space, Row(퐴). b) Find a basis for the column space, Col(퐴). What is rank(퐴)?

The give-away is:

1 Let ⎡ 2 −3 6 2 5 ⎤ ⎢−2 3 −3 −3 −4⎥ 퐴 = ⎢ ⎥ ⎢ 4 −6 9 5 9 ⎥ ⎣−2 3 3 −4 1 ⎦

However, sometimes it is not the obvious one, but one constructed from it. Exam Question: Autumn 2011, Problem 4

Let ⎡1 1 1 1 ⎤ 퐴 = ⎢2 0 −3 4 ⎥ ⎢ ⎥ ⎣0 2 5 −2⎦

a) Find a basis for the solution space Null(퐴) and a basis for the column space Col(퐴). Find the rank of 퐴, rank(퐴).

⎡1⎤ b) For which values of 푎 is 푏⃗ = ⎢1⎥ in Col(퐴)? ⎢ ⎥ ⎣푎⎦ c) Let 푇 be a linear transformation with standard matrix 퐴. Mark each of the following statements true or false (the answers should be justiﬁed).

(1) 푇 is a linear transformation from ℝ3 to ℝ4, (2) 푇 is a linear transformation from ℝ4 to ℝ3, (3) 푇 is onto, (4) 푇 is one-to-one.

Part b) suggests that the matrix to use is actually:

⎡1 1 1 1 1⎤ ⎢2 0 −3 4 1⎥ ⎢ ⎥ ⎣0 2 5 −2 푎⎦ Sometimes, it is hidden. It might be a linear system: Exam Question: Autumn 2010, Problem 5

Let 푉 ⊆ ℝ4 be the solution space of the system of equations 푥 + 푦 − 푧 + 푤 = 0 푥 + 2푦 − 2푧 + 푤 = 0 1) Find an orthogonal basis for 푉 . 2) Find the orthogonal projection of 푏⃗ = (1, 1, 1, 1) onto 푉 .

2 3) Find an orthogonal basis for ℝ4 where the ﬁrst two vectors of this basis are the vectors you found in part a).

Here, the matrix has to be extracted from the linear system. In this case it is: 1 1 −1 1 [1 2 −2 1] The matrix might be given as a linear transformation: Exam Question: Sprint 2012, Problem 5

2푥 + 푦 + 푧 푥 ⎡ ⎤ ⎛⎡ ⎤⎞ ⎢−푥 + 3푦 + 푧⎥ Let 푇 ∶ ℝ3 → ℝ4 be deﬁned by 푇 ⎜⎢푦⎥⎟ = . ⎜⎢ ⎥⎟ ⎢ 2푥 − 푧 ⎥ ⎣푧⎦ ⎢ ⎥ ⎝ ⎠ ⎣ 푦 + 4푧 ⎦

⎛⎡푥⎤⎞ ⎡푥⎤ 1) Find a matrix 퐴 such that 푇 ⎜⎢푦⎥⎟ = 퐴 ⎢푦⎥. ⎜⎢ ⎥⎟ ⎢ ⎥ ⎝⎣푧⎦⎠ ⎣푧⎦ 2) Find dim Null(퐴) and a basis for Col(퐴). Is 푇 one-to-one (injective)? Is 푇 onto (surjective)?

Part a) asks us explicitly to ﬁnd the matrix of 푇 . To do this, we extract the coeﬃcients from the description. Thus

⎡ 2 1 1 ⎤ ⎢−1 3 1 ⎥ 퐴 = ⎢ ⎥ ⎢ 2 0 −1⎥ ⎣ 0 1 4 ⎦

The matrix might be hidden as a family of vectors. Exam Question: Kont 2010, Problem 6

For which values of the parameter 푎 are the vectors 푣⃗1 = (1, −3, 푎), 푣⃗2 = (0, 1, 푎), and 푣⃗3 = (푎, 2, 0) linearly independent?

In this case, we put the vectors together to create the matrix (being sure to use column vectors):

⎡ 1 0 푎⎤ ⎢−3 1 2⎥ ⎢ ⎥ ⎣ 푎 푎 0⎦

2 Run Gaussian Elimination

1. Work from top-left to bottom-right,

3 ⎡ ∗ ∗ ∗ ∗ ∗ ⎤ ⎢ ∗ ∗ ∗ ∗ ∗ ⎥ ⎢ ∗ ∗ ∗ ∗ ∗ ⎥ ⎢ ⎥ ⎣ ∗ ∗ ∗ ∗ ∗ ⎦

2. Once we have processed 푖 − 1 rows and 푗 − 1 columns, we look at the remaining part of the matrix and pick a non-zero entry in the left-most non-zero column to be the next pivot.

⎡ ∗ ∗ ∗ ∗ ∗ ⎤ ⎢ 0 0 0 ∗ ∗ ⎥ ⎢ ⎥ ⎢ 0 0 2 ∗ ∗ ⎥ ⎢ ⎥ ⎣ 0 0 4 ∗ ∗ ⎦

3. We swap this into the 푖th row.

⎡ ∗ ∗ ∗ ∗ ∗ ⎤ ⎡ ∗ ∗ ∗ ∗ ∗ ⎤ ⎢ 0 0 0 ∗ ∗ ⎥ ⎢ 0 0 2 ∗ ∗ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ 0 0 2 ∗ ∗ ⎥ ⎢ 0 0 0 ∗ ∗ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ 0 0 4 ∗ ∗ ⎦ ⎣ 0 0 4 ∗ ∗ ⎦

4. Then use it to “purge” the lower non-zero entries from that column.

⎡ ∗ ∗ ∗ ∗ ∗ ⎤ ⎡ ∗ ∗ ∗ ∗ ∗ ⎤ ⎢ 0 0 2 ∗ ∗ ⎥ ×(−2) ⎢ 0 0 2 ∗ ∗ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ 0 0 0 ∗ ∗ ⎥ ⎢ 0 0 0 ∗ ∗ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ 0 0 4 ∗ ∗ ⎦ ⎣ 0 0 0 ∗ ∗ ⎦

5. Now we mark that row and column as “done” (and any columns to its left) and continue until we reach the edge of the matrix.

The result is in row echelon form:

⎡ • ∗ ∗ ∗ ∗ ⎤ ⎢ 0 0 • ∗ ∗ ⎥ ⎢ ⎥ ⎢ 0 0 0 0 • ⎥ ⎢ ⎥ ⎣ 0 0 0 0 0 ⎦

pivots

4 3 Interpret the Result

pivot columns zero rows

⎡ • ∗ ∗ ∗ ∗ ⎤ ⎢ 0 0 • ∗ ∗ ⎥ ⎢ ⎥ ⎢ 0 0 0 0 • ⎥ ⎢ ⎥ ⎣ 0 0 0 0 0 ⎦

pivot rows free columns

The exact interpretation depends on the question asked. The most important distinction is as to whether the matrix is the coeﬃcient matrix or the augmented matrix.

3.1 The Coeﬃcient Matrix

Pivot Columns

• Number of pivot columns = dim Col(퐴) = rank of matrix • Pivot columns in original matrix are a basis for the column space • Every column a pivot column matrix is injective, columns are linearly independent • If the matrix is square, every column a pivot column matrix is invertible

Free Columns

• Number of free columns = dim Null(퐴) = nullity of matrix • Each free column gives a basis vector for the null space by setting that entry to 1 and the entries corresponding to the other free columns to 0, and setting the other entries accord- ingly

• There are free columns matrix is not injective, columns are linearly dependent

5 Pivot Rows

• Number of pivot rows = dim Row(퐴) = rank of matrix • Pivot rows in echelon form are a basis for the row space

• Every row a pivot row matrix is surjective, rows are linarly independent

Zero Rows

• There are zero rows matrix is not surjective

3.2 The Augmented Matrix When considering the augmented matrix we are interested in the exis- tence and uniqueness of solutions to the equation 퐴⃗푥= 푏⃗. Free Columns

• The last column is a free column there is at least one solution. • There are no other free columns there is at most one solution.

3.3 Other Interpretations • Gaussian elimination is also useful for calculating determinants:

det 퐴 = (−1)#row swaps diagonal entries in echelon form ∏

• For a square matrix we can use the number of pivot columns to test for invertibility. If we determine that it is invertible, we can further use any of the equivalent characterisations of invertible matrices.

4 Characterisations of Invertible Matrices

For an 푛×푛–square matrix 퐴, the following are equivalent to the statement that 퐴 is invertible.

1. For any 푛–vector 푏⃗ the matrix equation 퐴⃗푥= 푏⃗ has a solution. 2. The linear transformation ⃗푥↦ 퐴⃗푥 is onto. 3. The columns of 퐴 span ℝ푛.

4. There is a 푛 × 푛–matrix 퐶 such that 퐴퐶 = 퐼푛.

6 5. For any 푛–vector 푏⃗ the matrix equation 퐴⃗푥= 푏⃗ has at most one solution.

6. The matrix equation 퐴⃗푥= 0⃗ has a unique solution.

7. The linear transformation ⃗푥↦ 퐴⃗푥 is one-to-one. 8. The columns of 퐴 are linearly independent.

9. There is a 푛 × 푛–matrix 퐷 such that 퐷퐴 = 퐼푛. 10. Every column is a pivot column.

11. Its reduced echelon form is 퐼푛. 12. det 퐴 ≠ 0.

13. All of the above with 퐴 replaced by 퐴⊤.