Review: Linear and a Linear System Lecture # 2: Systems of Linear Important Concepts Gaussian Elimination and the Gauss-Jordan Method 1 A linear equation in n variables defines an( n − 1)-dimensional hyperplane in an n-dimensional space.

2 Monson H. Hayes A linear system is a set of m linear equations in n variables, and the system defines a set of m hyperplanes in an n-dimensional space. [email protected]

A linear equation: a1x1 + a2x2 + ··· + anxn = b A system of m linear equations in n variables (unknowns):

a11x1 + a12x2 + ··· + a1nxn = b1 a21x1 + a22x2 + ··· + a2nxn = b2 . . . . This material is the property of the author and is for the sole and exclusive use of his students. It is not for publication, nor is it to be sold, reproduced, or generally distributed. am1x1 + am2x2 + ··· + amnxn = bm

M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 1 / 36 M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 2 / 36

Review: Systems of Linear Equations Solving a Linear System

A number of different things can happen when working with systems Reading for Today’s Class: Anton, pp. 11-24. of linear equations. Optional but recommended: The MATLAB tutorials written by Edward A system of linear equations may have: Neuman that are posted on the course web page.

1 No solution (inconsistent equations), 2 One and only one (unique) solution (consistent equations), or We now look at how to solve a linear system. 3 An infinite number of solutions (consistent equations). The first step will be the introduce to coefficient and the An important thing to look at when dealing with systems of linear . These matrices will help facilitate the solution equations is procedure. 1 The number of unknowns (variables) and 2 The number of equations. Then we will define a set of operations that are performed on the equations to put them in a form where the solution is easily What, if anything, can you say if there are (think visually!): determined. 1 More unknowns than equations, 2 More equations than unknowns. These operations are known, in one form, as Gaussian Elimination and in another form as the Gauss-Jordan Method.

M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 3 / 36 M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 4 / 36 What is a Matrix? The Coefficient Matrix

In mathematics, a matrix is a rectangular array of numbers, symbols, Consider the following linear system (two equations in two unknowns) or expressions. Page 1 of 1 The individual items in a matrix are called elements or entries. x1 − 2x2 = −1 In a general matrix with elements ai,j , the first index i is the row and −x1 + 3x2 = 3 the second index j is the column. A convenient and shorthand method for representing the coefficients that multiply the variables is to define a coefficient matrix,

ai,j n columns j changes   m x1 − 2x2 = −1 1 −2 rows a1,1 a1,2 a1,3 −x1 + 3x2 = 3 −1 3 (coefficient matrix)

a2,1 a2,2 a2,3

a3,1 a3,2 a3,3 ......

M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 5 / 36 M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 6 / 36

The Augmented Matrix Echelon Forms

We may also include the parameters on the right-hand side of the A matrix is said to be in if linear system into the coefficient matrix, which results in the 1 If there are any rows containing only zeros, then they are grouped augmented coefficient matrix, together at the bottom of the matrix.   2 The first nonzero number in every nonzero row is equal to one (this is x1 − 2x2 = −1 1 −2 −1 called a leading 1). −x1 + 3x3 = 3 −1 3 3 (augmented matrix) 3 For any two successive nonzero rows, the leading 1 in the lower row Sometimes it may be useful to partition the augmented matrix to occurs farther to the right than the leading 1 in the higher row. emphasize/remind us that the right hand column of the matrix represents the parameters of the linear system. A matrix is said to be in Reduced Row Echelon Form if, in addition to the above, x − 2x = −1  1 −2 −1  1 2 5 Each column that contains a leading 1, has zeros everywhere else in −x1 + 3x3 = 3 −1 3 3 that column (not just below the leading 1). .

M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 7 / 36 M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 8 / 36

file:///D:/LaTeX/Class/CAU%20Linear%20Algebra/Lectures/Lecture%2002/Matrixpng.svg 3/2/2012 Echelon Form Examples Algebraic Operations on a Linear System

Solving a linear system involves performing algebraic operations on the system that do not alter the solution set. Row Echelon Form Reduced Row Echelon Form Eventually, one can determine whether or not the linear system is 1 ∗ ∗ ∗ ∗ 1 0 0 ∗ ∗ consistent and, if it is, the solution set can be found. 0 1 ∗ ∗ ∗ 0 1 0 ∗ ∗     A set of elementary algebraic operations on a linear system are 0 0 1 ∗ ∗ 0 0 1 ∗ ∗ 1 Interchange the order of two equations. 0 0 0 0 0 0 0 0 0 0 2 Multiply an equation by a (nonzero) constant. 3 Add one equation to another. 0 1 ∗ ∗ ∗ ∗ ∗ ∗ ∗ 0 1 ∗ 0 ∗ 0 0 0 ∗ It is clear that each of these operations do not change the solution set 0 0 0 1 ∗ ∗ ∗ ∗ ∗ 0 0 0 1 ∗ 0 0 0 ∗ of a linear system.     0 0 0 0 0 1 ∗ ∗ ∗ 0 0 0 0 0 1 0 0 ∗     0 0 0 0 0 0 1 ∗ ∗ 0 0 0 0 0 0 1 0 ∗ 2x1 + x2 + 4x3 = 1 0 0 0 0 0 0 0 1 ∗ 0 0 0 0 0 0 0 1 ∗ 3x1 − x2 − 4x3 = 6

M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 9 / 36 M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 10 / 36

Elementary Row Operations Example

Since the rows of an augmented matrix correspond to equations in a linear   system, the elementary algebraic operations performed on the equations may x1 + x2 + 2x3 = 9 1 1 2 9 be performed on the augmented matrix. 2x1 + 4x2 − 3x3 = 1  24 −3 1  These correspond to a set of elementary row operations: 3x1 + 6x2 − 5x3 = 0 3 6 −5 0 We begin by making the leading term in the second equation equal to zero. Elementary Algebraic Operations Elementary Row Operations To do this, we add −2 times the first equation to the second equation. Interchange order of two equations Interchange the order of two rows   Multiply equation by a constant Multiply a row by a constant x1 + x2 + 2x3 = 9 1 1 2 9 Add one equation to another Add one row to another 2x2 − 7x3 = −17  0 2 −7 −17  3x1 + 6x2 − 5x3 = 0 36 −5 0 Note that the last two operations may be combined into a single operation of: Adding a constant times one row to another. Next, we would like to make the first term in the third equation equal to Solving a Linear System - Step 1: Perform elementary row operations on the zero, so add −3 times the first equation to the third equation. augmented matrix to put it into row echelon form.   x1 + x2 + 2x3 = 9 1 1 2 9 2x2 − 7x3 = −17  0 2 −7 −17  3x2 − 11x3 = −27 0 3 −11 −27

M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 11 / 36 M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 12 / 36 Example (cont.) Example (cont.)

    x1 + x2 + 2x3 = 9 1 1 2 9 x1 + x2 + 2x3 = 9 1 1 2 9 7 17 7 17 2x2 − 7x3 = −17  02 −7 −17  x2 − 2 x3 = − 2  0 1 − 2 − 2  1 3 1 3 3x2 − 11x3 = −27 0 3 −11 −27 − 2 x3 = − 2 0 0 − 2 − 2 The leading term in the second equation should be equal to one, so Finally, to make the leading term in the last row equal to one, multiply the multiply the second equation by 1/2. third equation by −2   x1 + x2 + 2x3 = 9 1 1 2 9 x + x + 2x = 9  1 1 2 9  7 17 7 17 1 2 3 x2 − 2 x3 = − 2  0 1 − 2 − 2  7 17 7 17 x2 − 2 x3 = − 2  0 1 − 2 − 2  3x2 − 11x3 = −27 03 −11 −27 x3 = 3 0 0 1 3 To make the first term in the last row equal to one, add −3 times the Note: The matrix is now in Row Echelon Form. second equation to the third equation,   x1 + x2 + 2x3 = 9 1 1 2 9 The linear system may be easily solved by a step called back-substitution. 7 17 7 17 x2 − 2 x3 = − 2  0 1 − 2 − 2  1 3 1 3 − 2 x3 = − 2 0 0 − 2 − 2

M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 13 / 36 M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 14 / 36

Example (cont.) Example (cont.)

x1 + x2 + 2x3 = 9 We may continue with row reductions, and put the matrix into 7 17 Reduced Row Echelon Form. x2 − 2 x3 = − 2 x3 = 3 The process is to begin with the rightmost entry in the last nonzero Back-Substitution row, and working upwards to the left, we create zeros above each leading entry. 1 x3 = 3

2 x = − 17 + 7 x = 2   2 2 2 3 x1 + x2 + 2x3 = 9 1 1 2 9 7 17 7 17 x2 − x3 = −  0 1 − −  3 x1 = 9 − x2 − 2x3 = 1 2 2 2 2 x3 = 3 0 013 This method of solving a linear system is called Gaussian Elimination, 7 and consists of two steps: Add 2 times the third equation to the second, 11 39  11 35  1 Step 1: Putting the augmented matrix in row echelon form, which is x1 + x2 + 2 x3 = 2 1 1 2 2 called Forward Elimination. x2 = 2  0 1 0 2  x3 = 3 0 013 2 Step 2: Back-Substitution

M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 15 / 36 M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 16 / 36 Example (cont.) Another Example

11 39  11 39    x1 + x2 + 2 x3 = 2 1 1 2 2 3x2 − 6x3 = 8 0 3 −6 8 x2 = 2  0 1 0 2  x1 − 2x2 + 3x3 = −1  1 −2 3 −1  x3 = 3 0 013 5x1 − 7x2 + 9x3 = 0 5 −7 9 0 11 Solution Add − 2 times the third equation to the first,  0 3 −6 8   1 −2 3 −1   1 −2 3 −1  x + x = 3  1 1 0 3  1 2 1 −2 3 −1 → 0 3 −6 8 → 0 3 −6 8 x = 2 0102       2   5 −7 9 0 0 3 −6 5 0 0 0 3 x3 = 3 0 0 1 3 So, the equations have the form Subtract the second equation from the first,   x1 − 2x3 + 3x3 = −1 x1 = 1 1 0 0 1 3x2 − 6x3 = 8 x2 = 2 0 1 0 2   0x3 = 3 ← Never True x3 = 3 0 0 1 3

Note that the matrix is in reduced echelon form, and the solution is now clear: The linear equations are inconsistent, and there is no solution. The equations correspond to three non-intersecting planes. [x1, x2, x3] = [1, 2, 3]. This is called the Gauss-Jordan Method.

M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 17 / 36 M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 18 / 36

One More Example One More Example (cont.)

    x1 + 3x3 = −1 1 0 3 −1 x1 + 3x3 = −1 1 0 3 −1 2x1 + x2 + 2x3 = 0  2 1 2 0  x2 − 4x3 = 2  0 1 −4 2  + 2x − 8x = 4 0 2 −8 4 2 3 0 0 0 0 Solution The last equation can be omitted. It places no restrictions on x , x , or x .       1 2 3 1 0 3 −1 1 0 3 1 1 0 3 −1 The remaining two equations in three variables define two planes in 3-d. 2 1 2 0 → 0 1 −4 2 → 0 1 −4 2       Since x and x correspond to leading 1’s in the echelon form of the 0 2 −8 4 0 2 −8 4 0 0 0 0 1 2 augmented matrix, they are called leading variables.

Note that the last row is a zero row. This means that the solution is not unique. The remaining variables are free variables, which in this case is x3. We have two equations in three unknowns. Therefore, provided that they are The free variable may be assigned any value, α, and for a given value of α, consistent (which they are), there will be an infinite number of solutions. the corresponding values for x1 and x2 can be found. The solution set, which depends on the value of the free variable, is given by

x1 = −1 − 3α ; x2 = 2 + 4α ; x3 = α Later we will see that this represents a line in 3-d.

M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 19 / 36 M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 20 / 36 Some Facts About Echelon Forms Example

1 Every matrix has a unique reduced row echelon form. 2 Row echelon forms are not unique (adding any multiple of a row to a  1 4 5 −9 7   1 4 5 −9 7  row above it will result in a different row echelon form of the matrix).  −1 −2 −1 3 1   0 2 4 −6 −6    −→   −→ 3 All row echelon forms of a matrix have the same number of zero  −2 −3 0 3 −1   0 5 10 −15 −15  rows, and the leading 1’s always occur in the same positions. These 0 −3 −6 4 9 0 −3 −6 4 9 are called the pivot positions of the matrix, and the column that contains the pivot is called a pivot column.

 1 4 5 −9 7   1 4 5 −9 −7  Example: Consider the following reduced echelon form of a matrix:  0 2 4 −6 −6   0 2 4 −6 −6    −→  −→  1205 −2 1   0 0 0 0 0   0 0 0 −5 0  0 0 0 −5 0 0 0 0 0 0  0 0143 −2  0 0 0 011 What are the pivot columns? Pivots are shown in red. The pivot columns are columns 1, 3, and 5.

M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 21 / 36 M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 22 / 36

Row Echelon Forms Row Echelon Forms

The row echelon form of a matrix may have one of four possible forms: 2 Second form: 1 The first form, which is only possible when the number of equations is  1 ∗∗∗∗∗∗∗∗  equal to the number of unknowns, is  0 0 0 1 ∗∗∗∗∗    1 ∗ ∗ ∗ ∗ ∗  0 0 0 0 0 1 ∗ ∗ ∗    0 1 ∗ ∗ ∗ ∗  0 0 0 0 0 0 1 ∗ ∗    0 0 1 ∗ ∗ ∗ 0 0 0 0 0 0 0 1 ∗   0 0 0 1 ∗ ∗   x1 x2 x3 x4 x5 x6 x7 x8 ∗ 0 0 0 0 1 ∗ In this case, the linear system has one or more free variables (for this Note that a * indicates an arbitrary number, which may or may not particular matrix, the free variables are x2, x3, and x5). be zero. When an augmented matrix has this row echelon form, the system of When an augmented matrix has this row echelon form, the system of linear equations has an infinite number of solutions. linear equations has a unique solution.

M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 23 / 36 M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 24 / 36 Row Echelon Forms Row Echelon Forms

3 Third form: 3 Fourth form:

 1 ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗   1 ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗   0 0 0 1 ∗ ∗ ∗ ∗ ∗   0 0 0 1 ∗ ∗ ∗ ∗ ∗       0 0 0 0 0 1 ∗ ∗ ∗  b 6= 0  0 0 0 0 0 1 ∗ ∗ ∗       0 0 0 0 0 0 1 ∗ ∗   0 0 0 0 0 0 1 ∗ ∗  0 0 0 0 0 0 0 0 b 0 0 0 0 0 0 0 0 0 In this case, the last equation is of the form This form is characterized by having one or more rows of zeros. 0 = b Zero rows can be ignored (eliminated), and after they are removed, the resulting row echelon form matrix will have one of previous three which can never be true, so these equations are inconsistent. forms. When an augmented matrix has this row echelon form, the linear system has no solutions.

M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 25 / 36 M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 26 / 36

Existence and Uniqueness Assessment

1 What is the largest possible number of pivots a 4 × 6 matrix can have? Important Results (Theorem)   1 A linear system is consistent if and only if the row echelon form of the a11 a12 a13 a14 a15 a16  a21 a22 a23 a24 a25 a26  augmented matrix does not have a row of the form    a31 a32 a33 a34 a35 a36  [0, 0,..., 0, b]; b 6= 0 a41 a42 a43 a44 a45 a46

2 If a linear system is consistent, then the solution either has 2 What is the largest possible number of pivots a 6 × 4 matrix can have? 1 A unique solution (there are no free variables), or   a11 a12 a13 a14 2 An infinite number of solutions (there is at least one free variable).  a21 a22 a23 a24     a31 a32 a33 a34     a41 a42 a43 a44     a51 a52 a53 a54  a61 a62 a63 a64

3 How many solutions does a consistent linear system of 3 equations and 4 unknowns have?

M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 27 / 36 M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 28 / 36 Homogeneous Linear Systems Homogeneous Linear Systems

A linear system is said to be homogeneous if the constant terms are equal to zero (the last column of the augmented matrix is zero). a11x1 + a12x2 + ··· + a1nxn = 0 a x + a x + ··· + a x = 0 a x + a x + ··· + a x = 0 21 1 22 2 2n n 11 1 12 2 1n n . a21x1 + a22x2 + ··· + a2nxn = 0 . . a x + a x + ··· + a x = 0 . m1 1 m2 2 mn n am1x1 + am2x2 + ··· + amnxn = 0 Each linear equation defines an (n − 1)-dimensional hyperplane that There are only two possibilities for the solution set of a homogeneous passes through the origin. linear system: A homogeneous linear system is always consistent since it always has 1 The system has only the trivial solution, the trivial solution, x1 = x2 = ··· = xn = 0. 2 The system has infinitely many solutions in addition to the trivial solution.

M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 29 / 36 M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 30 / 36

Illustration MATLAB and Octave - An Introduction

By now you should have either MATLAB or Octave installed on your

3 computer.

2 Linear algebra is a branch of mathematics that involves working with 1 matrices and vectors.

‐3 ‐2 ‐1 1 2 34 MATLAB and Octave excel in the manipulation and processing of matrices and linear systems. 3 Some nice tutorials written by Edward Neuman are posted on the 2 course web page. 1 At this point, I will get you started with MATLAB / Octave, and as we ‐3 ‐2 ‐1 1 2 34 progress through the course, ways to use MATLAB / Octave to solve linear algebra problems will be introduced. 3

2 We begin with a short tutorial on vectors, matrices, and finding the

1 row echelon form of a matrix.

‐3 ‐2 ‐1 1 2 34

M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 31 / 36 M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 32 / 36 MATLAB and Octave - An Introduction Vectors

When you run MATLAB / octave, a command window will appear that The following command will create a row vector (matrix with one row), allows users to enter commands. In MATLAB , there will be a prompt >> v = [ 1 2 3 4 5] that looks like this: v = 1 2 3 4 5 >> A column vector (a matrix with one column) may be created with the and in Octave, it will look something like: following command (Note the semicolons between the entries): >> v = [ 1 ; 2 ; 3 ] octave:1> v = 1 2 3 The operator will reverse rows and columns. Thus, the transpose of a row vector is a column vector and vice versa. For example, >> v = [ 1 ; 2 ; 3 ]’ is a row vector.

M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 33 / 36 M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 34 / 36

Matrices The MATLAB function rref

The following matrix rref: Reduced row echelon form 16 2 3 13  5 11 10 8  Syntax    9 7 6 12 4 14 15 1 >> R = rref(A)

may be created in MATLAB with the command Description >> A = [ 16 2 3 13 ; 5 11 10 8 ; 9 7 6 12 ; 4 14 15 1]; R = rref(A) produces the reduced row echelon form of A using Gauss We may concatenate matrices and vector (if we are careful). For example, Jordan elimination with partial pivoting. with A defined as above, we can form the matrix 16 2 3 13 1  5 11 10 8 1    9 7 6 12 1 4 14 15 1 1 as follows >> b = [1 ; 1 ; 1 ; 1]; >> B = [A b]; M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 35 / 36 M.Hayes (CAU-GT) Lecture # 2 March 6, 2014 36 / 36