122 Circulant Matrices × General 2 2 matrix × Ax = b ac x ax + cx b 1 = 1 2 = 1 bd x2 bx1 + dx2 b2 ∙ ¸ ∙ ¸ ∙ ¸ ∙ ¸ General 2 2 circulant matrix × ab x ax + bx b 1 = 1 2 = 1 ba x2 bx1 + ax2 b2 ∙ ¸ ∙ ¸ ∙ ¸ ∙ ¸ 1.1 Evaluating the Eigenvalues Find eigenvalues and eigenvectors of general 2 2 circulant matrix: × ab x x λx ax + bx 1 = λ 1 = 1 = 1 2 ba x2 · x2 λx2 bx1 + ax2 ∙ ¸ ∙ ¸ ∙ ¸ ∙ ¸ ∙ ¸ λx1 = ax1 + bx2

λx2 = bx1 + ax2

add these = λ (x1 + x2)=(a + b)(x1 + x2)= λ1 = a + b ⇒ ⇒ subtract these = λ (x1 x2)=(a b)(x1 x2)= λ2 = a b ⇒ − − − ⇒ − Adifferent method: ab x x λx 1 = λ 1 = 1 ba x2 · x2 λx2 ∙ ¸ ∙ ¸ ∙ ¸ ∙ ¸ λ 0 x 10 x = 1 = λ 1 0 λ x2 01 x2 ∙ ¸ ∙ ¸ ∙ ¸ ∙ ¸ Ax = λ Ix · = Ax λ Ix = 0 ⇒ − · ab λ 0 = (A λ I) x = x = 0 ⇒ − · ba − 0 λ µ∙ ¸ ∙ ¸¶ ab λ 0 a λb = − B ba − 0 λ baλ ≡ µ∙ ¸ ∙ ¸¶ ∙ − ¸

= the vector x is in the null subspace of the matrix B ⇒ 1 = B− does not exist ⇒ = determinant of B is zero ⇒

1 The determinant of the general 2 2 matrix is: × ac det = ad bc bd − ∙ ¸ a λb 2 det − =(a λ) b2 =0 baλ − − ∙ − ¸ λ2 2aλ + a2 b2 =0 − − ¡ ¢ ( 2a) ( 2a)2 4(a2 b2) λ = − − ± − − − q 2 √4a2 4a2 +4b2 =+a − ± 2 λ = a b ± By convention, we select

λ1 = a + b

λ2 = a b − 1.2 Evaluating the Eigenvectors Now we can find expressions for the eigenvectors:

ab x1 x1 x1 = λ1 =(a + b) ba x2 · x2 · x2 ∙ ¸ ∙ ¸1 ∙ ¸1 ∙ ¸1 ab x1 ax1 + bx2 (a + b) x1 = = · ba x2 bx1 + ax2 (a + b) x2 ∙ ¸ ∙ ¸1 ∙ ¸1 ∙ · ¸1 Equate expressions for first component : ax1 + bx2 =(a + b) x1 = ax1 + bx2 = ax1 + bx1 · ⇒ = bx2 = bx1 = x1 = x2 ⇒ ⇒

Equate expressions for second component : bx1 + ax2 =(a + b) x2 = bx1 + ax2 = ax2 + bx2 · ⇒ = bx1 = bx2 = x1 = x2, SAME RESULT ⇒ ⇒ So the two components of the “first” eigenvector (corresponding to our conven- tion for the “first” eigenvalue λ1 = a + b)areequal:

x1 x1 = x1 ∙ ¸ which thus lie along the identity line.

2 2 x2

1

-2 -1 1 2 x1

-1

-2 The components of the first eigenvector of the 2 2 circulant matrix lie along × the line x2 = x1 We often “normalize” the eigenvector to unit length, for reasons that will be more obvious later:

1 +1 xˆ = corresponding to λ1 = a + b 1 √2 +1 ∙ ¸ The components of the second eigenvector must simultaneously statisfy the two conditions:

ab x1 x1 x1 = λ2 =(a b) ba x2 · x2 − · x2 ∙ ¸ ∙ ¸2 ∙ ¸2 ∙ ¸2 ab x1 ax1 + bx2 (a b) x1 = = − · ba x2 bx1 + ax2 (a +b) x2 ∙ ¸ ∙ ¸2 ∙ ¸2 ∙ − · ¸2 Equate expressions for first component : ax1 + bx2 =(a b) x1 = ax1 + bx2 = ax1 bx1 − · ⇒ − = bx2 = bx1 = x1 = x2 ⇒ − ⇒−

Equate expressions for second component : bx1 + ax2 =(a b) x2 = bx1 + ax2 = ax2 bx2 − · ⇒ − = bx1 = bx2 = x1 = x2,SAMERESULT ⇒ − ⇒ − One choice for the normalized second eigenvector is:

1 +1 xˆ = corresponding to λ2 = a b 2 √2 1 − ∙ − ¸

3 1 1 Note that we could alternatively choose xˆ = − , but we choose that 2 √2 +1 with the positively valued first component by convention.∙ ¸ We can now plot the two normalized eigenvectors on the x-y plane:

2-D plane with “normal” (or “canonical”) basis vectors shown in black: +1 0 and , and the two normalized eigenvectors shown in red: 0 +1 ∙ ¸ ∙1 ¸ 1 √2 1 +1 √2 1 +1 x = 1 = and x = 1 = 1 √2 +1 2 √2 1 " √2 # ∙ ¸ " − √2 # ∙ − ¸

4 1.3 Projection of the Input Vector onto the Eigenvectors Clearly the two eigenvectors are orthogonal and normalized (unit length xˆ = | 1| xˆ2 =1), so they may be used as “reference” vectors to express any other 2-D | | +2 vector. For example, the vector x = may be written in the “normal” +3 basis as: ∙ ¸ 2 +1 0 =2 +3 3 · 0 · +1 ∙ ¸ ∙ ¸ ∙ ¸ where the “3” and “2” are the respective projections of the vector onto the unit reference vectors. We can find the projections onto the two eigenvectors by evaluating the scalar product. For example, the project ion of the vector +2 1 x = onto the first eigenvector xˆ = 1 is: +3 1 √2 1 ∙ ¸ ∙ ¸ 1 +1 2 xˆ x = 1 • √2 +1 • 3 ∙ ¸ ∙ ¸ + 1 √2 2 2 3 5 = 1 = + =+ + • 3 √2 √2 √2 " √2 # ∙ ¸

1 +1 2 xˆ x = 2 • √2 1 • 3 ∙ − ¸ ∙ ¸ + 1 √2 2 2 3 1 = 1 = = • 3 √2 − √2 −√2 " − √2 # ∙ ¸

5 2 Projection of the vector onto the two eigenvectors of the 2 2 circulant 3 × ∙ ¸ matrix. Note that the projection onto the second eigenvector xˆ2 is NEGATIVE, because the projection points in the “opposite” direction from the direction the vector points.

This “alternative” representation of the input vector is equivalent and some- times more useful (as we shall see): 5 + +2 √2 1 +5 x = = x0 +3 ⇐⇒ ⎡ 1 ⎤ √2 1 ≡ ∙ ¸ ∙ − ¸ ⎢ −√2 ⎥ ⎣ ⎦ where we just add a “prime” to the notation for the input vector x to denote

6 the representation in terms of the “new” basis vectors, that in turn were based on the 2 2 circulant matrix. × 1.4 Inverse of 2 2 Circulant Matrix × Recall the formula for the inverse of the general 2 2 matrix: ×

ac 1 1 d c A = = A− = − bd ⇒ ad bc ba ∙ ¸ − ∙ − ¸ As always, it is useful to confirm our memory:

1 1 d c ac A− A = − ad bc ba bd − ∙ − ¸ ∙ ¸ 1 ad bc dc cd = − − ad bc ba + ab bc + ad − ∙ − − ¸ 1 ad bc 0 10 = − = ad bc 0 bc + ad 01 − ∙ − ¸ ∙ ¸ which is the required identity matrix. You should confirm that the same result 1 is obtained from AA− Now to the circulant case:

ab 1 1 a b A = = A− = − ba ⇒ a2 b2 ba ∙ ¸ − ∙ − ¸ 1 1 a b ab 10 A− A = − = a2 b2 ba ba 01 − ∙ − ¸ ∙ ¸ ∙ ¸ Sothistellsusrightawaywhatmustbetrueabouta, b to ensure that the inverse matrix exists. The determinant must not be zero: ab det = a2 b2 =(a + b)(a b) =0 ba − − 6 ∙ ¸ = a + b =0AND a b =0 ⇒ 6 − 6 = a = b ⇒ 6 ± Also note that the determinant of the circulant is the product of the eigenvalues:

ab det A =det =(a + b)(a b)=λ1 λ2 ba − · ∙ ¸ which tells us that the matrix is invertible if neither eigenvalue is zero.

1.4.1 Example: a=b ab 11 A = = 1 ba 11 ∙ ¸ ∙ ¸

7 First apply the formulas for the eigenvalues:

λ1 = a + b =1+1=2

λ2 = a b =1 1=0 − − The two row vectors are identical, which tells us that the row subspace is 1-D:

1 1 1 x = α1 + α2 = α1 { R}1 1 1 1 ∙ ¸ ∙ ¸ ∙ ¸ which includes all vectors along the diagonal, i.e., all vectors that are propor- tional to the first eigenvector of the general circulant matrix:

11 Row subspace of the matrix A = includes all vectors along the line 1 11 ∙x2 = x1.¸

8 Apply the matrix to a vector in the row subspace:

11 1 2 1 A x = = =2 1 { R}1 11 1 2 · 1 ∙ ¸ ∙ ¸ ∙ ¸ ∙ ¸ which confirms that vectors in the row subspace of this matrix are eigenvectors with eigenvalue λ1 =+2. The null subspace of this matrix includes all vectors that are orthogonal to the vectors in the row subspace:

x x = x x =0 { N }1 ⊥ { R}1 ⇒ { N }1 •{ R}1 u u 1 x = = α1 = u + v =0 = v = u { N }1 v ⇒ v • 1 ⇒ ⇒ − ∙ ¸ ∙ ¸ ∙ ¸ So the vectors in the null subspace are proportional to the normalized vector:

1 +1 x { N }1 ∝ √2 1 ∙ − ¸ Check by applying the matrix to this vector:

11 1 +1 0 A x = = = 0 1 { N }1 11 √2 1 0 ∙ ¸µ ∙ − ¸¶ ∙ ¸

9 11 Null subspace of A = includes all vectors along the line x2 = x1. 1 11 − The null subspace of this∙ matrix¸ is the blue dashed line and the row subspace is the red dotted line.

1.4.2 Example: b=-a ab +1 1 A = = 2 ba 1+1− ∙ ¸ ∙ − ¸

λ1 = a + b =1+( 1) = 0 − λ2 = a b =1 ( 1) = 2 − − − −

10 The two row vectors are identical but for a change of sign:

+1 1 +1 +1 x = α1 + α2 − = α { R}2 1 +1 1 ∝ 1 ∙ − ¸ ∙ ¸ ∙ − ¸ ∙ − ¸ which tells us that the row subspace is 1-D. Note that the row subspace of A2 includes all of the vectors that were in the row subspace of A1 which means in turn that the null subspace of A2 must be identical to the row subspace of A1: +1 x { N }2 ∝ +1 ∙ ¸

As always, it is advisable to check it by applying the matrix A2 to the formula for the nullspace vector:

+1 1 +1 0 A x = − = = 0 2 { N }2 1+1 +1 0 ∙ − ¸ ∙ ¸ ∙ ¸ includes all vectors along the diagonal, i.e., all vectors that are proportional to the first eigenvector of the general circulant matrix:

11