Eigenproblems
There is perhaps no set of matrix techniques that are more important in practice!
Definition: Let A = [ajk]beagien] be a given n n matrix and consider the vector equation Ax = x, then the column vector solutions x are eigenvectors (characteristic vectors) and the corresponding values of are eigenvalues (characteristic values) More generally: Let A be a linear transformation that maps a linear vector space S (usually a Hilbert space in applications) into itself , A:S S, and let x S, then the solutions to Ax = x for x are eigenvectors and the corresponding values of are eigenvalues Remark: The number of independent eigenvectors is at most n for the n n matrix. The number can be infinite in general.
8.1
Eigenproblems
There is perhaps no set of matrix techniques that are more important in practice!
Definition: Let A = [ajk]beagien] be a given n n matrix and consider the vector equation Ax = x, then the column vector solutions x are eigenvectors (characteristic vectors) and the corresponding values of are eigenvalues (characteristic values) More generally: Let A be a linear transformation that maps a linear vector space S (usually a Hilbert space in applications) into itself , A:S S, and let x S, then the solutions to Ax = x for x are eigenvectors and the corresponding values of are eigenvalues Remark: A key question to which we will return multiple times is: Do the eigenvectors span the linear vector space n n or more generally S ? 8.2
1 Eigenproblems
52 Example 1: Consider the simple matrix A and the 22 corresponding characteristic equation Ax = x, then in this case, we may write in components
52xx11 Ax , which becomes 22 xx22
52 x1 ()0AI x or 0 22 x2 This equation only has a solution if (A – I) is singular or det(A – I) = 0. This condition is obeyed if ( –5 – )(–2 – ) – 4=04 = 0 or 2 +7+ 7 +6=0+ 6 = 0.
Solutions are 1 = and 2 = . The equation for the first eigenvalue x1 now becomes 42x11 1 0 x1 21 x21 2 where is an arbitrary constant.
8.3
Eigenproblems
Example 1 (continued): Similarly, the equation for the second eigenvalue x2 becomes 12x12 2 0 x2 24x22 1 where is an arbitrary constant. More generally: The eigenvalue equation (A – I) = 0 implies
aa11 12 a 1n aa a D() det(AI )21 22 2n 0
aann12 a nn This determinant is an n-th order polynomial in . We must find the roots of this polynomial to obtain the eigenvalues. This is a hard problem in general!
8.4
2 Eigenproblems
Theorem 1: The eigenvalues of a square n n matrix A are the roots of the characteristic equation det(A – I) = 0, which is an n-th order polynomial. Hence, each square matrix has at least one eigenvalue and no more than n . Proof: It is a consequence of the fundamental theorem of algebra that any n-th order polynomial has n (in general complex) roots. These are the eigenvalues. The number of distinct roots is between 1 and n. Remark: Once the eigenvalues are found, the eigenvectors may be found by Gaussian elimination. Theorem 2: The space of eigenvectors corresponding to an eigenvalue is a linear manifold . This manifold must have at least dimension 1 . Proof: The manifold of eigenvectors corresponds to the null space of A – I. Since this matrix is singular, we have nullity A – I 1
8.5
Eigenproblems
Example 2: Find the eigenvalues of the matrix 22 3 A 21 6 12 0 The characteristic determinant det(A – I) = 0 yields the equation 32 21 45 0 (5 )(3 ) 2 0
This yields the roots (eigenvalues) The first root yields the rank 2 matrix and eigenvalue
72 3 1 AI 246and2 x 1 125 1
8.6
3 Eigenproblems
Example 2: We next consider 12 3 123 AI3246 000 after row reduction 123 000 The null space has two dimensions, yielding two independent eigenvectors 23 xx 1and0 23 01
Definition: The order of an eigenvalue M is the algebraic multiplicity of . The number m of linearly independent vectors corresponding to an eigenvector is its geometric multiplicity. In general, M mThe difference M m is the defect of . A matrix that has any eigenvalues with a non-zero defect is defective.
8.7
Eigenproblems
Example 2: We next consider 12 3 123 AI3246 000 after row reduction 123 000 The null space has two dimensions, yielding two independent eigenvectors 23 xx 1and0 23 01
Remark: In Example 2 , we find M m and M m . So, neither eigenvalue is defective.
Remark: Defective isn’t necessarily bad! The problem of finding the parameters for critical damping of an oscillator can be framed mathematically as a search for parameters that yield a defective matrix. 8.8
4 Eigenproblems
Example 3 (A defective matrix): We next consider 21 A which has the eigenvalues 12 2 02 T with eigenvector x1 = [1 0] and no others! To see that, we note that any T eigenvector [x1 x2] would have to satisfy 2x1 + x2 = 2x1, so that x2 = 0. Hence, in the case M = 2 and m Definition: A matrix J of the form shown below is a Jordan matrix. 1 0 J 1 0
Theorem: A Jordan matrix has m = 1.
8.9
Eigenproblems
Example 4 (Real matrices with complex eigenvalues and eigenvectors): Since polynomials with real coefficients have complex roots, we expect that real matrices will have complex eigenvalues and eigenvectors. Consider
01 1 2 A for which det(AI ) ( 1) 10 1 with eigenvalues 1 = i and 2 = i. Corresponding eigenvectors are 11 xx12 and ii Note that these are a complex conjugate pair. Why? More generally: ab A has the eigenvalues aib ba
with the same eigenvectors 11 xx12 and ii 8.10
5 Eigenproblems
Example 5 (Boundary value problem): Consider the boundary value problem on the interval [0, 1] where we look for solutions of dx2 x 0 with the boundaryy()() conditions xx(0)( 1) 0 dt 2 These are called Dirichlet boundary conditions. The eigenvalues in this case 2 + are m = (m) , where m = 1, 2, …
The corresponding eigenvectors are xm(t) = sin( mt). Changing the boundary conditions changes the eigenproblem! (a) Neumann boundary conditions: dx dx 0 dttt01 dt 2 In this case, we find m = (m) , where m = 0, 1, 2, …
The corresponding eigenvectors are xm(t) = cos(mt).
8.11
Eigenproblems
Example 5 (Boundary value problem): (c) Periodic boundary conditions: dx dx x(()0)(x ()1) and dttt01 dt 2 In this case, we find m = (2m) , where m = …, 2, 1, 0, 1, 2, …
The corresponding eigenvectors are xm(t) = exp(i 2mt). Each of these three solutions to an eigenproblem is a complete orthogonal basis for 2(0,1). Connection to an eigenproblem is usually the key to demonstrating that a basis is complete.
8.12
6 Completeness
Demonstrating that the eigenvectors of an eigenproblem are complete is generally a difficult problem… …and impppportant in applications Remark 1: For n n matrix problems, we must determine if a matrix is defective. If it is, we must add extra vectors to the span the space. Remark 2: Similar but more complicated issues arise in infinite- dimensional problems.
8.13
Completeness
Why is this so important?
Original problem input: Decompppgose the input into eigenvectors time, space, pixel number, sample number A complicated transformation A much simpler transformation
Return to the original space
Original problem output 8.14
7 Completeness
A key engineering example: The Fourier transform!
Input time signal Input Fourier transform frequency signal A time-dependent filter: xout()tGttxtdt ( ) in ( ) Frequency domain: xGxout() () in ()
Ouput Inverse Fourier transform Ouput time signal frequency signal For this approach to work, we must be able to decompose any signal. The space of eigenmodes must be complete or easily completable 8.15
Example Applications
Example 1 (Markov process): (From slide 2.24) An MRI scan has detected a cancerous growth in which 30% of the cells are cancerous (state 1), 20% are abnormal (state 2), and 50% are normal (state 3). Therapy begins in January , and it is known that the transition probabilities from month to month are given by 0.8 0.1 0 P 0.1 0.7 0.1 0.1 0.2 0.9 What is the best that the treatment can do? How many months should the treatment go on? What fraction of the cells are cancerous when the treatment ends? Solution: To find the solution, we first note that: (1) all orthogonal matrices must have at least one eigenvalue whose value is equal to 1 and (2) no eigenvalues can have a real part whose values are greater than 1.
8.16
8 Example Applications
Solution (continued): We first find the eigenvector that the corresponds to = 1. Our matrix becomes 0.2 0.1 0 1/ 8 PI0.1 0.3 0.1 x 2 / 8 1 0.1 0.2 0.1 5 / 8 where we look for the solution that satisfies x11 + x21 + x31 = 1. The characteristic equation for P is 48 188 24 230 100 100 10 Dividing by 1 – , we find the reduced equation 48 14 2 00.8,0.6 100 10 23 The corresponding eigenvectors may be usefully chosen: T T x2 = [1/8 0 1/8] , x3 = [1/8 1/4 1/8]
8.17
Example Applications
Solution (continued): Note that the elements of x2 and x3 sum to zero. [This will be true for the eigenvectors of any stochastic matrix who have eigenvalues whose real parts are less than 1. The proof is left to students.]
As we iterate the treatments, we converge to x1. We conclude that at best we will converge to 62.5% normal cells, 25% pre-cancerous cells and 12.5% cancerous cells … not so good.
T To see what happens with our initial condition, xin = [0.3 0.2 0.5] , we must decompose this state into eigenvectors. We find 0.3 0.125 0.125 0.125 020.20 025.25 112.20 0 002.20 025.25 orxx 112.20 x 02.2 x i1in 12 2 3 0.5 0.625 0.125 0.125 So, we find that x after n months of treatment is given by nn xxn 121.2(0.8) x 0.2(0.6) x 3
8.18
9 Example Applications
Solution (continued): How many months should we treat? That depends on how close we want to get to the limit of 12.5% cancerous cells. Suppose we want to get to 15%, then the contribution from x2 to the cancerous cells will be about 2 .5% since the contribution from x3 decays more quickly. We estimate n = ln(0.025/0.15)/ln(0.8) = 8.03. If we go for 9 months, then the fraction of cancerous cells is 0.125[1 + 1.2(0.8)9 + 0.2(0.6)9] = 0.125[1 + 0.16 + 0.002] = 0.145, which meets our requirements.
8.19
Example Applications
Example 2 (Source-Free Circuits):
Source-free RLC Circuit The c ircu it tha t we w ill st ud y i s I0 shown at the left. From Kirchhoff’s voltage law, we have I V C dI dV 1 RI L V0, C I dtC dt C We now find d I RL/1/ L I dt VVCC1/C 0 This equation is a second-order differential equation. In matrix language, we may write dx I RL/1/ L AAxx,where and dt VC 1/C 0 8.20
10 Example Applications
Example 2 (continued): Design goal: Find a resistance that damps the circuit as rapidly as possible. We search for solutions of this equations in the form: x(t) = x(0)exp(t). Our equation for now becomes (A – )x = 0 and the characteristic equation is (/)RL 1/ L R 1 det(AI ) 2 0 1/C LLC which has the eigenvalues 1/2 1/2 RR2211 RR 12 22 , 24L L LC 24 L L LC There are three cases: (a) Underdamped: R < 2(L /C)1/2 complex eigenvalues 1/2 1/2 RR1122 RR 12 ii 22, 24L LCL 24 L LCL 8.21
Example Applications
Example 2 (continued): There are three cases: (a) Underdamped: R < 2(L /C)1/2 complex eigenvalues 1/2 1/2 RR1122 RR 12 ii 22, 24L LCL 24 L LCL with eigenvectors 11 1 1 221/2 1/2 xx12RLR, RLR L L 2 ii 1 24CC 24
These must come in complex conjugate pairs for real solutions
8.22
11 Example Applications
Example 2 (continued): There are three cases: (a) Underdamped: We find
xx()ttt 11 exp( ) x 2 exp( 2 ) with I(0)VL (0) / 1 C 12 We may rewrite this solution as I(tA ) exp t cos tB exp( t )sin( t )
VtC () ( A B ) L cos t ( A B ) L sin( t )
with 1/2 RR1 2 , 2 24LLCL
If R is small, then the circuit will oscillate for a long time
8.23
Example Applications
Example 2 (continued): There are three cases: (b) Overdamped: R > 2(L /C)1/2 real eigenvalues 1/2 1/2 RR2211 RR 12 22 , 24LLLC 24 LLLC 1/2 When RLC ( / ) , then 12 1/ RC and RL / One damping rate becomes small and the other becomes large T T As before, we have x1 = [1 2L] , x2 = [1 1L] , and
xx(ttt ) 11 exp( ) x 2 exp( 2 ) with I(0)VL (0) / IVL (0) (0) / 12CC, 12 12 If R is large, then the circuit takes a long time to damp.
8.24
12 Example Applications
Example 2 (continued): There are three cases: (c) Critically damped: R = 2(L /C)1/2 only one eigenvalue, T 1 = R /2L and one eigenvector x1 = [1 R /2] What do we do to deal with “arbitrary” initial conditions? We have to expand the function space!
xx()ttt1121 exp( ) x exp( t ) I(0) 1 R withx ,VI (0) (0) 2 C VC (0) L 2 Critical damping is what we want
8.25
Example Applications
Example 2 (continued): There are three cases: A numerical example
[L = 4 H, C = 0.25 F, R = 4, 8, 16 , I(0) = 1 A, VC (0) = 4 V ]
8.26
13 Example Applications
Example 2 (continued): d x The evolution of any linear passive circuit may be written: Ax, dt where x is a vector of n independent circuit components and A is an n n matrix. The general solution may be written m d j l xx jljlttexp( j ) jl10 where j is the j-th eigenvalue and dj is the defect of that eigenvalue; m n is the number of independent eigenvectors and the x are the jd j eigenvectors; the other xjl will be determined by the initial conditions.
8.27
Example Applications
Example 3 (Optimal signaling): A distorted signal and white Gaussian noise: Assume a transmitter sends an on-off-keyed (OOK) signal with a fixed total energy for a 1-bit and no energy for a 0-bit [0, T ]. The medium distorts and attenuates the signal and adds white Gaussian noise. What is the best waveform to use for the transmitted signal? Solution: The signal out may be written in terms of the signal in as T s ()tRttstdtnt (, ) ( ) () out0 in where R takes into account the effect of the distorting medium and n is the noise source. Suppose we can find a complete set of orthonormal 2 eigenmodes for R in (0, T ), em(t), 0, 1, 2, … such that T et() Rttetdtnt (, ) ( ) () mm0 m
and T ee,()() etetdt lm0 l m lm 8.28
14 Example Applications
Example 3 (continued): If we now write sin ()taet mm (), m0 where ase mm in , , then we find sout ()taet mmm (), m0 If we neglect noise, the goal is to maximize 22 Paoutmmwith the constraint Pa in m constant mm00
That may be done by choosing a mm . Since the noise is white Gaussian, the contributions of the noise to each component are equal and uncorrelated. Hence, the noise does not affect this choice. An important special case: Rtt(, ) Rt ( t ) The Fourier basis is the correct basis, and we find that the optimal choice
is sin ()tRt () This is the Wiener solution to optimal filtering problem except that we are matching the input to the filter. 8.29
Example Applications
Example 4 (waveguides): We consider a rectangular metal waveguide. For TM (transverse magnetic waves), the z-component of the electric field may be y z Exyzt( , , , ) ( xy , )exp( itiz ) b written as z x where obeys the wave equation 22 2 (,xy ) (, xy ) 2 a 222 (,xy ) 0 xyc This equation may be solved using separation of variables. We let (,x yXxYy ) ()() and we obtain dXx22() dYy () 22Xx() 0, Yy () 0 dx22xy dy
with the boundary conditions X(0) = X(a) = 0 and Y(0) = Y(b) = 0
8.30
15 Example Applications
Example 4 (waveguides): We use a sine expansion, and we find
xmmam/, 0,1,; yn nb / so that (x ,yamxanyb ) mn sin( / )sin( / ) mn1, 1 where the amn are determined by the input field. The allowed values of are given by 2 22 2 mn 2 ca b
8.31
16