Maths for Signals and Systems Linear Algebra in Engineering Lecture 12, Friday 6th November 2015 DR TANIA STATHAKI READER (ASSOCIATE PROFFESOR) IN SIGNAL PROCESSING IMPERIAL COLLEGE LONDON Stochastic matrices • Consider a matrix 퐴 with the following properties: All entries are positive and real. The elements of each column or each row or both each column and each row add up to 1. Based on the above a matrix that exhibits the above properties will have all entries ≤ 1. It is square. • This is called a stochastic matrix. • Stochastic matrices are also called Markov, probability, transition, or substitution matrices. • The entries of a stochastic matrix usually represent a probability. • Stochastic matrices are widely used in probability theory, statistics, mathematical finance and linear algebra, as well as computer science and population genetics. Stochastic matrices. Types. • There are several types of stochastic matrices: . A right stochastic matrix is a matrix of nonnegative real entries, with each row’s elements summing to 1. A left stochastic matrix is a matrix of nonnegative real entries, with each column’s elements summing to 1. A doubly stochastic matrix is a matrix of nonnegative real entries with each row’s and each column’s elements summing to 1. • A stochastic matrix often describes a so called Markov chain 푋푡 over a finite state space 푆. • Generally, an 푛 × 푛 stochastic matrix is related to 푛 “states”. • If the probability of moving from state 푖 to state 푗 in one time step is 푃푟 푗 푖 = [푝푖푗], th th the stochastic matrix 푃 is given by using 푝푖푗 as the 푖 row and 푗 column element: 푝11 푝12 … 푝1푛 푝21 푝22 … 푝2푛 푃 = ⋮ ⋮ ⋱ ⋮ 푝푛1 푝푛2 … 푝푛푛 • Depending on the particular problem the above matrix can be formulated in such a way so that is either right or left stochastic. Products of stochastic matrices. Stochastic vectors. • An example of a left stochastic matrix is the following: 0.1 0.01 0.3 퐴 = 0.2 0.99 0.3 0.7 0 04 • You can prove that if 퐴 and 퐵 are stochastic matrices of any type then 퐴퐵 is also a stochastic matrix of the same type. • Consider two left stochastic matrices 퐴 and 퐵 with elements 푎푖푗 and 푏푖푗 respectively, and 퐶 = 퐴퐵 with elements 푐푖푗. • Let us find the sum of the elements of the 푗th column of 퐶: 푛 푛 푛 푛 푛 푛 푛 푖=1 푐푖푗 = 푖=1 푘=1 푎푖푘푏푘푗 = 푘=1 푖=1 푎푖푘푏푘푗 = 푘=1 푏푘푗 푖=1 푎푖푘 = 1 ∙ 1 = 1 • Based on the above the power of a stochastic matrix is a stochastic matrix. • I am interested in the eigenvalues and eigenvectors of a stochastic matrix. • Let’s call a vector with real, nonnegative entries 푝푘, for which all the 푝푘 add up to 1, a stochastic vector. For a stochastic matrix, every column or row or both is a stochastic vector. Stochastic matrices and their eigenvalues • I would like to prove that 휆 = 1 is always an eigenvalue of a stochastic matrix. • Consider again a left stochastic matrix 퐴. • Since the elements of each column of 퐴 add up to 1, the elements of each row of 퐴푇 should add up to 1. Therefore: 1 1 1 퐴푇 1 = 1 = 1 ∙ 1 ⋮ ⋮ ⋮ 1 1 1 • Therefore, 1 is an eigenvalue of 퐴푇 . The eigenvalues of 퐴푇 are obtained through the equation det 퐴푇 − 휆퐼 = 0. But: det 퐴푇 − 휆퐼 = det 퐴푇 − 휆퐼푇 = det 퐴 − 휆퐼 푇 = det (퐴 − 휆퐼) • Hence, the eigenvalues of 퐴 and 퐴푇 are the same, which implies that 1 is also an eigenvalue of 퐴. • Since det 퐴 − 퐼 = 0, the matrix 퐴 − 퐼 is singular, which means that there is a vector 푥 for which 퐴 − 퐼 푥 = 0 ⇒ 퐴푥 = 푥 • A vector of the null space of 퐴 − 퐼 is the eigenvector of 퐴 that corresponds to eigenvalue 휆 = 1. Stochastic matrices and their eigenvalues cont. • I would like now to prove that the eigenvalues of a stochastic matrix have magnitude smaller or equal to 1. 푣1 푣2 • Assume that 푣 = ⋮ is an eigenvector of a stochastic matrix 퐴 with an 푣푛 푛 푛 푛 푛 푛 eigenvalue 휆 > 1. Then 퐴 푣 = 휆 푣 implies 푗=1[퐴 ]푖푗푣푗 = 휆 푣푖 • 휆푛 has exponentially growing length for 푛 ⇢ ∞. The maximum value that 푛 푛 푛 푛 푛 [퐴 ]푖푗푣푗 can take is 푣max ∙ max 퐴 푖푗 ≤ 푣max ∙ 푛 since the entries of 퐴 푗=1 푖 푗=1 are ≤ 1 . 푛 푛 푛 • The relationship 푗=1[퐴 ]푖푗푣푗 = 휆 푣푖 must be valid for any 푛. There is always a 푛 푛 푛 minimum 푛 for which 푗=1[퐴 ]푖푗푣푗 ≤ 푣max∙ 푛 < 휆 푣푖 if 휆 > 1. • Based on the above, the assumption of an eigenvalue being larger than 1 can not be valid. Application of stochastic matrices 푘 • Consider again the system described by an equation of the form 푢푘 = 퐴 푢0, where 퐴 is now a stochastic matrix. 푘 푘 푘 • Previously, we managed to write 푢푘 = 퐴 푢0 = 푐1휆1 푥1 + 푐2휆2 푥2 + ⋯ where 휆푖 and 푥푖 are the eigenvalues and eigenvectors of matrix 퐴, respectively. • Note that the above relationship requires a complete set of eigenvectors. • If 휆1 = 1 and 휆푖 < 1, 푖 > 1 then the steady state of the system is 푐1푥1 (which is part of the initial condition 푢0). • I will use an example where 퐴 is a 2 × 2 matrix. Generally, an 푛 × 푛 stochastic matrix is related to 푛 “states”. Assume that the 2 “states” are 2 UK cities. • I take London and Oxford. I am interested in the population of the two cities and how it evolves. • I assume that people who inhabit these two cities move between them only. 푢ox 0.9 0.2 푢ox = 푢lon 푡=푘+1 0.1 0.8 푢lon 푡=푘 • It is now obvious that the column elements are positive and also add up to 1 because they represent probabilities. Application of stochastic matrices (cont.) 푢ox 0 푢ox 0.9 0.2 0 200 • I assume that = . Then = = . 푢lon 푡=푘=0 1000 푢lon 푘=1 0.1 0.8 1000 800 Problem: What is the population of the two cities after a long time? Solution: 0.9 0.2 Consider the matrix . The eigenvalues are 휆 = 1 and 휆 = 0.7. 0.1 0.8 1 2 (Notice that the second eigenvalue is found by the trace of the matrix.) 2 −1 The eigenvectors of this matrix are 푥 = and 푥 = 1 1 2 1 푢ox 푘 2 푘 −1 2 푘 −1 = 푐1휆1 + 푐2휆2 = 푐1 + 푐20.7 푢lon 푘 1 1 1 1 푢ox 0 I find 푐1, 푐2 from the initial condition = 푢lon 푡=푘=0 1000 0 2 −1 1000 2000 =푐 + 푐 and therefore, 푐 = and 푐 = 1000 1 1 2 1 1 3 2 3 Application of stochastic matrices (cont.) • Stochastic models facilitate the modeling of various real life engineering applications. • An example is the modeling of the movement of people without gain or loss: total number of people is conserved. .