Eigenpairs Lecture 1 January 20th, 2021
1 Eigenpairs
2 . 1 Relevant sources for background self- study Ian Goodfellow, Yoshua Bengio and Aaron Courville: Deep Learning MIT Press, 2016. Jure Lescovec, Anand Rajaraman, Jeffrey D. Ullmann Mining of Massive datasets MIT Press, 2016.
2 . 2 Spectral Analysis
3 . 1 Eigenpairs Matrix A has a real λ and a vector e→ s.t. Ae→ = λe→
λ is an eigenvalue and e→ an eigenvector of A. In principle, if |A| = n there should be n such pairs. In practice,
they might not be real, nor ≠ 0
are always costly to nd.
3 . 2 A square matrix A is called positive semide nite iff
T A is symmetric (A = A ) T x→ Ax→ ≥ 0 for any x→
In such case its eigenvalues are non-negative: λi ≥ 0.
3 . 3 Underlying idea, I Eigenvectors, i.e., solutions to Ax = λx describe the direction along which matrix A operates an expansion as opposed to
rotation deformation
3 . 4 Example: shear mapping
[[1, .27], [0, 1]]
x x + 3 y [ ] ⟶ [ 11 ] y y
3 . 5 The blue line is unchanged:
T an [x, 0] eigenvector corresponding to λ = 1
3 . 6 Eigen. of user-activity matrices Eigenvectors are always orthogonal with each other: they describe alternative directions, interpretable as topic Eigenvalues expand one’s af liation to a speci c topic.
3 . 7 Simple all-pairs extraction via Numpy/LA:
√ 2 2 Caveat: e-values come normalized: λ1 + … λn = 1 1 hence, multiply them by √n
3 . 8 def find_eigenpairs(mat): """ Testing the quality of eigenvector/eigenvalue computation by numpy """
n = len(mat)
m = len(mat[0])
eig_vals, eig_vects = la.eig(mat)
# they come in ascending order, take the last one on the right dominant_eig = abs(eig_vals[-1])
print(dominant_eig)
print(eig_vals)
3 . 9 Norms, distances etc.
4 . 1 Euclidean norm √ T √ m 2 ||x|| = x x = ∑i=1 xi
General case: 1 m 1 p p p p p p ||x||p = (|x1| + |x1| + … |xm| ) = (∑i=1 |xi| )
4 . 2 The Frobenius norm || ⋅ ||F extends || ⋅ ||2 to matrices
4 . 3 Normalization The unit or normalized vector of x x 1 u = = ( )x ||x|| ||x||
1. keeps the directon 2. norm is set to 1.
4 . 4 Computing Eigenpairs
5 . 1 With Maths Mx = λx
Handbook solution: solve the equivalent (Mx − λI)x = 0
A non-zero x is associated to a solution of |Mx − λI| = 0
α In Numerical Analysis Θ(n ) methods are available.