Lecture 4 spectral method Basic perturbation Theory
related Aswe willsee in plantedcliqueandmany planted problems the about The first few eigenvectors of FLEX Contain information the undenYL planted structures Hee x denotethedatamatrix ingeneral e digue g Xcouldbe adjacencymath A in eg planted ELA STERN 1 I planted clique But we only observeX and do not know EEX and hope that these we may compute the Kay eigenvectors of are closeto leady eigenvectors of EE X
Write X FLEXI t X EEX We willshow that the error of estray the Kady eigenvectorsof FIX canbe bounded in terms of the size ofperturbation X ELXI
Reviewof Linear algebra suppose X EIR is symmetric matrix
Def eigenvalue and eigenvector C and ElRh is an The pain X V with A IR V eigen pair of X if Xv Xv j eigenvector eigenvalue X we can order the eigenvalues of by their sizes XlZX2Z Zan with the conespondy eigenvectors Vi V2 Vn Men we can write the egadeapos of X as X VAVT di Vivi
where u k Vn
a t I in Also notethat rank X h 7 exactly r nonzerodi's
Howabout rectangle matrix
Def singular value decomposition I
suppose XE 112mm is The singular value decopositionof X X U 2 VT E Gi Ui Vi hthogoygf.mn U Elli Uh EIR II gr t K I n Vn Ellen U leftsingular vector vi right Giza singular value and XTX We can obtain SVD of X from EVD of XXT Indeed 2 T XXT U UT XT X 2 so si xxT Fixx
Matrix norms
a Frobeniusnorm View as mn dim vector anddefine llxlt EHveccxslk fz.FI operator'onorm view as linear operator X IR H Hp IRM 11.11g with operator norm
HXHp q sup 11 Xv Hq 452 Forthis course the most relevant one is p L 2
H X Hop E 11 112 32 known as spectral norm as sometimes I'll write 11 12 22 simply 1 1112
Remark HXHop 6maxlX UIVIINE Pt HXHop2 sup INUK sup HE 11412 1 1142 2 2 Ui'sare 7 sup city orthogonal HUH Foi 6max4X H Hop is a norm and HXHop NXTHop
HXYHop E Hilltop 11THop If X X is a vector then 11 119 11 112
11 Hop is orthogonal invariant ie forany Reocn RR4342 2 and R C 0 m wehave
HR XRHop HXHop
Columns If X Xi Xn has orthogonal rows then 11 Followsfrom Gmax X L 11 op L
Remark
Note Matrix Inner product
x 17 E Vecchi veil's Xijtij Tr X'T
Thus LX nut sup CHA HXHop Gmax X sup Hulk HIEI AHAH rankCAFI Likewise If X is symmetric then minimax.im SupcXiWT7 llXfop 5maxlX1 nixpdmaxlXII supkxvrlVlkUNT 1 NIKI I U ter perturbation eigenstructures of whereVE Ui f Xiao i perturbation Assume X and f Xt Z whee Z is FX and close Question Are eigenvalues and eigenvectors of X T when Z is small
Answet No in general
consider and Xe
Then d CX 421 1 0 det XI Xe It AllXetF.dzHe Je deff A E Moregenerally consider
too Imax
Ail Xl 0 but Xi Xe f 1 E'T e2 here denotes
the imiginay point NotLipschitzcontinuous in E
Even in the symmetric matrix case eigenvector perturbationsmay fail dramatically Ii and t i det f HEE del f ex 1 The eigenvalues arethe same 112171 1 E AIX Nlt HE 1121 1 apart 1lover the eigenvectors are far KHE WH f WH Y but hat f between the eigenvalues a Spectre Lessie we need separation leigen gap perturbatinboundfreigenvalues
Let X f Z be real s c matrices in RMn
Let f Xt Z wehave that VVT di IX t Antz XCX t if CE t 1142 1 histhe L X vV'T CZ wht leagejuvofX C Xt't wt E 87k ACT saw E EYE W t
di X tdi IZ
ANH Edith dik e att 71211 INN AHH E Mah Hilal Hnat op Moregenerally we hue the foamy Weyl's inequality Hw
1hm Weyl's inequality LidsKii's inquality
x E 11211 Ail dik op perturbation bounds for eigenspaces matrices in Let X 7 Z be real symmetric IR
and f Xt Z In Suppose X Idi Vi Vi with dizdz f Pi ViVIT with A Ipa 2 2 Pn
bound UE u We want to prove a perturbation for and r E v and more generally for U Un Un
and vi Vz Vr
Caveat o u and V are determined upto theirsign
Similarly V and V are determinedupto orthogonal transformation
There are two possible workarounds Consider spectral distance dslay E Min llntsvlk SETH F.nl J2 2050 d2 25m20s 251nF Moregenerally dslUYE inf HU VRHop REOCH
Consider projectiondistance 2 since Hunt w HE 2 l LUN
In the generalcase consider 11001WTVF or Hult VVTHop
Thm Davis Kahan Let coso KUN I Then spectrol 112419 gap since s ma gap role PI WLOG assure pizdz otherwise switch of X and T Let us start from the eigenvalue equation Xu Jiu and TV pi V
Denote Us uz un CIR then uix IYE.lx IIII.tt IIII
Hence Viv Utter put UI 2 n Ruin in Join utter
Vivi 7 VIZ n
Ta l oy the Il Hop norm on both sides gives
Hop Ig Hop Hop HUI Hutter HE a HUI ViHz 11211op P1 da Finallynote that vilkuih vifI yut.lv HUIhlli 2 Te d Lu v SIRO R h al 1 11 1112112 ll orthogonal invariant 11 11 1112 12 any norm ll Hope generally for 11211 VIL E HUI matar rtJ
cased I AHHH I spectralgap case2 T T T Prez Phi Nr Ary X
known one can generalize this to singular rectos by a technique as self adjoint dilation
Let X U E VT Y TE TT Xlmth consider plinth Oy Xo J E
and similarly for f
obserlethat Hana III It ri I o III I Thus we can apply Davis Kahan Thm to obtain
25M since smeifu I Yj Hop 2 HEE El mail.to a e2HEIok fix 04411