Lecture 4 spectral method Basic perturbation Theory

related Aswe willsee in plantedcliqueandmany planted problems about The first few eigenvectors of FLEX Contain information the undenYL planted structures Hee x denotethedatamatrix ingeneral digue g Xcouldbe adjacencymath in eg planted ELA STERN 1 planted clique But only observeX and do not know EEX and hope that these we may compute the Kay eigenvectors of are closeto leady eigenvectors of EE X

Write X FLEXI t X EEX We willshow that the error of estray the Kady eigenvectorsof FIX canbe bounded in terms of the size ofperturbation X ELXI

Reviewof Linear algebra suppose X EIR is symmetric matrix

Def eigenvalue and eigenvector C and ElRh is an The pain X V with A IR V eigen pair of X if Xv Xv eigenvector eigenvalue X we can order the eigenvalues of by their sizes XlZX2Z Zan with the conespondy eigenvectors Vi V2 Vn Men we can write the egadeapos of X as X VAVT di Vivi

where k Vn

a t I in Also notethat rank X h 7 exactly r nonzerodi's

Howabout rectangle matrix

Def singular value decomposition I

suppose XE 112mm is The singular value decopositionof X X U 2 VT E Gi Ui Vi hthogoygf.mn U Elli Uh EIR II gr t K I n Vn Ellen U leftsingular vector vi right Giza singular value and XTX We can obtain SVD of X from EVD of XXT Indeed 2 T XXT U UT XT X 2 so si xxT Fixx

Matrix norms

a Frobeniusnorm View as mn dim vector anddefine llxlt EHveccxslk fz.FI operator'onorm view as linear operator X IR H Hp IRM 11.11g with operator norm

HXHp q sup 11 Xv Hq 452 Forthis course the most relevant one is p L 2

H X Hop E 11 112 32 known as spectral norm as sometimes I'll write 11 12 22 simply 1 1112

Remark HXHop 6maxlX UIVIINE Pt HXHop2 sup INUK sup HE 11412 1 1142 2 2 Ui'sare 7 sup city orthogonal HUH Foi 6max4X H Hop is a norm and HXHop NXTHop

HXYHop E Hilltop 11THop If X X is a vector then 11 119 11 112

11 Hop is orthogonal invariant ie forany Reocn RR4342 2 and R C 0 m wehave

HR XRHop HXHop

Columns If X Xi Xn has orthogonal rows then 11 Followsfrom Gmax X L 11 op L

Remark

Note Matrix Inner product

x 17 E Vecchi veil's Xijtij Tr X'T

Thus LX nut sup CHA HXHop Gmax X sup Hulk HIEI AHAH rankCAFI Likewise If X is symmetric then minimax.im SupcXiWT7 llXfop 5maxlX1 nixpdmaxlXII supkxvrlVlkUNT 1 NIKI I U ter perturbation eigenstructures of whereVE Ui f Xiao i perturbation Assume X and f Xt Z whee Z is FX and close Question Are eigenvalues and eigenvectors of X T when Z is small

Answet No in general

consider and Xe

Then d CX 421 1 0 det XI Xe It AllXetF. Je deff A E Moregenerally consider

too Imax

Ail Xl 0 but Xi Xe f 1 E'T e2 here denotes

the imiginay point NotLipschitzcontinuous in E

Even in the symmetric matrix case eigenvector perturbationsmay fail dramatically Ii and t i det f HEE del f ex 1 The eigenvalues arethe same 112171 1 E AIX Nlt HE 1121 1 apart 1lover the eigenvectors are far KHE WH f WH but hat f between the eigenvalues a Spectre Lessie we need separation leigen gap perturbatinboundfreigenvalues

Let X f Z real s c matrices in RMn

Let f Xt Z wehave that VVT di IX t Antz XCX t if CE t 1142 1 histhe L X vV'T CZ wht leagejuvofX C Xt't wt E 87k ACT saw E EYE W t

di X tdi IZ

ANH Edith dik e att 71211 INN AHH E Mah Hilal Hnat op Moregenerally we hue the foamy Weyl's inequality Hw

1hm Weyl's inequality LidsKii's inquality

x E 11211 Ail dik op perturbation bounds for eigenspaces matrices in Let X 7 Z be real symmetric IR

and f Xt Z In Suppose X Idi Vi Vi with dizdz f Pi ViVIT with A Ipa 2 2 Pn

bound u We want to prove a perturbation for and r E v and more generally for U Un Un

and vi Vz Vr

Caveat u and V are determined upto theirsign

Similarly V and V are determinedupto orthogonal transformation

There are two possible workarounds Consider spectral distance dslay E Min llntsvlk SETH F.nl J2 2050 d2 25m20s 251nF Moregenerally dslUYE inf HU VRHop REOCH

Consider projectiondistance 2 since Hunt w HE 2 l LUN

In the generalcase consider 11001WTVF or Hult VVTHop

Thm Davis Kahan Let coso KUN I Then spectrol 112419 gap since s ma gap role PI WLOG assure pizdz otherwise switch of X and T Let us start from the eigenvalue equation Xu Jiu and TV pi V

Denote Us uz un CIR then uix IYE.lx IIII.tt IIII

Hence Viv Utter put UI 2 n Ruin in Join utter

Vivi 7 VIZ n

Ta l oy the Il Hop norm on both sides gives

Hop Ig Hop Hop HUI Hutter HE a HUI ViHz 11211op P1 da Finallynote that vilkuih vifI yut.lv HUIhlli 2 d Lu v SIRO R h al 1 11 1112112 ll orthogonal invariant 11 11 1112 12 any norm ll Hope generally for 11211 VIL E HUI matar rtJ

cased I AHHH I spectralgap case2 T T T Prez Phi Nr Ary X

known one can generalize this to singular rectos by a technique as self adjoint dilation

Let X U E VT Y TE TT Xlmth consider plinth Oy Xo J E

and similarly for f

obserlethat Hana III It ri I o III I Thus we can apply Davis Kahan Thm to obtain

25M since smeifu I Yj Hop 2 HEE mail.to a e2HEIok fix 04411