Analysis Meets Graphs

Analysis meets Graphs

Matthew Begue´

Norbert Wiener Center Department of Mathematics University of Maryland, College Park Why study graphs?

Graph theory has developed into a useful tool in applied mathematics. Vertices correspond to different sensors, observations, or data points. Edges represent connections, similarities, or correlations among those points. Wikipedia graph Graphs in pure mathematics too! Why study Analysis? Why study Analysis?

Real Analysis Complex Analysis Partial Differential Equations Ordinary Differential Equations HARMONIC ANALYSIS Probability Differential Geometry Functional Analysis Compressive Sensing Stochastic Processes Measure Theory Brownian Motion Operator Theory Spectral Theory Mathematical Modeling Numerical Analysis Why study Analysis?

Translation Convolution Modulation Fourier Transform Analysis toolbox

Translation Convolution Modulation Fourier Transform Outline

1 Fourier Transform

2 Modulation

3 Convolution

4 Translation Motivation

In the classical setting, the Fourier transform on R is given by Z ˆf (ξ) = f (t)e−2πiξt dt = hf , e2πiξt i. R

This is precisely the inner product of f with the eigenfunctions of the Laplace operator. Motivation

In the classical setting, the Fourier transform on R is given by Z ˆf (ξ) = f (t)e−2πiξt dt = hf , e2πiξt i. R

This is precisely the inner product of f with the eigenfunctions of the Laplace operator. Graph Laplacian

Deﬁnition The pointwise formulation for the Laplacian acting on a function f : V → is R X ∆f (x) = f (x) − f (y). y∼x

For a ﬁnite graph, the Laplacian can be represented as a matrix. Let D denote the N × N degree matrix, D = diag(dx ). Let A denote the N × N adjacency matrix,

1, if x ∼ x A(i, j) = i j 0, otherwise.

Then the unweighted graph Laplacian can be written as

L = D − A. Spectrum of the Laplacian

L is a real symmetric matrix and therefore has nonnegative N−1 eigenvalues {λk }k=0 with associated orthonormal eigenvectors N−1 {ϕk }k=0 . If G is ﬁnite and connected, then we have

0 = λ0 < λ1 ≤ λ2 ≤ · · · ≤ λN−1. √ Easy to show that ϕ0 ≡ 1/ N. Data Sets - Minnesota Road Network

50 50 50

49 49 49

48 48 48

47 47 47

46 46 46

45 45 45

44 44 44

43 43 43 −98 −97 −96 −95 −94 −93 −92 −91 −90 −89 −98 −97 −96 −95 −94 −93 −92 −91 −90 −89 −98 −97 −96 −95 −94 −93 −92 −91 −90 −89

(a) λ3 (b) λ4 (c) λ5

50 50 50

49 49 49

48 48 48

47 47 47

46 46 46

45 45 45

44 44 44

43 43 43 −98 −97 −96 −95 −94 −93 −92 −91 −90 −89 −98 −97 −96 −95 −94 −93 −92 −91 −90 −89 −98 −97 −96 −95 −94 −93 −92 −91 −90 −89

(d) λ6 (e) λ7 (f) λ8 Figure: Eigenfunctions corresponding to the ﬁrst six nonzero eigenvalues. Minnesota road graph (2642 vertices) Data Sets - Sierpinski gasket graph approximation

1.8 1.8 1.8

1.6 1.6 1.6

1.4 1.4 1.4

1.2 1.2 1.2

1 1 1

0.8 0.8 0.8

0.6 0.6 0.6

0.4 0.4 0.4

0.2 0.2 0.2

0 0 0 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1

(a) λ3 (b) λ4 (c) λ5

1.8 1.8 1.8

1.6 1.6 1.6

1.4 1.4 1.4

1.2 1.2 1.2

1 1 1

0.8 0.8 0.8

0.6 0.6 0.6

0.4 0.4 0.4

0.2 0.2 0.2

0 0 0 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1 −1 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8 1

(d) λ6 (e) λ7 (f) λ8 Figure: Eigenfunctions corresponding to the ﬁrst six nonzero eigenvalues. Level-8 graph approximation to Sierpinski gasket (9843 vertices) Data Sets - Sierpinski gasket graph approximation

0.02 0.01 0.015

0.015 0.005 0.01 0.01 0.005 0 0.005 0 0 −0.005 −0.005 −0.005 −0.01 −0.01 −0.01 −0.015 −0.015 −0.015

−0.02 −0.02 −0.02 2 2 2

1.5 1 1.5 1 1.5 1 0.5 0.5 0.5 1 1 1 0 0 0 0.5 0.5 0.5 −0.5 −0.5 −0.5 0 −1 0 −1 0 −1

(a) λ3 (b) λ4 (c) λ5

0.02 0.02 0.02

0.015 0.015 0.015

0.01 0.01 0.01

0.005 0.005 0.005

0 0 0

−0.005 −0.005 −0.005

−0.01 −0.01 −0.01

−0.015 −0.015 −0.015

−0.02 −0.02 −0.02 2 2 2

1.5 1 1.5 1 1.5 1 0.5 0.5 0.5 1 1 1 0 0 0 0.5 0.5 0.5 −0.5 −0.5 −0.5 0 −1 0 −1 0 −1

(d) λ6 (e) λ7 (f) λ8 Figure: Eigenfunctions corresponding to the ﬁrst six nonzero eigenvalues. Level-8 graph approximation to Sierpinski gasket (9843 vertices) Graph Fourier Transform

Deﬁnition The graph Fourier transform is deﬁned as

N ˆ X ∗ f (λl ) = hf , ϕl i = f (n)ϕl (n). n=1

Notice that the graph Fourier transform is only deﬁned on values of σ(L). The inverse Fourier transform is then given by

N−1 X ˆ f (n) = f (λl )ϕl (n). l=0 Outline

1 Fourier Transform

2 Modulation

3 Convolution

4 Translation Graph Modulation

In Euclidean setting, modulation is multiplication of a Laplacian eigenfunction.

Deﬁnition

For any k = 0, 1, ..., N − 1 the graph modulation operator Mk , is deﬁned as √ (Mk f )(n) = Nf (n)ϕk (n).

On R, modulation in the time domain = translation in the frequency domain ˆ Mdξf (ω) = f (ω − ξ). Example - Classical Setting Example - Movie

G =SG6 ˆ f (λl ) = δ2(l) =⇒ f = ϕ2.

0.02

0.015

0.01

0.005

−0.005

−0.01 2

1.5 1 0.5 1 0 0.5 −0.5 0 −1 Example - Movie

G =Minnesota ˆ f (λl ) = δ2(l) =⇒ f = ϕ2.

43 −98 −97 −96 −95 −94 −93 −92 −91 −90 −89 Outline

1 Fourier Transform

2 Modulation

3 Convolution

4 Translation Graph Convolution - Motivation and Deﬁnition

Classically, for signals f , g ∈ L2(R) we deﬁne the convolution as Z f ∗ g(t) = f (u)g(t − u) du. R However, there is no clear analogue of translation in the graph setting. So we exploit the property

(fd∗ g)(ξ) = ˆf (ξ)gˆ(ξ),

and then take inverse Fourier transform.

Deﬁnition For f , g : V → R, we deﬁne the graph convolution of f and g as

N−1 X ˆ f ∗ g(n) = f (λl )gˆ(λl )ϕl (n). l=0 Outline

1 Fourier Transform

2 Modulation

3 Convolution

4 Translation Graph Translation

2 For signal f ∈ L (R), the translation operator, Tu, can be thought of as a convolution with δu. On R we can calculate ˆ R −2πikx −2πiku ∗ δu(k) = δu(x)e dx = e (= ϕ (u)). R k Then by taking the convolution on R we have Z Z ˆ ˆ ˆ ∗ (Tuf )(t) = (f ∗δu)(t) = f (k)δu(k)ϕk (t) dk = f (k)ϕk (u)ϕk (t) dk R R

Deﬁnition

For any f : V → R the graph translation operator, Ti , is deﬁned to be

√ √ N−1 X ˆ ∗ (Ti f )(n) = N(f ∗ δi )(n) = N f (λl )ϕl (i)ϕl (n). l=0 Example - Classical Setting

Translation in the time domain = Modulation in the frequency domain Example - Movie

G =Minnesota f = 11

1 1 0.8

0.8 0.6

0.4 0.6

0.2 0.4 0 50 0.2

49 0

48 −0.2 47

−0.4 46

−0.6 45

−0.8 44

−1 43 0 1 2 3 4 5 6 −98 −97 −96 −95 −94 −93 −92 −91 −90 −89 Example - Movie

G =Minnesota ˆ −5λl f (λl ) = e

0.12

0.05 0.1 0

−0.05 0.08

−0.1

50 0.06

0.04 48

47 0.02 46

45 0

44 −90 −89 −92 −91 −94 −93 −0.02 43 −96 −95 0 1 2 3 4 5 6 7 −98 −97 Example - Movie

G =SG6 ˆ −5λl f (λl ) = e

0.8

0.1 0.6 0.08

0.06 0.4 0.04

0.02 0.2

0 0 −0.02

−0.04 −0.2 −0.06

−0.08 −0.4 −0.1 2 −0.6 1.5 1 0.5 1 −0.8 0 0.5 −0.5 −1 0 −1 0 1 2 3 4 5 6 Example - Movie

G =Minnesota ˆf ≡ 1

10 1

0 0.8

−5 0.6 50

0.4 49

0.2 48

47 −0.2

46 −0.4

45 −0.6

44 −0.8

−1 43 −93 −92 −91 −90 −89 0 1 2 3 4 5 6 −98 −97 −96 −95 −94 Nice Properties of Graph Translation

Theorem For any f , g : V → R and i, j ∈ {1, 2, ..., N} then 1 Ti (f ∗ g) = (Ti f ) ∗ g = f ∗ (Ti g).

2 Ti Tj f = Tj Ti f . Not-so-nice Properties of Translation operator

The translation operator is not isometric

kTi f k`2 6= kf k`2

N In general, the set of translation operators {Ti }i=1 do not form a group like in the classical Euclidean setting.

Ti Tj 6= Ti+j

In general, Can we even hope for Ti Tj = Ti•j for some semigroup operation, • : {1, ..., N} × {1, ..., N} → {1, ..., N}?

Theorem (B. & O.)

Given a graph, G, with eigenvector matrix Φ = [ϕ0| · · · |ϕN−1]. Graph translation on G is a semigroup, i.e. Ti Tj = Ti•j for some semigroup√ operator • : {1, ..., N} × {1, ..., N} → {1, ..., N}, only if Φ = (1/ N)H, where H is a Hadamard matrix. Not-so-nice Properties of Translation operator

The translation operator is not isometric

kTi f k`2 6= kf k`2

N In general, the set of translation operators {Ti }i=1 do not form a group like in the classical Euclidean setting.

Ti Tj 6= Ti+j

In general, Can we even hope for Ti Tj = Ti•j for some semigroup operation, • : {1, ..., N} × {1, ..., N} → {1, ..., N}?

Theorem (B. & O.)

The translation operator is not isometric

kTi f k`2 6= kf k`2

N In general, the set of translation operators {Ti }i=1 do not form a group like in the classical Euclidean setting.

Ti Tj 6= Ti+j

In general, Can we even hope for Ti Tj = Ti•j for some semigroup operation, • : {1, ..., N} × {1, ..., N} → {1, ..., N}?

Theorem (B. & O.)

The translation operator is not isometric

kTi f k`2 6= kf k`2

N In general, the set of translation operators {Ti }i=1 do not form a group like in the classical Euclidean setting.

Ti Tj 6= Ti+j

In general, Can we even hope for Ti Tj = Ti•j for some semigroup operation, • : {1, ..., N} × {1, ..., N} → {1, ..., N}?

Theorem (B. & O.)

With these time-frequency operators generalized to vertex-frequency operators, we are able to convert many nice results and theories from harmonic analysis to the graph setting. Wavelets and Wavelet Transform Windowed/Short-Time Fourier Transform (STFT) Windowed Fourier Frames Why should we care? Why should we care? Thank you!