Numerical Methods for Spectral Clustering a Spectral Cluster Analysis of the European Air Traffic Network, Using Schur-Wielandt Deflation

DEGREE PROJECT IN TECHNOLOGY, FIRST CYCLE, 15 CREDITS STOCKHOLM, SWEDEN 2020

Numerical Methods for Spectral Clustering A Spectral Cluster analysis of the European Air Traffic Network, using Schur-Wielandt Deflation

JOHAN LARSSON

ISAK ÅGREN

KTH ROYAL INSTITUTE OF TECHNOLOGY SCHOOL OF ENGINEERING SCIENCES Abstract

The Aviation industry is important to the European economy and development, therefore a study of the sensitivity of the European flight network is interesting. If clusters exist within the network, that could indicate possible vulnerabilities or bottlenecks, since that would represent a group of airports poorly connected to other parts of the network. In this paper a cluster analysis using spectral clustering is performed with flight data from 34 different European countries. The report also looks at how to implement the spectral clustering algorithm for large data sets. After performing the spectral clustering it appears as if the European flight network is not clustered, and thus does not appear to be sensitive. Sammanfattning

Flygindustrin ¨arviktig f¨orden europeiska ekonomin och utvecklingen, d¨arf¨or¨aren studie av k¨ansligheten f¨ordet europeiska flygn¨atetintressant. Om det finns kluster i n¨atverket kan det indikera m¨ojligas˚arbarhetereller flaskhalsar, eftersom det skulle representera en grupp flygplatser som ¨ard˚aligtanslutna till andra delar av n¨atver- ket. I denna rapport utf¨orsen klusteranalys med spektralklustering p˚aflygdata fr˚an 34 olika europeiska l¨ander.Rapporten tittar ocks˚ap˚ahur man implementerar spek- tralklustering f¨orstora datam¨angder.Efter att ha utf¨ortspektralklustering verkar det som om det europeiska flygn¨atverket inte ¨arklusterat och d¨arf¨orverkar det inte som att det ¨ark¨ansligt. Acknowledgments

This thesis has been under the supervision of Assoc. Prof. Elias Jarlebring and Assoc. Prof Mattias Sandberg. We are thankful for their guidance and support throughout the entirety of this degree project. Contents

1 Introduction 1 1.1 Background ...... 1 1.1.1 Aviation industry ...... 1 1.1.2 Clustering ...... 1 1.2 Problem ...... 1

2 Clustering 3 2.1 Graphs ...... 3 2.1.1 Methods for Constructing a Graph ...... 4 2.2 K-means ...... 4 2.3 Unnormalized Laplacian ...... 5 2.4 Normalized Laplacian ...... 6 2.5 Algebraic connectivity ...... 6 2.6 Cuts ...... 7 2.7 Perturbation Viewpoint ...... 8 2.8 Spectral Clustering Algorithm ...... 9 2.8.1 Spectral Clustering (RatioCut) ...... 9 2.8.2 Spectral Clustering (NCut) ...... 9

3 Eigenvectors 10 3.1 Eigenvectors and Rayleigh-Ritz Quotients ...... 10 3.2 Single Eigenvalue Algorithms ...... 11 3.2.1 Power Iteration ...... 11 3.2.2 Inverse Iteration ...... 12 3.3 Deflation ...... 12 3.3.1 Wielandt Deflation ...... 13 3.3.2 Hotelling’s Deflation ...... 13 3.3.3 Schur-Wielandt Deflation ...... 14 3.4 LU-Decomposition ...... 15 3.4.1 Inverse Power Iteration using LU-decomposition ...... 16 3.5 Eigenvector Algorithm for Spectral Clustering ...... 17

4 Result 19 4.1 Data ...... 19 4.1.1 Base Model and Similarity Matrix ...... 19 4.2 Unweighted Graph ...... 19 4.3 Passenger Weighted Graph ...... 21 4.4 2018 ...... 23 5 Analysis and Conclusions 26

A Notation 29

B Airports 30 1. Introduction

1.1 Background

1.1.1 Aviation industry Air traffic and airports play an important role in our modern society, connecting cities all over the world. The European Commission concludes that ”The European aviation sector is one of the best performing parts of the European economy, and is a world leading industry. 900 million air passengers travel each year to, from and within the European Union, making up one third of the world market.” [2] The economic factor in aviation industry is not only due to flights travelling between airports, but also due to jobs created at airports by shops, cafeterias, mechanics and transport to and from the airport [5]. These two factors, bringing people together and driving economic growth is of great importance to regional development and will in itself generate regional development as well as high tech industries [10]. But it will also lead to a more globalised Europe.

1.1.2 Clustering Clustering is a method where a set is divided into subsets based on some charac- teristics of the set. This method is often used in science to group different objects, for example in machine learning when one want to group objects and the main goal is that objects within one cluster should be similar to each other but dissimilar to objects in other clusters [11]. The idea of clustering is not new, it can be dated back to the early 1900s when they started using this idea in psychology. The more modern and numerical ap- proaches came later and 1963 the Principal of Numerical Taxonomy proposed the use of clusters in biological classification [1]. Since then research involving clustering has been done on many occasions. When clustering data, there is not a single method or algorithm for finding clusters. Instead there are several ones with different properties [11]. In this paper we will look at Spectral Clustering that was proposed back in 1973 [4] and uses the Laplacian and spectrum of the Laplacian to perform clustering.

1.2 Problem

The great importance of the Aviation industry and airports in general was discussed in Section 1.1. This raises several questions on how vulnerable the aviation industry

J. Larsson, I. Agren˚ 1 1. Introduction 1.2. Problem and especially the airports are to sudden changes. What would happen if some airports shut down or if some routes closed? In this report we want to look into these cases with spectral cluster analysis. In particular we want to see if Europe can be divided into several clusters of airports, and how those clusters are connected. If some routes or airports were to shut down suddenly, how would that affect the ability to travel between different regions of Europe?

J. Larsson, I. Agren˚ 2 2. Clustering

Clustering can be performed with many algorithms and one way to do it is by spectral clustering. To do this a basic knowledge about graphs and representation of graphs through matrices is important since spectral clustering uses the spectrum of the Laplacian matrix to perform clustering. A basic knowledge about basic standard machine learning algorithms like K-means is also needed.

2.1 Graphs

A graph is a set of points represented as nodes, which are connected to each other via edges. The edges can be either weighted, meaning some connections are stronger than others, or unweighted, meaning that all connections are equally strong. There are different ways of representing graphs, in this report the notation a graph G = (V,W ) is used, where V is a set of nodes and W is a set of edges between the nodes. This set of edges W can itself be represented by a matrix. Using this matrix notation, one ca say that the edge between node vi and vj has the weight wi,j, where W = (wi,j). Definition 2.1.1 (Weighted Graph) Let G = (V,W ) be a graph with a set of nodes V = {v1, ··· , vn} and a set of edges n×n W ∈ R where wi,j is the weight between node i and node j. Definition 2.1.2 (Symmetric Graph) A graph G = (V,W ) is said to be symmetric if

Wi,j = Wj,i.

Definition 2.1.3 (Degree Matrix) The degree matrix D is defined as the diagonal matrix

n X Di,i := di := Wi,j. j=1

Definition 2.1.4 (Number of nodes in Graph) The number of nodes in Graph G is written as |G|.

Definition 2.1.5 (Volume of Graph) The volume of a Graph is written as vol(G) and is defined as

n X vol(G) := di. i=0

J. Larsson, I. Agren˚ 3 2. Clustering 2.2. K-means

2.1.1 Methods for Constructing a Graph

p Data can be represented as points x(1),..., x(n) ∈ R , where p ∈ N and represented as nodes in a graph. A norm can be used to defined a distance between all data points. There are several norms, for example the euclidean norm. We denote the distance or similarity from point i to point j as sij. Using the norm, decisions whether or not two nodes should be connected or not can be made. This is often done using ε-neighbourhood (Definition 2.1.6), k-nearest neighbour (Definition 2.1.7) or by setting wi,j = si,j. A graph is said to be fully connected if all nodes have non zero edges to all other nodes, that is wi,j > 0, i 6= j. Definition 2.1.6 (ε-neighbourhood graph) Given an value ε, all nodes with a distance sij < ε is connected. So ( 1, if si,j < ε wi,k = . 0, otherwise

Definition 2.1.7 (k-nearest neighbour graph) Two vertices are connected if the vertex vi is among the k nearest neighbours of the vertex vj. An undirected graph can be created in several ways. The two most common ones are that two vertices are connected only if vi is among the k nearest neighbours of vj and vj is among the k nearest neighbours of vi, or that two nodes are connected if one of them is among the k nearest neighbours of the other.

Gaussian Similarity

In a data set with the data points x(1),..., x(n) one can use a Gaussian function to compare the similarity between two points i and j

2 ! − x(i) − x(j) s = exp , i,j 2σ2 where σ ∈ R.

Cosine Similarity

In a data set with data points x(1),..., x(n) the angle between the point i and j can be measured using cos θ. If the similarity of two points is equal to the cosine of the angle between them, we get a cosine similarity matrix,

hx(i)|x(j)i si,j = cos θi,j = . x(i) x(j)

2.2 K-means

In order to understand Spectral Clustering, understanding the K-means algorithm is important. It is a popular but naive clustering algorithm, operating on data in p p R dimensional space. Consider the data points x(1),..., x(n) ∈ R . To cluster these

J. Larsson, I. Agren˚ 4 2. Clustering 2.3. Unnormalized Laplacian

p data points into k clusters, k random points have to be defined, v(1),..., v(k) ∈ R . Each data point x(i) get assigned to the closest point v(j). Then v(j) is set to the average position of all data points assigned to it. The last two steps are repeated until the points v(j) does not change. This can be summarised in Algorithm 2.1.

Algorithm 2.1 The K-means algorithm p Create k random points in v(1),..., v(k) ∈ R . while v(1),..., v(k) have not converged do

Set Aj := {}. k Add the data points x(i) to Aj if v(j) is the closest point in the set {v(a)}a=1. 1 P v(j) := x(i). |Aj | i∈Aj end while

2.3 Unnormalized Laplacian

In order to perform Spectral Clustering, a special graph representation of the graph is required, called a Laplacian. The Laplacian contains the same information as a weight matrix but the spectrum of the Laplacian contains additional properties about the graph. Definition 2.3.1 (Unnormalized Laplacian) Let G=(V,W) be a symmetric graph with the degree matrix D. The unnormalized Laplacian L is defined as

L := D − W.

Theorem 2.3.1 The unnormalized Laplacian L satisfies 1. For all vectors f ∈ Rn n 1 X f T Lf = w (f − f )2. 2 i,j i j i,j=1

2. L is positive semidefinite, that is f T Lf ≥ 0.

1 . 3. The smallest eigenvalue to L is 0 with the eigenvector 1 := .. 1

4. L has n non negative eigenvalues such that 0 = λ1 ≤ λ2 ≤ · · · ≤ λn. Proof. See [8, page 4]. Theorem 2.3.2 (Multiplicity of eigenvalue 0) The multiplicity of the eigenvalue 0 to the Laplacian L belonging to a graph G is equal to the number of disjoint components in the graph G. Proof. See [8, page 4].

J. Larsson, I. Agren˚ 5 2. Clustering 2.5. Algebraic connectivity

2.4 Normalized Laplacian

In addition to the unnormalized Laplacian, Spectral Clustering also requires the normalized Laplacian, which also have some similar and interesting features. Definition 2.4.1 −1/2 p Let D = (di,i)i=j be a diagonal matrix. Then D = (1/ di,i)i=j, di,j = 0 if i 6= j Definition 2.4.2 (Normalized Laplacian) The normalized Laplacian Lsym is defined as

−1/2 −1/2 −1/2 −1/2 Lsym := D LD = I − D WD . Theorem 2.4.1 The normalized Laplacian Lsym satisfies 1. For all vectors f ∈ Rn

n !2 1 X fi fj f T L f = w √ − . sym 2 i,j d p i,j=1 i dj

1 −1/2 . 2. 0 is an eigenvalue to Lsym with the eigenvector D 1 := .. 1

3. Lsym is positive semi-definite and has n non negative eigenvalues such that 0 = λ1 ≤ λ2 ≤ · · · ≤ λn. Proof. See [8, page 5].

2.5 Algebraic connectivity

Algebraic connectivity of a graph G is the second smallest eigenvalue for the Lapla- cian of the graph G. This value gives information on how well-connected the graph is. Definition 2.5.1 (Algebraic connectivity) Let G be a graph and L its corresponding Laplacian, we then denote the algebraic connectivity of L as a(L). a(L) is the second smallest eigenvalue of L and A(L) is the eigenvector corresponding to that eigenvalue. Theorem 2.5.1 If a(L) = 0 the graph can be divided into two disjoint graphs. Proof. Follows directly from Theorem 2.3.2. Theorem 2.5.2 If G1 is a subgraph of G with same set of nodes and fewer edges, then a(L1) ≤ a(L). Proof. See [3]. From Theorem 2.5.2 it is clear that if an edge is removed from a graph, the algebraic connectivity will be lower or the same. The interpretation of this is that removing an edge can make a(L) lower, leading to a better so called cut when clustering.

J. Larsson, I. Agren˚ 6 2. Clustering 2.6. Cuts

2.6 Cuts

To cluster a graph, it is sometimes useful to study the cut of a graph. Lets use the notation Ac for the compliment of A. To partition the graph G into k parts A1,A2, ..., Ak it can be useful to study the following definitions. Definition 2.6.1 (Cut) The cut of a graph G into the parts A1,A2, ..., Ak is defined as

k 1 X Cut(A , ..., A ) = W (A ,Ac) 1 k 2 i i i=1 X W (A, B) = wi,j. i∈A,j∈B

Figure 2.1: An example of a unweighted graph.

Intuitively one can imagine that minimizing the cut function would lead to good clustering, since the cut represent the weight of the nodes that are being cut. So minimizing the cut function would yield the clusters whose weights between the different clusters are as small as possible. However the cut function does not take into account the number of nodes in the different clusters or the volume of the different clusters, so one risks ending up with clusters of very different sizes. For example the graph in Figure 2.1 would cut between node 1 and 2, making two very different sized clusters. To make the clusters roughly the same size, one can minimize RatioCut or NCut instead.

Definition 2.6.2 (RatioCut) The RatioCut of a graph G over the subgraphs A1,A2, ..., Ak is defined as

k c 1 X W (Ai,A ) RatioCut(A , ..., A ) = i . 1 k 2 |A | i=1 i Definition 2.6.3 (NCut) The NCut of a graph G over the subgraphs A1,A2, ..., Ak is defined as

k c 1 X W (Ai,A ) NCut(A , ..., A ) = i . 1 k 2 vol(A ) i=1 i

J. Larsson, I. Agren˚ 7 2. Clustering 2.7. Perturbation Viewpoint

Minimising RatioCut will encourage the clusters to have roughly the same number of nodes in them, and minimising NCut will encourage the clusters to have roughly the same volume. For example the graph in Figure 2.1 would cut the edges between nodes 3, 10 and 7, 8. However minimising these functions is not trivial, it is in fact NP-hard, meaning it can not be solved in polynomial time. It is however possible to approximate the minimisation of RatioCut and the minimisation of NCut using the Laplacian of the graph. It has been shown that the minimisation of RatioCut(Ai, ..., Ak) into k different cluster can be approximated by the k-1 eigenvectors belonging to the eigenvalues λ2, ..., λk of L [8, page 10]. Likewise it has also been shown that the minimisation of NCut(Ai, ..., Ak) into k different cluster can be approximated by the k-1 eigenvectors belonging to the eigenvalues λ2, ..., λk of Lsym [8, page 12].

2.7 Perturbation Viewpoint

To gain a more intuitive understanding of why the eigenvectors belonging to the Laplacian works to approximate clusters, one can consider the weight matrix as a sum of two different matrices, one representing two disjoint graphs and one repre- senting a perturbation, or a few week connections between the two clusters. Consider the following Laplacian

 2 −1 −1 0 0 0  −1 2 −1 0 0 0    −1 −1 2 0 0 0  L =   .  0 0 0 2 −1 −1    0 0 0 −1 2 −1 0 0 0 −1 −1 2

It is trivial that L has two eigenvectors with eigenvalue 0. The vectors are linear combinations of 1 and [1 1 1 − 1 − 1 − 1]T . The second vector can be thought of as an indicator vector, showing which nodes belong to which cluster. Now consider adding a small connection between the two clusters with the weight ε

 2 −1 −1 0 0 0  −1 2 −1 0 0 0    −1 −1 2 + ε −ε 0 0  L =   .  0 0 −ε 2 + ε −1 −1    0 0 0 −1 2 −1 0 0 0 −1 −1 2

Now one assumption needs to be made. The eigenvalues and the components of the eigenvectors are continuous functions of ε, so when ε change a little, then the eigenvalue and components should also change a little. If ε > 0 then the eigenvalue 0 will only have one eigenvector, 1, but using the previous assumption and assuming that ε is small, the eigenvector to the second smallest eigenvalue should be [1 1 1 − T 1 − 1 − 1] + g(ε), where gi(ε) is small, so the vector can still be seen as a indicator vector if ε is small. If the connection between two clusters are weak, then the second eigenvector will be an indicator vector showing which nodes belong to which cluster.

J. Larsson, I. Agren˚ 8 2. Clustering 2.8. Spectral Clustering Algorithm

2.8 Spectral Clustering Algorithm

The approximations of RatioCut and Ncut for k clusters create k−1 vectors. Let V = n×k−1 [v(2), ..., v(k)] ∈ R where n is the number of data points,then treat each row in V as a coordinate in a k−1 dimensional space. These n coordinates each correspond to the n different nodes. If the data is clustered according to RatioCut/NCut then the data will be clusterd in the Rk−1 space. By using K-means clustering the clusters in the Rk−1 space can be identified.

2.8.1 Spectral Clustering (RatioCut)

The Spectral Clustering algorithm for finding k clusters C1,C2, ..., Ck using the un- normalized Laplacian as an approximation of minumum RatioCut is described in Algorithm 2.2. V contains n nodes v1, v2, ..., vn.

Algorithm 2.2 Approximation of RatioCut 1: Create Graph G=(V, W). 2: Compute the Laplacian L := D − W .

3: Find the eigenvectors u(1), u(2),..., u(k) of the matrix L belonging to the k small- est eigenvalues and create the matrix U = [u(1), u(2),..., u(k)].

4: Treat each row in U as a data point x(1), x(2),..., x(n) and perform K-means clustering on the points into k clusters A1,A2, ..., Ak.

2.8.2 Spectral Clustering (NCut)

The Spectral Clustering algorithm for finding k clusters C1,C2, ..., Ck using the nor- malized Laplacian as an approximation of min NCut is described in Algorithm 2.3. V contains n nodes v1, v2, ..., vn.

Algorithm 2.3 Approximation of NCut 1: Create Graph G=(V, W). −1/2 −1/2 2: Compute the normalized Laplacian Lsym := D LD .

3: Find the eigenvectors u(1), u(2), ..., u(k) of the matrix Lsym belonging to the k smallest eigenvalues and create the matrix U = [u(1), u(2), ..., u(k)] = (ui,j). P 2
1/2 4: Create T = (ti,j) and set ti,j := ui,j/ k ui,k . That is let T contain the normalized rows of U.

5: Treat each row in T as a data point x(1), x(2), ..., x(n) and perform K-means clustering on the points into k clusters A1,A2, ..., Ak.

J. Larsson, I. Agren˚ 9 3. Eigenvectors

In many areas of natural sciences, eigenvalues and eigenvectors play an important role. For example they can be used in physics to describe several problems and their solutions. In graph theory, the spectrum is used in the same way. By exploiting eigenvectors and eigenvalues of the matrix representing the graph one can find char- acteristics of the graph. This can then be used in algorithms to perform clustering, exploiting these characteristics.

3.1 Eigenvectors and Rayleigh-Ritz Quotients

Definition 3.1.1 (Eigenvalues) Let T be a linear transformation from a vector space V to V over a field F and let v be a non zero vector in V . v is an eigenvector with the eigenvalue λ if T (v) = λv. Definition 3.1.2 (Eigenvalues of a real square matrix) Let A ∈ Rn×n and v ∈ Rn, v 6= 0. v is an eigenvector to A with the eigenvalue λ if Av = λv. Definition 3.1.3 (Spectrum) Let A be a square matrix with the eigenvalues λ1, λ2, ..., λk. The spectrum of A is the set of all eigenvalues of A Λ(A) := {λ|Av = λv, v 6= 0} . Definition 3.1.4 (Spectral radius) Let A be a matrix with the spectrum Λ(A). The spectral radius of A is defined as ρ(A) := max |λ|. λ∈Λ(A) Theorem 3.1.1 (Rayleigh–Ritz) Let A be a real, symmetric n×n matrix with the eigenvalues λ1 ≤ λ2 ≤ λ3 ≤ ... ≤ λn n with the corresponding eigenvectors v(1), v(2), v(3), ..., v(n), and let {v(i)}i=1 be an orthonormal basis of A, then xT Ax max = λn x6=0 xT x  and if Vk := Span v(1), v(2), ..., v(k) then xT Ax max T = λk. x6=0,x∈Vk x x

J. Larsson, I. Agren˚ 10 3. Eigenvectors 3.2. Single Eigenvalue Algorithms

Proof. See [7, section 5.5].

Theorem 3.1.2 (Rayleigh–Ritz Eigenvalue) Let A be a real, symmetric n × n matrix with eigenvalue λ with the corresponding eigenvector v. Then

vT Av = λ. vT v Proof. See [7, section 5.5].

3.2 Single Eigenvalue Algorithms

3.2.1 Power Iteration Power iteration is a method for finding an eigenvector corresponding to the eigen- value λn that has the largest absolute value. The method is simple to implement and if combined with the Rayleigh-Ritz Eigenvalue theorem it will also find the eigenvalue corresponding to the eigenvector. The method takes a starting vector x(0) and iterates according to

Ax(k) x(k+1) := , kAx(k)k until the vector x converges to an eigenvector. The eigenvalue corresponding to x can then be found by using Theorem 3.1.2.

Theorem 3.2.1 (Power Iteration) Let A be a real n × n matrix with the eigenvalues λ1 ≤ λ2 ≤ λ3 ≤ ... < λn with the n corresponding eigenvectors v(1), v(2), v(3), ..., v(n), and let {v(i)}i=1 be an orthonormal eigenbasis of A. Using the starting vector x(0), in the following iteration x(k) will con- verge to the eigenvector with the largest absolute eigenvalue with a rate proportional to λn , λn−1

n (0) X x := αiv(i) , αi ∈ R and αn 6= 0 i=1 Ax(k) x(k+1) := . kAx(k)k

Proof.

n n n−1  k ! X X X λi Ax(k) = Akα v = λkα v = λk α v + α v i (i) i i (i) n n (n) λ i (i) i=1 i=1 i=1 n (k) k lim Ax −→ λnαnv(n) k→∞ Ax(k) lim ∝ v(n) k→∞ kAx(k)k

J. Larsson, I. Agren˚ 11 3. Eigenvectors 3.3. Deflation

3.2.2 Inverse Iteration Power iteration (Theorem 3.2.1) finds the largest eigenvalue and eigenvector of a matrix, but to find the smallest eigenvector, inverse iteration is requierd. Consider the Theorems 3.2.2, 3.2.3 and 3.2.4. Theorem 3.2.2 (Eigenvalues of inverted matrix) Let A be a n × n matrix with det(A) 6= 0 and eigenvalues λ1 ≤ λ2 ≤ λ3 ≤ ... ≤ λn −1 with the corresponding eigenvectors v(1), v(2), v(3), ..., v(n). Then A have the same eigenvectors and the eigenvalues 1 , 1 , 1 , ..., 1 . λ1 λ2 λ3 λn Proof. 1 Av = λv ⇐⇒ A−1Av = λA−1v ⇐⇒ A−1v = v. λ

Theorem 3.2.3 (Eigenvalues of shifted matrix) Let A be a n × n matrix with eigenvalues λ1 ≤ λ2 ≤ λ3 ≤ ... ≤ λn with the corresponding eigenvectors v(1), v(2), v(3), ..., v(n) and let µ be a real number called the shift. Then (A − µI) have the eigenvalues λ1 − µ, λ2 − µ, λ3 − µ, ..., λn − µ and the same eigenvectors as A. Proof.

(A − µI)v = Av − µIv = λv − µv = (λ − µ)v.

Theorem 3.2.4 (Inverse Iteration) Let A be a real n × n matrix with eigenvalues λ1 ≤ λ2 ≤ λ3 ≤ ... ≤ λn with the n corresponding eigenvectors v(1), v(2), v(3), ..., v(n), and let {v(i)}i=1 be an orthonormal basis of A. Using the starting vector x(0) and iterating as follows, x(k) will converge to the eigenvector with the eigenvalue closest to µ.

n (0) X x := αiv(i) , αk 6= 0, where λk is closest to µ i=1 (A − µI)−1x(k) x(k+1) := . k(A − µI)−1x(k)k Proof. From Theorem 3.2.2 and Theorem 3.2.3 it follows that the largest eigenvalue to (A−µI)−1 is the eigenvalue closest to µ. The rest follows from Theorem 3.2.1.

3.3 Deflation

Now there is a set of tools to find the smallest and largest eigenvalues and their corresponding eigenvectors of a matrix A. However, to perform Spectral Clustering there is a need to find multiple eigenvectors. One method for doing this is by modifying a matrix A by removing the largest eigenvalue from it, creating a matrix B with the spectrum Λ(B) = {λ1, ..., λn−1, 0}. This makes it possible to find the largest eigenvalue of B, that corresponds to the second largest eigenvalue of A.

J. Larsson, I. Agren˚ 12 3. Eigenvectors 3.3. Deflation

3.3.1 Wielandt Deflation

To change the eigenvalue λ1 of A by µ, create A1 as

T A1 := A − µv(1)u , were hv(1)|ui = 1 and µ ∈ R. The eigenvalues of A1 are the same as of A except for the eigenvalue λ1 that will be λ1 − µ. Furthermore the eigenvector v˜(j) of A1 will be a linear combination of v(1) and v(j). Theorem 3.3.1 (Wielandt deflation) Let {v(1), v(2), v(3), ...} be eigenvectors of A with the eigenvalues {λ1, λ2, λ3, ...}. The deflated matrix A1 will have the spectrum Λ(A1) = {λ1 −µ, λ2, λ3, ...} and the eigen- vectors

v˜(i) = v(i) − γiv(1), i = 2, 3, ... T µu v(i) γi := . µ − λ1 − λi Proof. See [12].

3.3.2 Hotelling’s Deflation In Wielandt deflation, the vector u can be chosen in many different ways. By setting (u||1) and µ = λ1, one gets Hotelling’s deflation. This can be combined with power iteration to find the second eigenvalue of the matrix A.

Theorem 3.3.2 (Hotelling’s Deflation) Let A be a real n × n matrix with eigenvalues λ1 ≤ λ2 ≤ λ3 ≤ ... ≤ λn with the corresponding eigenvectors v(1), v(2), v(3), ..., v(n) and let (u||1) and hv(1)|ui = 1. Then

(k+1) (k) T (k) xo := Ax − λnv(n)u x (k+1) (k+1) xo x := (k+1) xo

(∞) will converge to the vector x that is an eigenvector to A1 with the eigenvalue λn−1. The eigenvector v(n−1) with the eigenvalue λn−1 belonging to the matrix A is

(∞) T (∞) (∞) v(n−1) = Ax − v(n)Ax v(n) , if x = v(n) = 1 (3.1)

Proof. The fact that it converges to the eigenvector of A1 belonging to λn−1 fol- low from Theorem 3.3.1 and Theorem 3.2.1. To prove equation (3.1), assume (∞) (∞) that x = v(n) = 1. It is known that x = αv(n) + βv(n−1) and that hv(n)|v(n−1)i = 0,

(∞) T (∞) x = Aαv(n) + Aβv(n−1) = λnαv(n) + λn−1βv(n−1)Ax = λnα (∞) T (∞) λ2βv(n−1) = Ax − v(n)Ax v(n).

Since λ2βv(n−1) is an eigenvector, v(n−1) is also an eigenvector.

J. Larsson, I. Agren˚ 13 3. Eigenvectors 3.3. Deflation

3.3.3 Schur-Wielandt Deflation Gram-Schmidt For the implementation of Schur-Wielandt deflation the vectors have to be orthonor- malized. To do this the Gram-Schmidt method can be used, presented in Theorem 3.3.3. Definition 3.3.1 (Orthonormal) A set of vectors {v(1), v(2), ..., v(n)} is said to be orthonormal if

hv(i)|v(j)i =0, if i 6= j

hv(i)|v(i)i =1. Theorem 3.3.3 (Gram-Schmidt) Let {v(1), v(2), ..., v(n)} be a set of vectors. In order to orthonormalize the vectors into a set of orthonormal vectors {u(1), u(2), ..., u(n)} one can use

j−1 0 X v(j) :=v(j) − hv(j)|u(i)iu(i) i=1 0 v(j) u(j) := . v0 (j) The process is repeated for j = 1, 2, 3, ..., n. Proof. See [14, page 183]

Schur Decomposition Definition 3.3.2 (Schur Decomposition) Let A ∈ Rn×n. The Schur decomposition of A is A = VTV H ,  where V is a unitary matrix and T is an upper triangular matrix. Here v(1), ..., v(n) are Schurvectors and V = [v(1), ..., v(n)]. Definition 3.3.3 (Unitary Matrix) Let V ∈ Rn×n. V is said to be unitary if V H V = VV H = I. If V is real then V is unitary if V T V = VV T = I. Theorem 3.3.4 n×n n×j Let A ∈ R and Vj = [v(1), v(2), ..., v(j)] ∈ R where v(i) are Schurvectors of A associated with the eigenvalues λ1, λ2, ..., λj and let D = diag(µ1, µ2, ..., µj). Then the matrix

H Aj := A − VjDVj has the eigenvalues λi − µi for i ≤ j and λi for i > j.

J. Larsson, I. Agren˚ 14 3. Eigenvectors 3.4. LU-Decomposition

Proof. See [12, page 8].

Now, recall Wielandt deflation (Theorem 3.3.1). By choosing the vector u = vi when deflating one can achieve Schur-Wielandt deflation. The eigenvector that we found in the deflated matrix will be a linear combination of the vectors used to deflate it, and the actual eigenvector to the undeflated matrix. Consider the Algorithm 3.1.

Algorithm 3.1 Schur Wielandt deflation for finding p eigenvectors A ∈ Rn×n.

Find largest eigenvalue λ1 and its corresponding eigenvector v(1) belonging to A. for j ∈ [1, p − 1] do Use Wielandt deflation with the eigenvectors as u

0 X T A = A − λiv(i)v(i) .

0 Find largest eigenvalue λk+1 with its eigenvector v(k+1) of A .  As v(k+1) is a linear combination of v(1), ..., v(k) , orthonormalize v(k+1)  against v(1), ..., v(k) . end for

3.4 LU-Decomposition

Definition 3.4.1 (LU-Decomposition) Let A be a square matrix. The LU-decomposition with pivoting of A is

PD−1AQ = LU where L is a lower triangular matrix (all the elements above the diagonal are 0) and U is a upper triangular matrix (all the elements below the diagonal are 0), P and Q are permutation matrices (orthogonal matrices with only one entry per row) and D is a diagonal matrix.

The main benefit of doing a LU-decomposition is that solving systems of linear equations are more efficient, since one can use back substitution, see Defintion 3.4.2, on triangular matrices, and do not need to calculate inverses. Consider the equation . .   .  .. . .      a b c g   x =    0 d e h ··· 0 0 f i

Then solve for x. Then it can be seen that fxn = i =⇒ xn = i/f. Then solve dxn−1 + exn = h =⇒ dxn−1 = h − ei/f =⇒ xn−1 = (h − ei/f)d. In this way the system can be solved without having to resort to Gaussian elimination or other computational expensive methods. But in order to use this method, the matrix have to be triangular.

J. Larsson, I. Agren˚ 15 3. Eigenvectors 3.4. LU-Decomposition

Definition 3.4.2 ( ) Let U ∈ Rn×n be a triangular matrix and b ∈ Rn a vector. The solution x to the system Ux = b solved by back substitution can be written as x = U b

3.4.1 Inverse Power Iteration using LU-decomposition By combining LU-decomposition with power iteration (Theorem 3.2.1) one gets something that is similar to inverse power iteration (Theorem 3.2.4), but that does not require the inverse of matrix A to be calculated, see Algorithm 3.2.

Algorithm 3.2 LU aided Inverse Iteration LU-decompose A into L and U. Set v to a random vector. repeat Solve Av0 = v using the LU-decomposition of A. v0 Set v := kv0k . until v converges.

The issue here is how to solve Av0 = v. Using LU-decomposition one can rewrite

PD−1AQ = LU A = DP −1LUQ−1 A−1 = QU −1L−1PD−1.

For inverse power iteration, one want to calculate Av0 = v ⇐⇒ v0 = A−1v. This gives

v0 = QU −1L−1PD−1v = Q (U (L (P (D v)))) (3.2)

Solving the system Av0 = v as in equation (3.2) have a number of advantages. First, no matrix inverse has to be calculated. Second , Q, P and D are sparse matrices and L and U are likely to be sparse. Third, there are no matrix multiplications taking place, only matrix-vector multiplication and back substitution. Since the matrices are likely to be sparse the multiplication and back substitution are likely to be fast. This brings us to this new version of inverse power iteration. Theorem 3.4.1 (Inverse Iteration using LU-decomposition) Let A be a real n × n matrix with the eigenvalues λ1 ≤ λ2 ≤ λ3 ≤ ... ≤ λn with the n corresponding eigenvectors v(1), v(2), v(3), ..., v(n), and let {v(i)}i=1 be an orthonormal basis of A. Furthermore let PD−1AQ = LU be a LU-decomposition of A. Using the starting vector x(0) in the following iteration, x will converge to the eigenvector whose corresponding eigenvalue has the smallest absolute value.

n (0) X x := αiv(i) , αk 6= 0, where λk is closest to 0 i=1 y(k+1) := Q U L P D x(k)



y(k+1) x(k+1) := . ky(k+1)k J. Larsson, I. Agren˚ 16 3. Eigenvectors 3.5. Eigenvector Algorithm for Spectral Clustering

3.5 Eigenvector Algorithm for Spectral Cluster- ing

Using the algorithms and methods described above we are now able to construct the algorithm for finding eigenvectors to the Laplacian. The Laplacian have the eigenvalues 0 = λ1 ≤ λ2 ≤ λ3 ≤ ... ≤ λn, and Spectral Clustering requires the k eigenvectors belonging to λ1, ..., λk to divide the graph into k clusters. The first thing needed to be handled was the fact that the Laplacian have an eigenvalue equal to 0, and therefore it has det(L) = 0 and therefore it is not possible to LU-decompose it. To solve this Theorem 3.2.3 was used and a small shift µ ∈ R was chosen. Here the shift must be sufficiently small such that λ 0 < µ < 2 , 2 otherwise problems with convergence would appear since Theorem 3.4.1 would not converge to λ1 if this was not satisfied. −1 Using inverse iteration (Theorem 3.2.4) (λ1 − µ) and v(1) belonging to L could be found. To find more vectors, Schur-Wielandt deflation (Theorem 3.3.4) was used. Orthonormalization was performed by Gram-Schmidt (Theorem 3.3.3). All together this resulted in the Algorithm 3.3.

J. Larsson, I. Agren˚ 17 3. Eigenvectors 3.5. Eigenvector Algorithm for Spectral Clustering

Algorithm 3.3 Schur-Wielandt Deflation using LU aided Inverse Power Iteration Input symmetric L ∈ Rn×n.

λ2 Choose a shift µ that satisfies 0 < µ < 2 . Create the matrix A := (L − µI). Finding the smallest eigenvalues to L is now equivalent to finding the largest eigenvalue to A−1. LU-decomposize A into PD−1AQ = LU. −1 Find eigenvalue σ1 and eigenvector v(1) to A using power iteration with LU- decomposition (Section 3.4.1). The eigenvector should satisfy v(1) ∝ 1, if not, then µ was too large.

Set u(1) := v(1).

Create matrix Ui = [u(1)u(2)...u(i)] and Σi = Diag (σ1, σ2, ..., σi). for i ∈ [1, k − 1] do (0) Choose random vector v(i+1). repeat

(p+1)     (p)    T (p)  w(i+1) := Q U L P D v(i+1) − Ui Σi Ui v(i+1) (p+1) (p+1) w(i+1) v(i+1) := w(p+1) (i+1)

(p) until v(i+1) converges.

Calculate the eigenvalue σi+1 from v(i+1) using Theorem (3.1.2). k Orthonormalize v(i+1) against (u(j))j=1 and add the vector u(i+1) to Ui+1 using: i X v˜(i+1) :=v(i+1) − hv(i+1)|u(j)iu(j) j=1

v˜(i+1) u(i+1) := . v˜(i+1) end for

After Algorithm 3.3 is done, the matrix Uk will contain the k normalized eigen- vectors belonging to λ1, ...λk.

J. Larsson, I. Agren˚ 18 4. Result

4.1 Data

The data was collected from the European Unions official statistics office named Eurostat [6]. We gathered data from 34 different European countries. The data was divided into several files and each file contained the number of passengers flying between different airports in Europe during the period 1993 to 2019. The statistics was not complete since not all countries had been reporting data for every year. We decided to filter out two different years where data was reported from all countries. The years chosen were 2009 and 2018, since 2018 was the last year where all countries had reported. From this, all self-loops were removed, meaning all flights taking off and landing at the same airport.

4.1.1 Base Model and Similarity Matrix To represent the flight network a graph with nodes representing airports is used. The similarity matrix S is created such that si,j is equal to the number of passengers that travel between the destinations.

4.2 Unweighted Graph

A simple model of the flight network is to use the base model (Section 4.1.1) and add a connection between two nodes if there are passengers travelling between the airports. That is ( 1, if si,j > 0 wi,j = . 0, otherwise

Performing RatioCut spectral clustering on this model using data from 2009 with five clusters, it appears that cluster 1 is much larger then the other ones (see Table 4.1).

Table 4.1: Clusters found for unweighted graph.

Cluster Number of nodes Cluster 1 362 Cluster 2 9 Cluster 3 7 Cluster 4 6 Cluster 5 8

J. Larsson, I. Agren˚ 19 4. Result 4.2. Unweighted Graph

Even though one cluster is much larger than the others, the clusters do exist. By plotting the 4D-spectral space on which K-means has been run, it can be seen that hyper planes do exist (see Figure 4.1 and Figure 4.2), even though the clusters 2-5 are quite small compared to cluster 1.

Figure 4.1: The 4 dimensional spectral space on which K-means is performed. The nodes align according to the eigenvectors, and colour according to cluster assignment after running K-means. Cluseri=Ci. Every dot in the plot represents an airport.

By plotting the graph, the shape of the clusters can be analysed more clearly. Cluster 1 is a main cluster connecting large parts of the continent, while the other clusters are offshoots of this main cluster. By studying Figure 4.3 and cross refer- encing with the list of airports (Appendix B), one can get a list of the airports in different clusters.

Figure 4.2: The eigenvectors plotted, with index/node on the x-axis and value of the eigenvectors components on the y-axis.

Cluster 2 and 3 contain airports in northern Norway. Cluster 2 contains Tromsø, Alta and other small airports in the northern most part of Norway, while cluster 3 consists of Bodø airport acting like a hub, connecting out to other small airports in the north-west of Norway, like Sandnessjøens, Leknes and Narvik airport. Cluster 4 contains airports in the West Indies, Caribbean and northern South America where there are French colonies. Martinique Aim´eC´esairesInternational

J. Larsson, I. Agren˚ 20 4. Result 4.3. Passenger Weighted Graph

Airport, Cayenne-Rochambeau Airport and Pointe-`a-PitreInternational Airport are some of the airports in this cluster. Cluster 5 consists of airports in the Portuguese Azores, containing among others Jo˜aoPaulo II, Horta and Lajes International Airport.

Figure 4.3: The unweighted graph, with nodes coloured according to cluster assign- ment after running Spectral Clustering. Cluseri=Ci.

4.3 Passenger Weighted Graph

A more realistic model of the flight network than the one in Section 4.2 is to take into account the number of passengers travelling between the different airports. This was achieved by using the base model (Section 4.1.1) and setting

wi,j = si,j. Performing RatioCut spectral clustering on this model using data from 2009 with five clusters it appeared that the clusters were much smaller than the clusters from the unweighted graph (see Table 4.2). By observing Figure 4.4 and Figure 4.5 it became clear that this was not just an issue with convergence of K-means but in fact the actual clusters indicated by the RatioCut algorithm.

Table 4.2: Clusters found for unweighted graph.

Cluster Number of nodes Cluster 1 387 Cluster 2 1 Cluster 3 2 Cluster 4 1 Cluster 5 1

J. Larsson, I. Agren˚ 21 4. Result 4.3. Passenger Weighted Graph

Figure 4.4: The 4 dimensional spectral space on which K-means is performed. The nodes alignment according to the eigenvectors, and colour according to cluster as- signment after running K-means. Cluseri=Ci.

Figure 4.5: The eigenvectors plotted, with index/node on the x-axis and value of the eigenvectors components on the y-axis.

From Figure 4.6 it can be seen that all of the clusters 2-5 are leafs to cluster 1.

J. Larsson, I. Agren˚ 22 4. Result 4.4. 2018

Figure 4.6: The passenger weighted graph, with nodes coloured according to cluster assignment after running Spectral Clustering. Cluseri=Ci.

4.4 2018

Using the most up to date data from 2018, performing the specral clustering al- gorithm using both models described above yields a slightly different result. The unweighted graph model from 2018 has smaller clusters which can be seen in Figure 4.7 and 4.8, however using the 2018 data with the the passenger weighted model yields a similar result as to 2009, as can be seen in Figure 4.9 and 4.10.

J. Larsson, I. Agren˚ 23 4. Result 4.4. 2018

Figure 4.7: The unweighted graph, with nodes coloured according to cluster assign- ment after running Spectral Clustering using data from 2018. Cluseri=Ci.

Figure 4.8: The 4 dimensional spectral space on which K-means is performed. The nodes alignment according to the eigenvectors, and colour according to cluster as- signment after running K-means, using data from 2018 with the unweighted model. Cluseri=Ci.

J. Larsson, I. Agren˚ 24 4. Result 4.4. 2018

Figure 4.9: The passenger weighted graph, with nodes coloured according to cluster assignment after running Spectral Clustering using data from 2018. Cluseri=Ci.

Figure 4.10: The 4 dimensional spectral space on which K-means is performed. The nodes alignment according to the eigenvectors, and colour according to cluster as- signment after running K-means, using data from 2018 with the passenger weighted model. Cluseri=Ci.

J. Larsson, I. Agren˚ 25 5. Analysis and Conclusions

As can be seen in Section 4.2 and Section 4.3 the European flight network in 2009 was not very clustered. There were a few groups of airports clustered and poorly connected to the rest of Europe, mainly in Norway, as could be seen in the un- weighted model. However the fact that the clusters disappeared in the weighted graph suggests that very few people travels between the airports inside the clusters. This does not mean that there are a small amount of people travelling to the clus- ters, but that the individual leaf airports inside the clusters are likely not heavily trafficked. When looking at the data from 2018 (Section 4.4) it is clear that the Euro- pean flight network has gotten a little bit less clustered, mainly that the Norwegian clusters has gotten smaller, as can be seen in the unweighted model. There it can also be seen that the European colonies who were previously detected as clusters have gotten better connected to the continent to the extent that they are no longer detected as clusters. The weighted model is still unclustered with the 2018 data. After performing the cluster analysis on the European flight network, it seems as if the airports in Europe are not very clustered, and the few clusters that can be found appear to be small, regional airports with low volumes of passengers. The system does not appear to be very sensitive. If a few airports were disabled for some reason, the ability to reach different airports from others would not change drastically. We can also see that it has become less sensitive, meaning that the network have become less clustered from 2009 and 2018. However, this study have not looked at the capacity of the airports, so whether or not travel can be sustained at current volumes if a few airports becomes disabled is something that needs to be further studied.

J. Larsson, I. Agren˚ 26 Bibliography

[1] Roger K. Blashfield and Mark S. Aldenderfer. “The Methods and Problems of Cluster Analysis”. In: Handbook of Multivariate Experimental Psychology. 1988, pp. 447–473. url: https://doi.org/10.1007/978-1-4613-0893- 5_14. [2] European Commission. Airports. 2020. url: https://ec.europa.eu/transport/ modes/air/airports_en (visited on 04/11/2020). [3] James Demmel. Graph Partitioning, Part 2. 1999. url: https://people. eecs.berkeley.edu/~demmel/cs267/lecture20/lecture20.html. [4] W. E. Donath and A. J. Hoffman. “Lower Bounds for the Partitioning of Graphs”. In: IBM Journal of Research and Development 17.5 (1973), pp. 420– 425. [5] Economywatch. Airports, World Airports, Economic Impact of Airports. 2010. url: https://www.economywatch.com/world- country/airports.html (visited on 04/11/2020). [6] Eurostat. Air Transport, Air transport measurment - passengers, Detailed air passenger transport by reporting country and routes. 2020. url: https: //ec.europa.eu/eurostat/web/transport/data/database (visited on 02/06/2020). [7] Helmut L¨utkepohl. Handbook of Matrices. Wiley, 1997. [8] Ulrike von Luxburg. A Tutorial on Spectral Clustering. 2007. url: https: //arxiv.org/abs/0711.0189. [9] Lukas Polthier. Algebraic Multilevel Methods for Markov Chains. 2017. url: https://arxiv.org/abs/1711.04332. [10] Tomas Holgersson Richard Florida Charlotta Mellander. “Up in the Air: The role of Airports for Regional Economic Development”. In: The Annals of Re- gional Science (2012), pp. 14–15. [11] H. Charles Romesburg. Cluster Analysis for Researchers. Lulu Press, 2004. [12] Youcef Saad. Numerical Solution of Large Nonsymmetric Eigenvalue Problems. 1988. url: https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/ 19890017268.pdf. [13] Yousef Saad. Numerical Solution of Large Nonsymmetric Eigenvalue Problems, 2nd Edition. Society for Industrial and Applied Mathematics, 2011. [14] Axler Sheldon. Linear Algebra Done Right. Springer Cham Heidelberg New York Dordrecht London, 2015.

J. Larsson, I. Agren˚ 27 5. Bibliography .

[15] Mikael Johansson Vien V. Mai. “Noisy Accelerated Power Method for Eigen- problems With Applications”. In: IEEE TRANSACTIONS ON SIGNAL PRO- CESSING (2019).

J. Larsson, I. Agren˚ 28 A. Notation

The dot product between two vectors x ∈ Rn, y ∈ Rn is defined as hx|yi = x · y = xT y.

Kroneckers delta δi,j is defined as ( 1 , if i = j δi,j = . 0 , otherwise The matrix A ∈ R3×3 is written as   a1,1 a1,2 a1,3 A = (ai,j)i,j = a2,1 a2,2 a3,3 . a3,1 a3,2 a3,3 The identity matrix I is written as 1 0 0 ··· 0 1 0    I = (δi,j)i,j = 0 0 1  . . .  . .. diag(a, b, c, ...) is a diagonal matrix with the elements a, b, c, ... on the diagonal a 0 0 ··· 0 b 0    diag(a, b, c, ...) = 0 0 c  . . .  . ..

n Accessing the i:th component of the vector v ∈ R is done by indexing vi.

In a set of vectors {v(j)}k one accesses the i : th vector as v(i).

T T The transpose of a matrix A is written as A and defined as Ai,j = Aj,i.

The conjugate of a complex number z = x + iy isz ¯ = x − iy.

H H
¯ The Conjugate Transpose of A is A i,j = Aj,i.

A variable x is assigned the value y by x := y.

n×n The determinant of a matrix A ∈ R with eigenvalues λ1, λ2, ..., λn is det(A) = λ1 · λ2 · ... · λn

J. Larsson, I. Agren˚ 29 B. Airports

A list of all airports considered in the report.

ICAO code Airport name EL LGKP Karpathos Airport AT LOWG Graz Airport EL LGKR Ioannis Kapodistrias International Airport AT LOWI Innsbruck Airport EL LGKV Alexander the Great International Airport AT LOWK Klagenfurt Airport EL LGLM Limnos Airport AT LOWL Linz Airport EL LGMK Mikonos Airport AT LOWS Salzburg Airport EL LGML Milos Airport AT LOWW Vienna International Airport EL LGMT Mytilene International Airport BE EBAW Antwerp International Airport (Deurne) EL LGNX Naxos Airport BE EBBR Brussels Airport EL LGPA Paros National Airport BE EBCI Brussels South Charleroi Airport EL LGPZ Aktion National Airport BE EBLG Li`egeAirport EL LGRP Diagoras Airport BE EBOS Ostend-Bruges International Airport EL LGSA Chania International Airport BG LBBG Burgas Airport EL LGSK Skiathos Island National Airport BG LBSF Sofia Airport EL LGSM Samos Airport BG LBWN Varna Airport EL LGSR Santorini Airport CH LSGG Geneva Cointrin International Airport EL LGTS Thessaloniki Macedonia International Airport CH LSZA Lugano Airport EL LGZA Zakynthos International Airport CH LSZB Bern Belp Airport ES GCFV Fuerteventura Airport CH LSZH Z¨urich Airport ES GCHI Hierro Airport CH LSZR St Gallen Altenrhein Airport ES GCLA La Palma Airport CY LCLK Larnaca International Airport ES GCLP Gran Canaria Airport CY LCPH Paphos International Airport ES GCRR Lanzarote Airport CZ LKMT Ostrava LeoˇsJan´aˇcekAirport ES GCTS Tenerife South Airport CZ LKPR V´aclav Havels Airport Prague ES GCXO Tenerife Norte Airport CZ LKTB Brno-Tuˇranys Airport ES GEML Melilla Airport DE EDDB Berlin Sch¨onefeldAirport ES LEAL Alicante International Airport DE EDDC Dresden Airport ES LEAM Almer´ıaInternational Airport DE EDDE Erfurt Airport ES LEAS Asturias Airport DE EDDF Frankfurt am Main Airport ES LEBB Bilbao Airport DE EDDG M¨unsterOsnabr¨ucks Airport ES LEBL Barcelona International Airport DE EDDH Hamburg Airport ES LECO A Coru˜naAirport DE EDDK Cologne Bonn Airport ES LEGE Girona Airport DE EDDL D¨usseldorfAirport ES LEGR Federico Garcia Lorca Airport DE EDDM Munich Airport ES LEIB Ibiza Airport DE EDDN Nuremberg Airport ES LEJR Jerez Airport DE EDDP Leipzig/Halle Airport ES LELC San Javier Airport DE EDDR Saarbr¨ucken Airport ES LEMD Adolfo Su´arezMadrid–Barajass Airport DE EDDS Stuttgart Airport ES LEMG M´alagaAirport DE EDDT Berlin-Tegel Airport ES LEMH Menorca Airport DE EDDV Hannover Airport ES LEPA Palma De Mallorca Airport DE EDDW Bremen Airport ES LEPP Pamplona Airport DE EDFH Frankfurt-Hahn Airport ES LERS Reus Air Base DE EDHI Hamburg-Finkenwerder Airport ES LESO San Sebastian Airport DE EDJA Memmingen Allgau Airport ES LEST Santiago de Compostela Airport DE EDLP Paderborn Lippstadt Airport ES LEVC Valencia Airport DE EDLV Weeze Airport ES LEVD Valladolid Airport DE EDLW Dortmund Airport ES LEVT Vitoria/Foronda Airport DE EDNY Friedrichshafen Airport ES LEVX Vigo Airport DE EDSB Karlsruhe Baden-Baden Airport ES LEXJ Santander Airport DE ETNL Rostock-Laage Airport ES LEZG Zaragoza Air Base DK EKAH Aarhus Airport ES LEZL Sevilla Airport DK EKBI Billund Airport FI EFHK Helsinki Vantaa Airport DK EKCH Copenhagen Kastrup Airport FI EFIV Ivalo Airport DK EKKA Karup Airport FI EFJO Joensuu Airport DK EKRN Bornholm Airport FI EFJY Jyvaskyla Airport DK EKVG Vagar Airport FI EFKE Kemi-Tornio Airport DK EKYT Aalborg Airport FI EFKI Kajaani Airport EE EETN Lennart Meri Tallinn Airport FI EFKK Kokkola-Pietarsaari Airport EL LGAL Dimokritos Airport FI EFKS Kuusamo Airport EL LGAV Eleftherios Venizelos International Airport FI EFKT Kittil¨aAirport EL LGHI Chios Island National Airport FI EFKU Kuopio Airport EL LGIO Ioannina Airport FI EFMA Mariehamn Airport EL LGIR Heraklion International Airport FI EFOU Oulu Airport EL LGKF Kefallinia Airport FI EFRO Rovaniemi Airport EL LGKL Kalamata Airport FI EFTP Tampere-Pirkkala Airport EL LGKO Kos Airport FI EFTU Turku Airport

J. Larsson, I. Agren˚ 30 B. Airports .

FI EFVA Vaasa Airport IS BIKF Keflavik International Airport FR FMCZ Dzaoudzi Pamandzi International Airport IT LIBC Crotone Airport FR FMEE Roland Garros Airport IT LIBD Bari Karol WojtylaAirport FR LFBD Bordeaux-M´erignacAirport IT LIBP Pescara International Airport FR LFBE Bergerac Dordogne P´erigordAirport IT LIBR Brindisi Airport FR LFBH La Rochelle – ˆIle de R´eAirport IT LICA Lamezia Terme Airport FR LFBI Poitiers-Biard Airport IT LICB Comiso Airport FR LFBL Limoges Airport IT LICC Catania-Fontanarossa Airport FR LFBO Toulouse-Blagnac Airport IT LICD Lampedusa Airport FR LFBP Pau Pyr´en´eesAirport IT LICG Pantelleria Airport FR LFBT Tarbes–Lourdes–Pyr´en´eesAirport IT LICJ Falcone Borsellino Airport FR LFBZ Biarritz-Anglet-Bayonne Airport IT LICR Reggio Calabria Airport FR LFCK Castres-Mazamet Airport IT LICT Vincenzo Florio Airport Trapani-Birgi FR LFCR Rodez-Marcillac Airport IT LIEA Alghero-Fertilia Airport FR LFJL Metz-Nancy-Lorraine Airport IT LIEE Cagliari Elmas Airport FR LFKB Bastia-Poretta Airport IT LIEO Olbia Costa Smeralda Airport FR LFKC Calvi-Sainte-Catherine Airport IT LIMC Malpensa International Airport FR LFKF Figari Sud-Corse Airport IT LIME Il Caravaggio International Airport FR LFKJ Ajaccio Napoleon Bonapartes Airport IT LIMF Turin Airport FR LFLB Chamb´ery-Savoie Airport IT LIMJ Genoa Cristoforo Colombo Airport FR LFLC Clermont-Ferrand Auvergne Airport IT LIML Milano Linate Airport FR LFLL Lyon-Saint Exup´eryAirport IT LIMP Parma Airport FR LFLS Grenoble-Is`ereAirport IT LIMZ Cuneo International Airport FR LFMK Carcassonne Airport IT LIPE Bologna Guglielmo Marconi Airport FR LFML Marseille Provence Airport IT LIPH Treviso-Sant’Angelo Airport FR LFMN Nice-Cˆoted’Azur Airport IT LIPQ Trieste - Friuli Venezia Giulia Airport FR LFMP Perpignan-Rivesaltes (Llaban`ere)Airport IT LIPR Federico Fellini International Airport FR LFMT Montpellier-M´editerran´eeAirport IT LIPX Verona Villafranca Airport FR LFMU B´eziersCap d’Agde Airport IT LIPY Ancona Falconara Airport FR LFOB Paris Beauvais Till´eAirport IT LIPZ Venice Marco Polo Airport FR LFOK Chˆalons-Vatry Airport IT LIRA Ciampino G. B. Pastine International Airport FR LFOT Tours-Val-de-Loire Airport IT LIRF Leonardo da Vinciˆa- Fiumicino Airport FR LFPG Charles de Gaulle International Airport IT LIRN Naples International Airport FR LFPO Paris-Orly Airport IT LIRP Pisa International Airport FR LFQQ Lille-Lesquin Airport IT LIRQ Peretola Airport FR LFRB Brest Bretagne Airport IT LIRZ Perugia San Francesco d’Assisi FR LFRD Dinard-Pleurtuit-Saint-Malo Airport LT EYKA Kaunas International Airport FR LFRG Deauville-Saint-Gatien Airport LT EYPA Palanga International Airport FR LFRH Lorient South Brittany (Bretagne Sud) Airport LT EYVI Vilnius International Airport FR LFRK Caen-Carpiquet Airport LU ELLX Luxembourg-Findel International Airport FR LFRN Rennes-Saint-Jacques Airport LV EVRA Riga International Airport FR LFRQ Quimper-Cornouaille Airport ME LYPG Podgorica Airport FR LFRS Nantes Atlantique Airport ME LYTV Tivat Airport FR LFSB EuroAirport Basel-Mulhouse-Freiburg Airport MK LWOH Ohrid St. Paul the Apostle Airport FR LFSL Brive-Souillac MK LWSK Skopje International Airport FR LFST Strasbourg Airport MT LMML Malta International Airport FR LFTH Toulon-Hy`eresAirport NL EHAM Amsterdam Airport Schiphol FR LFTW Nˆımes–Al`es–Camargue–C´evennes Airport NL EHBK Maastricht Aachen Airport FR SOCA Cayenne-Rochambeau Airport NL EHEH Eindhoven Airport FR SOOA Maripasoula Airport NL EHGG Eelde Airport FR TFFF Martinique Aim´eC´esaireInternational Airport NL EHRD Rotterdam The Hague Airport FR TFFG Grand Case-Esp´eranceAirport NO ENAL Alesund˚ Airport FR TFFJ Gustaf III Airport NO ENAN Andøya Airport FR TFFR Pointe-`a-PitreLe Raizet NO ENAT Alta Airport HR LDDU Dubrovnik Airport NO ENBN Brønnøysund Airport HR LDPL Pula Airport NO ENBO Bodø¸ Airport HR LDSP Split Airport NO ENBR Bergen Airport Flesland HR LDZA Zagreb Airport NO ENBS B˚atsfjordAirport HR LDZD Zadar Airport NO ENCN Kristiansand Airport HU LHBP Budapest Liszt Ferenc International Airport NO ENDU Bardufoss Airport HU LHDC Debrecen International Airport NO ENEV Harstad/Narvik Airport, Evenes IE EICK Cork Airport NO ENFL Florø¸ Airport IE EIDW Dublin Airport NO ENGM Oslo Gardermoen Airport IE EIKN Ireland West Knock Airport NO ENHD Haugesund Airport IE EIKY Kerry Airport NO ENHF Hammerfest Airport IE EINN Shannon Airport NO ENHK Hasvik Airport

J. Larsson, I. Agren˚ 31 B. Airports .

NO ENHV Valan Airport SE ESNS Skellefte˚aAirport NO ENKB Kristiansund Airport (Kvernberget) SE ESNU Ume˚aAirport NO ENKR Kirkenes Airport (Høybuktmoen) SE ESNZ Are˚ ¨ostersund Airport NO ENLK Leknes Airport SE ESOE Orebro¨ Airport NO ENML Molde Airport SE ESOK Karlstad Airport NO ENMS Mosjøen Airport (Kjærstad) SE ESPA Lule˚aAirport NO ENNA Banak Airport SE ESSA Stockholm-Arlanda Airport NO ENOV Ørsta-Volda Airport, Hovden SE ESSB Stockholm-Bromma Airport NO ENRA Mo i Rana Airport, Røssvoll SE ESSL Link¨opingCity Airport NO ENSB Svalbard Airport, Longyear SE ESSV Visby Airport NO ENSD Sandane Airport (Anda) SE ESTA Angelholm-Helsingborg¨ Airport NO ENSG Sogndal Airport SI LJLJ Ljubljana JoˇzePuˇcnikAirport NO ENSH Svolvær Helle Airport SK LZIB M. R. Stef´anikAirportˇ NO ENSK Stokmarknes Skagen Airport SK LZKZ KoˇsiceAirport NO ENSS Vardø Airport, Svartnes TR LTAC Esenbo˘gaInternational Airport NO ENST Sandnessjøen Airport (Stokka) TR LTAF Adana Airport NO ENTC Tromsø Airport TR LTAI Antalya International Airport NO ENTO Sandefjord Airport, Torp TR LTAJ Gaziantep International Airport NO ENVA Trondheim Airport Værnes TR LTAN Konya Airport NO ENVD Vadsø Airport TR LTAP Amasya Merzifon Airport NO ENZV Stavanger Airport Sola TR LTAR Sivas Nuri Demirag Airport PL EPBY Bydgoszcz Ignacy Jan Paderewski Airport TR LTAT Malatya ErhaA˜ § Airport PL EPGD Gda´nskLech WalesaAirport TR LTAU Kayseri Erkilet Airport PL EPKK Krak´owJohn Paul II International Airport TR LTAY Denizli C¸ardak Airport PL EPKT Katowice International Airport TR LTAZ Nev¸sehirKapadokya Airport PL EPLL L´od´zW ladys law Reymont Airport TR LTBA Atat¨urkInternational Airport PL EPMO Modlin Airport TR LTBH C¸anakkale Airport PL EPPO Pozna´n–Lawica Airport TR LTBJ Adnan Menderes International Airport PL EPRZ Rzesz´ow–Jasionka Airport TR LTBQ Cengiz Topel Airport PL EPSC Solidarity Szczecin–Goleni´owAirport TR LTBR Bursa Yeni¸sehirAirport PL EPSY Olsztyn-Mazury Airport TR LTBS Dalaman International Airport PL EPWA Warsaw Chopin Airport TR LTBU Tekirda˘gC¸orlu Airport PL EPWR Copernicus Wroclaw Airport TR LTBY Anadolu Airport PT LPFL Flores Airport TR LTCA Elazı˘gAirport PT LPFR Faro Airport TR LTCC Diyarbakir Airport PT LPGR Graciosa Airport TR LTCD Erzincan Airport PT LPHR Horta Airport TR LTCE Erzurum International Airport PT LPLA Lajes Airport TR LTCF Kars Airport PT LPMA Madeira Airport TR LTCG Trabzon International Airport PT LPPD Jo˜aoPaulo II Airport TR LTCI Van Ferit Melen Airport PT LPPI Pico Airport TR LTCJ Batman Airport PT LPPR Francisco de Sa Carneiro Airport TR LTCK Mu¸sAirport PT LPPS Porto Santo Airport TR LTCM Sinop Airport PT LPPT Lisbon Portela Airport TR LTCN Kahramanmara¸sAirport PT LPSJ S˜aoJorge Airport TR LTCO A˘grıAirport RO LRBC Bac˘auAirport TR LTCP Adıyaman Airport RO LRCL Cluj-Napoca International Airport TR LTCR Mardin Airport RO LRCV Craiova Airport TR LTCS S¸anlıurfa GAP Airport RO LRIA Ias,i Airport TR LTCV S¸ırnak S¸erafettin El¸ciAirport RO LROD Oradea International Airport TR LTCW Hakkari–Y¨uksekova Airport RO LROP Henri Coand˘aInternational Airport TR LTDA Hatay Airport RO LRSB Sibiu International Airport TR LTFC S¨uleymanDemirel International Airport RO LRSV Suceava Stefan cel Mare Airport TR LTFD Balıkesir Koca Seyit Airport RO LRTR Timis,oara Traian Vuia Airport TR LTFE Milas Bodrum International Airport SE ESDF Ronneby Airport TR LTFH Samsun C¸ar¸samba Airport SE ESGG Gothenburg-Landvetter Airport TR LTFJ Sabiha G¨ok¸cenInternational Airport SE ESGJ J¨onk¨opingAirport TR LTFM Istanbul Airport SE ESGT Trollh¨attan-V¨anersborg Airport UK EGAA Belfast International Airport SE ESKN Stockholm Skavsta Airport UK EGAC George Best Belfast City Airport SE ESMQ Kalmar Airport UK EGAE City of Derry Airport SE ESMS Malm¨oSturup Airport UK EGBB Birmingham International Airport SE ESMT Halmstad Airport UK EGCC Manchester Airport SE ESMX V¨axj¨oKronoberg Airport UK EGCN Robin Hood Doncaster Sheffield Airport SE ESNN Sundsvall-H¨arn¨osandAirport UK EGFF Cardiff International Airport ¨ SE ESNO Ornsk¨oldsvikAirport UK EGGD Bristol Airport SE ESNQ Kiruna Airport UK EGGP Liverpool John Lennon Airport

J. Larsson, I. Agren˚ 32 B. Airports .

UK EGGW London Luton Airport UK EGHE St. Mary’s Airport UK EGHH Bournemouth Airport UK EGHI Southampton Airport UK EGHQ Newquay Cornwall Airport UK EGJA Alderney Airport UK EGJB Guernsey Airport UK EGJJ Jersey Airport UK EGKK London Gatwick Airport UK EGLC London City Airport UK EGLL London Heathrow Airport UK EGMC Southend Airport UK EGNJ Humberside Airport UK EGNM Leeds Bradford Airport UK EGNS Isle of Man Airport UK EGNT Newcastle Airport UK EGNV Durham Tees Valley Airport UK EGNX East Midlands Airport UK EGPA Kirkwall Airport UK EGPB Sumburgh Airport UK EGPD Aberdeen Dyce Airport UK EGPE Inverness Airport UK EGPF Glasgow International Airport UK EGPH Edinburgh Airport UK EGPK Glasgow Prestwick Airport UK EGPM Scatsta Airport UK EGPO Stornoway Airport UK EGSH Norwich International Airport UK EGSS London Stansted Airport UK EGTE Exeter International Airport

J. Larsson, I. Agren˚ 33 www.kth.se