Arxiv:2010.15067V1 [Cs.CL] 28 Oct 2020 Yn Oisadcnetdie Ruig Fatce.Ti Si Cont in Is to This Approaches Articles

arXiv:2010.15067v1 [cs.CL] 28 Oct 2020 yn oisadcnetdie ruig fatce.Ti si cont in is to This approaches articles. of ditional s groupings content; can content-driven interpretable and that extract topics tools to fying for text need raw increasing classify an o and is and info analyse news There to monitor time. to real access time-consuming in instantaneous and formation genera difficult and content it makes journalistic extended media, and with online news coupled of globe, amount the the in explosion The Introduction 1 aeso r-eindctgre n irrhe [ hierarchies and categories pre-designed of labels utsaegahpriinn ehdta sapidt emti sim geometric a to applied is that method partitioning graph multiscale a co text from extraction topic [ work unsupervised previous on research of surge content. news of alter emergence provide and thus generation could categories the or monitor labels d content external on without based text articles of clustering unsupervised automatic, ftx hog o2e [ Doc2vec vec through paragraph text of of advantages the combines that documents text nrcn er,feldb dacsi ttsia erig there learning, statistical in advances by fuelled years, recent In rp-ae oi xrcinfo Vector from Extraction Topic Graph-based plcto oaCru fNw Articles News of Corpus a to Application Abstract. attoigta a eelntrlpriin tdifferen at partitions natural making reveal mult without can from tools that with partitioning processing ve language powerful natural together from brings dings similarity. that content framework from unsupervised corpora an intrinsically a unstructured stem is stratify that in there provide and ‘topics’ text, can areas, of that content amount methods efficient emerging sheer develop the to monitor need creasing and manage help To fU escvrg uigtepeieta lcinya of year election presidential the during coverage news analysi an US through of showcased is work transformers comparative recent This the Bert. embe to vector t Doc2Vec text to different and Bag-of-Words evaluate classic clustering also and popular clusterin methods, other graph-based elling with of advantages comparisons the end-to-end show We corpus. the .TrkAltuncu Tarik M. eateto Mathematics of Department mednso etDocuments: Text of Embeddings 1 ,w eeoe rp-ae rmwr o h lseigof clustering the for framework graph-based a developed we ], rdcino escneti rwn ta soihn rate astonishing an at growing is content news of Production meilCleeLno,Lno,UK London, London, College Imperial priori a etclassification text ∗ 8 ohaN Yaliraki N. Sophia , ihMro tblt M)cmuiydtcin[ detection community (MS) Stability Markov with ] supin bu h ubro lsesin clusters of number the about assumptions ∗ n eateto Chemistry of Department and yial ein nhuman-annotated on reliant typically , † n arcoBarahona Mauricio and , 3 .Mtoooista provide that Methodologies ]. eew present we Here resolutions t dns from ddings, ae model based o representation tor facorpus a of s trembed- ctor eicly identi- pecifically, saegraph iscale ftx into text of 2016. ihsinto sights pcmod- opic rcl rmfree from irectly through g pr [ rpora aiewy to ways native atwt tra- with rast † pre-process, mto via rmation , in- n a ena been has e across ted 6 ∗ . , pinion ilarity 9 .In ]. 4 ], 2 Altuncu et al. graph of documents derived from the high-dimensional vectors representing the text. Through this approach, we obtained robust clusters at different resolutions (from finer to coarser) corresponding to groupings of documents with similar content at different granularity (from more specific to more generic). Because the similarity graph of documents is based on vector embeddings that capture syntactic and semantic features of the text, the graph partitions produce clusters of documents with consistent content in an unsupervised manner, i.e., without training a classifier on hand-coded, pre-designed categories. Our original application in Ref. [1] was a very large and highly specialised corpus of incident reports in a healthcare setting with specific challenges, i.e., highly specialised terminol- ogy mixed with informal and irregular use of language and abbreviations. The large amount of data (13 million records) in that corpus allowed us to train a specific Doc2vec model to circumvent successfully these restrictions. In this work, we present the extension of our work in Ref. [1] to a more general use-case setting by focussing on topic extraction from a relatively small number of documents in standard English. We illustrate our approach through a corpus of ∼9,000 news articles published in the US during 2016. Although for such small corpora, training a specific language model is less feasible, the use of standard language allows us to employ English language models trained on generic corpora (e.g., Wikipedia articles). Here, we use pre-trained language models obtained with Doc2Vec [8] and with recent transformer models like Bert [5], which are based on deeper neural networks, and compare them to classic Bag-of-Words (BoW) based features. In addition, we evaluate our multiscale graph-based partitioning (MS) against classic probabilistic topic clustering methods like LDA [2] and other widely-used clustering methods (k-means and hierarchical clustering).

2 Feature Vector Generation from Free Text

Data: The corpus consists of 8,748 online news articles published by US-based Vox Media during 2016, a presidential election year in the USA1. The corpus ex- cludes articles without speciﬁc news content, without an identiﬁable publication date, or with very short text (less than 30 tokens).

Pre-processing, tokenisation and normalisation: We removed HTML tags, code pieces, and repeated wrapper sentences (header or footer scripts, legal notes, di- rections to interact with multimedia content, signatures). We replaced accented characters with their nearest ASCII characters; removed white space characters; and divided the corpus into lists of word tokens via regex word token divider ‘\w+’. To reduce semantic sparsity, we used Part of Speech (POS) tags to leave out all sentence structures but adjectives, nouns, verbs, and adverbs. The re- maining tokens are lowered and converted to lemmas using the WordNet Lem- matizer [11]. Lastly, we removed common (and thus less meaningful) tokens2. 1 The news corpus is accessible on https://data.world/elenadata/vox-articles 2 The full list of common words is: {‘be’, ‘have’, ‘do’, ‘make’, ‘get’, ‘more’, ‘even’, ‘also’, ‘just’, ‘much’, ‘other’, ‘n’t’, ‘not’, ‘say’, ‘tell’, ‘re’} Graph-based Topic Extraction from Document Vector Embeddings 3

Bag-of-Words based features: We produce TF-IDF (Term Frequency-Inverse Document Frequency) BoW features filtered with Latent Semantic Analysis (LSA) [14] to reduce the high-dimensional sparse TF-IDF vectors to a 300 dimensional continuous space. Neural network based features: Doc2vec [8], a version of Word2vec [10] adapted to longer forms of text, was one of the original neural network based methods with the capability to represent full length free text documents into an embedded fixed-dimensional vector space. Before it can be used to infer a vector for a given document, Doc2Vec must be trained on a corpus of text documents that share similar context and vocabulary with the documents to be embedded. As a source of common English compatible with the broad usage in news articles, we used Gensim version 3.8.0 [17] to train a Doc2Vec language model on a recent Wikipedia dump consisting of 5.4 million articles pre-processed as described above 3. The optimised model had hyperparameters {training method = dbow, number of dimensions for feature vectors size = 300, number of epochs = 10, window size = 5, minimum count = 20, number of negative samples = 5, random down-sampling threshold for frequent words = 0.001}. Transformers based features: Natural Language Processing (NLP) is currently evolving rapidly with the emergence of transformers-based, deep learning methods such as ELMo [15], GPT [16] and BERT, the recent state-of-the-art model by Google [5]. The first step for these models, called pre-training, uses different learning tasks (e.g., next sentence prediction or masked language model) to model the language of the training corpus without explicit supervision. Whereas the neural network in Doc2vec only has two layers, transformers-based models involve deeper neural networks, and thus require much more data and massive computational resources to pre-train. Fortunately, these methods have publi- cally available models for popular languages. Here, we use the model ‘BERT base, Uncased’ with 12-layers and 100 million parameters, which produces 768- dimensional vectors based on a pre-training using BookCorpus [22] and English Wikipedia articles. Because Bert models carry out their own pre-processing (WordPiece tokenisation and Out-Of-Vocabulary handling steps [20]), we do not apply our tokenisation and normalisation routine. Since transformers based models cannot process text longer than a few sentences (510 tokens in total) due to memory limits in GPUs, we analyse the sentences in an article individ- ually; obtain embedded vectors for all; and compute the feature vector of the document as the average of sentence vectors. We obtained two feature vectors from Bert: (i) baas: reduced mean vector of each token’s embedded vector on the second from last layer among the 12 layers, as given by bert-as-service4; (ii) sbert: sentence level vector computed with Sentence Transformers5 [18], which

3 English Wikipedia corpus (1 December 2019) downloaded from https://dumps.wikimedia.org/enwiki/. 4 bert-as-service is an open-source library published at github.com/hanxiao/bert-as-service. 5 Sentence Transformers is an open-source library published at github.com/UKPLab/sentence-transformers. 4 Altuncu et al. improves the pre-trained Bert model using a 3-way softmax classifier for Natural Language Inference (NLI) tasks. Although BERT models are powerful, they are optimised for supervised downstream tasks and need to be fine-tuned further through a secondary step on specifically labelled annotated data sets. Hence pre-trained BERT models without fine-tuning are not optimised for the quality of their vector embedding, which is the primary input for our unsupervised clustering task.

3 Finding Topic Clusters Using Feature Vectors

The above methods lead to ﬁve diﬀerent feature vectors for the documents in the corpus. These vectors are then clustered through the graph-based MS framework, which we benchmark against alternative graph-less clustering methods.

3.1 Graph Construction and Community Detection

In many real-world applications, there is an absence of accurate prior knowledge or ground truth labels about the underlying data. In such cases, unsupervised clustering methods must be applied. Rather than fixing the number of clusters a priori, we use here Markov Stability, a multiscale method that produces a series of intrinsically robust partitions at different levels of resolution. MS thus allows flexibility in choosing the granularity of topics as appropriate for the analysis. Constructing a Sparsified Similarity Graph: From the feature vectors of the 8,748 articles, we compute all pairwise cosine similarities. This dense similarity matrix is sparsified by using the MST-kNN method [21] to construct a sparse geometric graph. MST-kNN consists of two steps: obtain the Minimum Spanning Tree (MST) to ensure global connectivity in the graph followed by linking the k Nearest Neighbours (kNN) to each document. so as to connect highly similar documents thus preserving the local geometry. Based on our work in [1], we set k = 13. We construct MST-kNN similarity graphs from each of the five sets of feature vectors: Gtfidf, Gtfidf+lsa, Gd2v, Gbaas, Gsbert. Multiscale Graph Partitioning with MS: We applied the MS partitioning algo- rithm [4, 7, 19] to these five similarity graphs. We compute the partitions Pt that optimise the MS cost function as we vary the Markov time t, a parameter that modifies the coarseness of the partitions. We choose three partitions at different resolutions that are most robust, both across t and to the optimisation process [7, 19], and label them as fine (F), medium (M), and coarse (C). The three partitions for the five similarity graphs shown in Figure 1 are:

1. Gbaas: [22, 10, 5] communities at t = [0.849, 3.881, 18.557] 2. Gsbert: [20, 10, 4] communities at t = [0.931, 4.159, 14.756] 3. Gtﬁdf: [22, 11, 5] communities at t = [2.037, 9.749, 34.592] 4. Gtﬁdf+lsa: [23, 11, 5] communities at t = [3.793, 15.811, 52.356] 5. Gd2v: [20, 9, 4] communities at t = [1.315, 2.944, 21.329]. Graph-based Topic Extraction from Document Vector Embeddings 5

MS plot with geometric graph from baas features MS plot with geometric graph from sbert features "!%& !!" !"#$%&# .)" !"#$%&# 0" !"#$%&# %" !" !"#$%&'$ -" !"#$%&'$ . !"#$%&'$ %"

'$"$()*+,- '$"$(/*++. '$"$(.+*001 #' "!%& (% %)"*+,- (% %).*-/+ (% %)-.*0/1 #'

#" #" "!#$

"!#$

!' $",$12/3*(./'0( 3 !' $",$12/3*(./'0( 3

"!!" !" !" "!!" )" &"*&"+$",$-+," '"*&"+ &"*&"+$",$-+," )" '"*&"+ &"*&"+$",$-+," )" "!!" "!!" ' '

"!"" "!"" !" " !" ! !" " !" ! !" !"#$%&'( !" !"#$%&'( MS plot with geometric graph from tfidf features MS plot with geometric graph from tfidf+lsa features !!" !"#$%&# --" !"#$%&# 0" !"#$%&# !" !"#$%&'$ -- !"#$%&'$ . !"#$%&'$'" #" "" '$"$(!)*+, '$"$(.),/. '$"$(+/)0.! "!!$ (% %)"*+," (% %)-.*/-- (% %).!*".0 #" "!!$ &"

./'0( $",$12/3*(./'0( 3 "!!# $",$12/3*(./'0( 3 %" "!!# %"

"!!" "!!" !" !" )" &"*&"+$",$-+," '"*&"+ &"*&"+$",$-+," )" '"*&"+ &"*&"+$",$-+," )"

"!"" "!""

!" " !" ! !" " !" ! !" !"#$%&'( !" !"#$%&'( MS plot with geometric graph from d2v features "!'" !" !"#$%&'$ . !"#$%&'$ / !"#$%&'$ '" (% %)*+,*- (% %)!+.// (% %)!*+,!. '" "!%& %"

%" "!%% ./'0( $",$12/3*(./'0( 3 !" "!!$

!" "!!#

)" &"*&"+$",$-+," '"*&"+ &"*&"+$",$-+," )" "!!"

" "!""

"!"" !" " !" ! !" !"#$%&'(

Fig. 1. Results of Markov Stability for the ﬁve similarity graphs constructed from different document feature vectors. As a function of Markov time t, we show: the number of clusters of the optimised partition Pt (red line); the variation of information VI(Pt) for the ensemble of optimised solutions at each t (blue line); and the variation of information VI(Pt, Pt′ ) between the optimised partitions across Markov time (background colourmap). Robust partitions correspond to dips of VI(Pt) and extended plateaux of the number of communities and VI(Pt, Pt′ ). The selected partitions are indicated by vertical lines: green (Fine), orange (Medium) and purple (Coarse).

3.2 Graph-less clustering methods as benchmarks

We benchmark our graph-based clustering against: (i) two widely used graph-less clustering methods: k-means and hierarchical clustering with Ward linkage. Both methods are applied using the implementation and default parameters in Scikit- Learn version 0.22; (ii) LDA probabilistic topic models [2] for each resolution (F, M, C) using the state-of-the-art implementation, i.e., default parameters of LdaMulticore in Gensim version 3.8.0 setting the number of passes to 100 to enaure convergence. Unlike the other clustering methods, LDA does not produce a hard assignment of news articles into clusters, but returns probabilities over all clusters. We assign each article to its most probable topic (i.e, cluster). Because LDA works on term frequencies (tf ), we cannot use the other feature vectors. We remark that, had we not already obtained our MS results, it would not be easy to make an informed a priori choice of number of topics/clusters for any of these methods. Hence we use the three MS levels (F=20, M=9, C=4) as the number of clusters in the benchmark methods. Counting ﬁve diﬀerent 6 Altuncu et al. feature vectors and three clustering algorithms along with the LDA, we have 16 experiments per each resolution level (F,M,C). The next sections are devoted to the quantitative and qualitative comparison of all 48 experiments.

4 Evaluation of Topic Clusters 4.1 Quantitative Analysis of Partition Quality

In the absence of ’ground truth’ topics in our corpus, we cannot score directly the quality of the clusters. Instead, we compute two different measures of the consistency and relevance of cluster content. Measuring intrinsic topic coherence with the Aggregate PMI: Following [12, 13] we create an intrinsic measure of topic coherence without reference to external ground truth topics. OUr measure considers the association of frequent ‘word couples’ in each cluster and computes the pointwise mutual information (PMI) compared to its reference score in a standard corpus (in our case, the English Wikipedia). The topical coherence of articles within topic averaged over all topics d gives the Aggregate PMI scores (PMI) for each experiment is shown in Table 1. Our results of topic coherence show that LDA perform poorly as the least coherent topics, whereas MS partitions are the most coherent overall for most types of features (except for a few cases where k-means and hierarchical Ward are better). On average across all features, MS is the best, followed by k-means and Ward. Regarding text features, there is not a best performing one across all resolutions. Interestingly, tfidf features produce coherent topic clusters, perhaps due to the fact that tfidf and tfidf+lsa features are based on counts of word occurrence, and hence very similar in spirit to PMI scores. Among the neural network based features, baas vectors from Bert work best in the fine and medium levels while d2v is best at the coarse level.

Resolution: Fine Medium Coarse Clustering: MS Ward k-means LDA MS Ward k-means LDA MS Ward k-means LDA baas 1.866 1.692 1.832 - 1.775 1.553 1.717 - 1.656 1.332 1.498 - d2v 1.843 1.660 1.856 - 1.584 1.672 1.673 - 1.724 1.532 1.349 - sbert 1.731 1.706 1.829 - 1.772 1.639 1.638 - 1.482 1.405 1.635 - tf - - - 1.473 - - - 1.292 - - - 1.182

Features tﬁdf 1.844 1.803 1.860 - 1.855 1.584 1.665 - 1.845 1.481 1.209 - tﬁdf+lsa 2.026 1.870 1.992 - 1.579 1.664 1.632 - 1.727 1.602 1.606 - d Table 1. Aggregate topic coherence (PMI) of unsupervised topic clusters (all clusterings and document features) at three resolution levels. Best clustering for each resolution level in boldface.

Comparison to external commercial classifications: An alternative measure of topic cluster quality is to compare to other external classifications. Here we use categorical labels of semantic information produced by two commercial text classification services: Google Cloud Platform’s (GCP) Natural Language API, and Open Calais (OC) by Thomson Reuters. We compare our unsupervised topic Graph-based Topic Extraction from Document Vector Embeddings 7 clusters to the classes obtained by these commercial products (assumed to be trained on clean, human labelled data) using the Normalised Mutual Information (NMI) and the Adjusted Rand Index (ARI) as shown in Table 2.

Commercial Resolution: Fine Medium Coarse service Clustering: MS Ward k-means LDA MS Ward k-means LDA MS Ward k-means LDA GCP baas 0.356 0.335 0.345 - 0.395 0.325 0.355 - 0.400 0.339 0.345 - d2v 0.363 0.356 0.363 - 0.391 0.351 0.380 - 0.425 0.365 0.383 - sbert 0.347 0.312 0.320 - 0.370 0.303 0.308 - 0.381 0.286 0.258 - tf - - - 0.183 - - - 0.161 - - - 0.180

Features tﬁdf 0.353 0.339 0.369 - 0.404 0.332 0.377 - 0.408 0.294 0.387 - tﬁdf+lsa 0.389 0.350 0.370 - 0.400 0.327 0.345 - 0.428 0.313 0.387 - OC baas 0.362 0.319 0.349 - 0.392 0.315 0.350 - 0.391 0.318 0.343 - d2v 0.372 0.349 0.363 - 0.391 0.330 0.361 - 0.413 0.351 0.371 - sbert 0.354 0.309 0.321 - 0.361 0.284 0.293 - 0.394 0.259 0.225 - tf - - - 0.168 - - - 0.175 - - - 0.191

Features tﬁdf 0.364 0.341 0.354 - 0.410 0.310 0.371 - 0.393 0.273 0.387 - tﬁdf+lsa 0.389 0.341 0.372 - 0.378 0.303 0.332 - 0.411 0.296 0.344 - (a) Normalised Mutual Information

Commercial Resolution: Fine Medium Coarse service Clustering: MS Ward k-means LDA MS Ward k-means LDA MS Ward k-means LDA GCP baas 0.171 0.152 0.163 - 0.349 0.241 0.259 - 0.483 0.295 0.306 - d2v 0.212 0.180 0.150 - 0.360 0.252 0.254 - 0.559 0.494 0.415 - sbert 0.161 0.151 0.147 - 0.319 0.292 0.227 - 0.487 0.304 0.291 - tf - - - 0.127 - - - 0.075 - - - -0.004

Features tﬁdf 0.190 0.156 0.167 - 0.488 0.253 0.245 - 0.520 0.195 0.312 - tﬁdf+lsa 0.301 0.173 0.167 - 0.476 0.149 0.190 - 0.552 0.151 0.342 - OC baas 0.171 0.150 0.162 - 0.332 0.234 0.250 - 0.405 0.251 0.282 - d2v 0.215 0.176 0.157 - 0.336 0.229 0.224 - 0.468 0.428 0.408 - sbert 0.173 0.156 0.159 - 0.311 0.252 0.213 - 0.462 0.269 0.253 - tf - - - 0.102 - - - 0.114 - - - 0.019

Features tfidf 0.197 0.149 0.147 - 0.422 0.194 0.224 - 0.409 0.164 0.301 - tfidf+lsa 0.284 0.161 0.160 - 0.399 0.146 0.171 - 0.457 0.171 0.269 - (b) Adjusted Rand Index Table 2. Unsupervised topic clusters (all features and clusterings) at three resolution levels scored against two commercial classification labels: Google Cloud Platform (GCP) and Open Calais (OC). Best clustering for each resolution level in boldface.

From the NMI and ARI scores for all our experiments, we find that MS graph-partitioning provides the best correspondence to the external classifications followed by k-means and Ward. LDA provides the worst scores against the external labels. Again, there is no clear winner among the features: tfidf+lsa, tfidf and d2v (all with MS) produce the best results against external classes at the F, M and C levels. Among the Bert variants, baas performs better than sbert but they do not outperform d2v. As visualisation aids, we use multilevel Sankey diagrams to represent relationships between partitions, and wordclouds to represent the content of topic clusters. In Figure 2, we show the mapping of the external classes of the commercial services OC and GCP against the best unsupervised clustering at the F level, given by MS of Gtfidf+lsa. Our clustering shows strong agreement with the large categories in both OC and GCP with additional detail in several groupings. 8 Altuncu et al.

tfidf+lsa + MS - F Level (23) - Training subset - Comm. 4 w. 1159 documents tfidf+lsa + MS - F Level (23) - Training subset - Comm. 23 w. 63 documents tfidf+lsa + MS - F Level (23) - Training subset - Comm. 14 w. 270 documents tfidf+lsa + MS - F Level (23) - Training subset - Comm. 5 w. 77 documents 4tell story todd vanderwerff 23python part caroline framke 14 doesn range color purple new album 5 showrunners david jeffrey wright george martin song ice MS (F-level) with grey poupon dorne plot second season tf df+lsa david bowie sansa stark batman superman same boat episode second season lady gaga amber rise benioff weiss main character baseball bat carpool karaoke abad santos force awaken america civil episode bring performance ned stark todd vanderwerff elliot robot fear walk dead purple rain season finale best picture pop culture zombie apocalypse wind winter motion picture civil war season unreal dead season kanye west seven kingdoms warner bros second season country music game throne season box office luke cage game throne alexander hamilton third season make sense new black walk dead season man black bring performance halftime show ice fire lady stoneheart faceless men evan rachel wood evan rachel west kanyewest iron fist episode walk taylor swift already know pop music orangecomic new black philipbook elizabeth more less iron man doctor strange test card kendrick lamar album year game throne bran stark big reveal three eye raven throne season alex abad santos episode walk dead stranger things walk dead martin book book readers david benioff star trek video game season premiere OC Topics GCP Topics super bowl season game song ice fire three eye black panther season todd sixth season fear walk music video hamilton cast late show late late today episode bring apple music showrunners david benioff back season time travel sam esmail life pablo season episode todd vanderwerff waller bridge ramsay bolton gilmore girls pet sound king land book stranger much more game throne manuel miranda real life suicide squad rachel wood west kanyewest february plot line spider man first season even more kanye west kanyewest roll stone battle bastards stream service house card dean morgan red wed sand snake star war harry potter darius ruby rachel quinn comic book alex abad white walkers lord light captain america civil lin manuel miranda orange new america civil war mad max make sense rick group jeffrey dean kim kardashian pop star dance dragons arya stark other character calvin harris black women season game throne anthony hopkins kanyewest february book show captain america twice far red woman jon snow black mirror episode second quinn rachel lin manuel late late show wonder woman episode end musical theater night watch sixth season high sparrow new york robot season season finale new york take place 23 jam corden hip hop tony award iron throne little bite first season theme park jeffrey dean morgan shuffle along lloyd webber britney spear david benioff weiss Arts & Entertainment Arts & Entertainment tfidf+lsa + MS - F Level (23) - Training subset - Comm. 2 w. 1938 documents tfidf+lsa + MS - F Level (23) - Training subset - Comm. 20 w. 121 documents tfidf+lsa + MS - F Level (23) - Training subset - Comm. 19 w. 532 documents tfidf+lsa + MS - F Level (23) - Training subset - Comm. 16 w. 155 documents health care new hampshire presidential debate special district republican establishment hampshire primary trump campaign presidential election 2 20 19 16 november voxdotcom bill clinton latelast show stephen weekpolitical comedy tonight bernie sanders unauthorized immigrants jon stewart bernie sanders host trevor trump supporter frontal samantha iowa new hampshire win nomination election day cast ballot white people election day voter turnout federal government general election vote machine vote state major party presidential election democratic national convention donald trump 5 hillary clinton vote cast york time trump say unite state frontal samantha bee democratic primary win state jeb bush stewart guy sanders campaign republican debate third party ted cruz national convention credit report electoral college clinton win presidential campaign samantha bee vote right cartoon hillary vote right act voter lawsfederal land donaldmore delegate trump chris christie percent vote african americans minority voters donald trump trump win new hampshire primary clinton campaign full frontal megyn kelly sanders win white house republican delegate congressional district president obama mitt romney nightly show noah say oliver add john oliver comedy central voterearly registration vote national convention new york hillary clinton cruz win clark county republican party say trump marco rubio sanders supporters stephen colbert fox news new jersey winner take electoral vote rig election foreign policy bernie sanders host john oliver hillary clinton election result voter suppression cliven bundy trump win much more night host new york george bush late night show pledge delegate show stephen republican primary week tonight host oliver say clinton win win iowa super tuesday voter fraud home state contest convention black voters hillary clinton anti trump vote restrictions week tonight colbert say stop trump election officials electoral integrity wildlife refuge support trump trump supporters full frontal samantha last night tonight host john second place more likely show host iowa new make vote swing state new york time ted cruz delegate lead vote count popular vote paul ryan tonight host mail ballot swing state trump campaign john kasich register vote daily show donald trump new york democratic national show stephen colbert iowa caucus right act election rig white voters wall street spell bee trump cruz state vote voter impersonation trevor noah need win delegate stake majority delegate new vote democratic party white work national spell night show bee say north carolina daily show host work class make america late show new hampshire american politics late night host john republican party poll place people vote barack obama last week host trevor noah south carolina long line poll workers supreme court

tfidf+lsa + MS - F Level (23) - Training subset - Comm. 6 w. 605 documents tfidf+lsa + MS - F Level (23) - Training subset - Comm. 9 w. 277 documents tfidf+lsa + MS - F Level (23) - Training subset - Comm. 22 w. 38 documents tfidf+lsa + MS - F Level (23) - Training subset - Comm. 10 w. 116 documents israeli palestinian conflict benjamin netanyahu 6 fight isis 9 right wing israeli conservative party independence party national security trump say trump campaign 22 minister benjamin 10brexit supporters iraq syria palestinian authority muslim americans accuse sexual leave wing british people sex tape york time unite kingdom vote unite kingdom sexual harassment palestinian people middle east trump say accuse sexual assault other word west bank vote remain labour party british pound terrorist attack nuclear deal sexual misconduct locker room talk unite state israel palestine conflict british exit american muslims netanyahu government hillary clinton president obama bill cosby assault women minister benjamin netanyahu nigel farage classify information sexual violence come forward vote leave britain leave south korea israel palestinemiddle east women accuse new york time israeli government european leaders new york sexually harass palestine conflict other countries david cameronmember state common currency trump brag security council jewish state northrussian government korea presidential debate anti semitic foreign policy kingdom vote Politics council resolution leave european secretary state leave supporters clinton email rape sexual peace deal britain exit brexit referendum bashar assad islamic state private email donald trump israeli prime york time victims sexual international law pro israel vote leave european fox news security council resolution leave wing unite nations final status unite state financial crisis nuclear weapons billy bush bill clinton other women bank settlements obama administration terrorist group say trump leave european union assad regime world war hillary clinton israeli palestinian british economy prime minister david white house israel right five star assault allegations room talk trump comment right wing support israel jeremy corbyn remain campaign Politics wing israeli obama administration israeli prime minister foreign policy trump university miss universe ambassador israel donald trump prime minister iraq war president elect rape culture anti immigrant private email server state department roger ail say ail new york palestinian state italian bank locker room second presidential west bank settlements clinton campaign kerry speech brexit vote rape sexual assault alicia machado britain vote leave minister david cameron donald trump brock turner prime minister benjamin cold war prime minister east jerusalem private server law enforcement women come forward palestinian conflict leave campaign anti muslim prime minister european parliament boris johnson israelis palestinians central bank human right sexual assault israeli politics peace process israeli settlement britain vote minister david barack obama unite state nuclear test sexually assault leak audio civil war women come former miss european union saudi arabia state solution leave vote new york time ivanka trump vladimir putin email server grab pussy second presidential debate israeli settlements british voters northern ireland

tfidf+lsa + MS - F Level (23) - Training subset - Comm. 7 w. 239 documents tfidf+lsa + MS - F Level (23) - Training subset - Comm. 3 w. 644 documents tfidf+lsa + MS - F Level (23) - Training subset - Comm. 17 w. 282 documents tfidf+lsa + MS - F Level (23) - Training subset - Comm. 18 w. 211 documents live matter movement video game school government build discrimination base justice scalia kill police year old leslie jones lgbtq advocate sex marriage 7 whole woman health 3 17 18 religious freedom pro life movement police departments abortion right baton rouge york time apple watch tim cook people use federal civil supreme court justices justice antonin black people more gun pepe frog anti lgbtq protect lgbtq abortion laws anti abortion laws tough crime marijuana legalization pokémon game apple help government build new york online harassment align gender trans people use protect lgbtq people court justice trump supporters lgbtq discrimination supreme court rule law enforcement time warner people discrimination merrick garland war drug medical marijuana trump rally new york abortion clinics roe wade anti semitic white people white supremacy lgbtq people farook iphone wall street hillary clinton bathroom gender police officer touch bar right wing lgbtq law search engine orientation gender mass incarceration racial bias donald trump most state supreme court supreme court let trans people justice department reality game anthony kennedy justice antonin scalia movement black gun control abortion access violent crime racial disparities Conflict gender identitymass shoot lower court african americans black americans alt right fine brothers discrimination lgbtq public accommodations nondiscrimination laws plan parenthood fetal tissue mass shoot tech company life movement law enforcement police department live matter real life white nationalist many people north carolinabathroom gender identity reproductive right death penalty matter movementcriminal justice system lgbtq right supreme court nominee court nominee week ban augment reality court justices help fbi transgender people circuit court technology company new york time workplace house pro choice sexual orientation gender locker room abortion ban senate republicans admit privilege criminal justice african american public health shoot kill trump campaign court rule news stories abortion restrictions antonin scalia 22 blog post news sit transalign gender identity people black live matter sex discrimination woman health abortion providers hate speech federal civil right school government court appeal more likely gun violence augmentfake reality game news women health white people use force national anthem people use bathroom anti abortion real world let trans sexual orientation whole woman health care police shoot chuck tingle donald trump overturn roe president obama san bernardino headphone jack police kill white supremacist boaty mcboatface civil right black live make murderer opioid painkiller use bathroom ban abortion pro life gay people election year movement black live Social Issues anti lgbtq law use bathroom gender supreme court justice gun laws justice system same sex court decision hillary clinton background check police brutality socialwhite supremacists mediacatch pokémon same sex marriage orientation gender identity

tfidf+lsa + MS - F Level (23) - Training subset - Comm. 8 w. 551 documents tfidf+lsa + MS - F Level (23) - Training subset - Comm. 11 w. 88 documents Sensitive Subjects tfidf+lsa + MS - F Level (23) - Training subset - Comm. 12 w. 39 documents tfidf+lsa + MS - F Level (23) - Training subset - Comm. 1 w. 838 documents wall street trump tax tax return child care corruption scandal medal count 1 8 bernie sanders 11 black children michigan12 gov drink water high school peer review safe space read more city water level lead risk lead years ago care act tax cut olympic gold win gold environmental quality many people too much uninsured rate usa gymnastics summer olympics trump plan social security balance beam wage gap water crisis york city gabby douglas children lead emergency managers tax policy public option publicmore people health income tax gold medal female athletes flint river lose weight drug price even more affordable care act international olympic gold medals flint water lead water learn more sanders plan federal government past years silver medal national park dilma rousseff flint water crisis new york time right now team usa percent children national team lead paint service line soda tax Law & Government flint crisis health care race walk women gymnastics much moreaffirmative action top percent payroll tax interest rat detroit leadpublic school poison financial crisis olympicanti dope agency game children test most people eastern time Human Interest crisis flint city god exposure risk flint residents new york affordable care international olympic committee lead pip sugary drink add sugar more likely san francisco crisis flint michigan make sure researchers find river water departmentblood environmental quality leaddepartment environmental new study year old health care last year open ceremony coral reef right now healthmedicaid expansion insurance Religion ezra klein show people eat tax credit middle class olympics rio petrobras scandalbrazil economy People & Society stock market track field drug test lead exposure water flint lead crisis rick snyder minimum wagetax plan detroit public happen flint obama administration Sports unite state low income tax rate hurricane matthew fruit vegetables conflict interest unite state simone biles great barrier new york city study find vila autódromo anti dope blood lead level new york gov rick gymnastics team rio janeiro york time single payer meter race michael phelps flint michigan pay more remove lead hillary clinton panama paper women athletes katie ledecky tumble pass dope agency Law Sports michigan gov rick lead blood silicon valley insurance plan elevate blood weightfood company loss lead service line mental health health plan game rio lead service heart disease center mass low level study publish federal reserve rio olympics level children donald trump win olympic long term economic growth pay tax lead levelemergency manager climate change basic income olympic committeeamerican athletes women sport Travel people flint gov rick snyder barrier reef great barrier reef first time tfidf+lsa + MS - F Level (23) - Training subset - Comm. 13 w. 304 documents tfidf+lsa + MS - F Level (23) - Training subset - Comm. 21 w. 114 documents Business tfidf+lsa + MS - F Level (23) - Training subset - Comm. 24 w. 87 documents clean power plan zika outbreak public health emergency sexually transmit hubble deep international space station 13 global warm 21 24elon musk carbon price get pregnant Business hubble deep field coal mine mosquito bear solar eclipse visible light guillain barré mosquito control center disease control light pollution natural gas people infect latin america 12 yellow fever Education ice cream clean power wind solar teen birthrate Technology wind power coal industry electric vehicles public health nasa explain black hole drive car dark matter last year virus spread Jobs & Education periodic table solar power aedes aegypti solar system big bang energy efficiency gene drive air pollutionfossil fuel greenhouse gas aedes albopictus transmit virus Technology dark energy zika spread disease control prevention far away total solar eclipse new year health emergency infectious diseases Human Interest renewable energy case zika plan parenthood astronaut ice saudi arabia northern hemisphere self drive car teenage girls able see light years away international space drive cars power plan birth infectdefect zika nuclear power leap second paris climate electric cars Health link zika space station health officials Health magnetic field dwarf planet ride hail sexual transmission leap year atomic number self drive mosquito bite control prevention oil price long term unite state zika case gravitational wave oil gas carbon tax guinea worm orbit earth self drive cars coal plant zikaworld health virus organization virus circulate scott kelly utility scale stand rock bike share Accident Industrial winter solstice full moon pregnant women Weather light years dark sky zika cause night sky nasa report climate change puerto rico center disease space telescope spread virus Weather astronaut ice cream car companypower plant solar panel unite state world health earth rotation barrel day climate policy carbon emissions deep field meteor shower net meter space weather new elements birthhealth organization control Environment Science clean energy cap trade press statement milky way total solar carbon dioxide rooftop solar coal company disease control teen birth summer solstice space exploration show symptoms aegypti mosquitoes 2 Autos & Vehicles

Fig. 2. Sankey diagram and wordclouds for MS partitions of Gtﬁdf+lsa and the external labels of the commercial services GCP and OC.

Overall, our results show that MS usually performs better than the other clustering algorithms both in terms of intrinsic word consistency and when com- paring to commercial hand labelled external classes. 4.2 Qualitative analysis of the topics Using a multilevel Sankey diagramme and wordclouds, Figure 3 shows the MS topic clusters obtained from Gd2v at the three resolutions: fine, medium, coarse. We find four main topics at the C level, which, using information from the wordclouds, we label as ‘Politics & Elections’, ‘Societal Issues’, ‘Entertainment’, and ‘Health & Environment’. The last one divides into three communities on the medium resolution: ‘Public health & Medicine’, ‘Energy & Environment’, and ‘Healthcare & Insurance’. Among those, the last one also involves some contribution from the ‘Politics & Elections’ community over the discourse of healthcare policy of the presidential candidates. A similar mixed contribution is observed on the M4, a finer resolution community of ‘Societal Issues’. M4 contains news articles related to ‘Black Lives Matter (BLM)’ and lies close to ‘Politics & Elections’. The other finer level of ‘Societal Issues’ is M6 that mixes ‘Politics & Elections’ and ‘Health & Envi- ronment’ as it involves ‘gender identity’ issues in the US as well as discussions Graph-based Topic Extraction from Document Vector Embeddings 9

MS partitioning with geometric graph from d2v features Fine Level Medium Level Coarse Level )84$9$:;$<$=$>"4"1$?8@A$<$B "!(!(5$.%&."#$<$C/''D$8@$*D$EF$)/,%'"(#. &64$7$89$:$;$<"4"#$=6>?$:$@)"-(-(5$!!2!"%$:$A'**B$6C$1B$D>E$&'/!*"(%! !75+8+9:+;+9+<.5."+=>?+;+@$!) ) 0+-%3-.#+;+A"&&B+C+1B+DE7F+!",%&. #- +"3$.'//#! 2 ".!)"(# "() "* +",-./( 2-##$/'!2. 1'*"($"//!!" !"( !"( 2/1!#!,"1$,/ ",#(".. )","(# 1/)5" &!11 #%&'"( 5)"2 +!!!. -4"(3" %)!*+ !#)" !"+," 5. #)" !76#8#9:#;#<#=-6-"#>?@#;#A%!* * 2#/&./-$#;#<"''B#7#0B#?CDE#!",&'- $/ ("#!4"$"'" !,"(. 2"2" $,% "(,3 "//!!" !" !"# ""%) 4"/ .1- ""%) 0/ '" $2 ".!)"(#$"() "* #$%&' -%''"$#.$- -!*+#$%&' "%$) ,!%""* ! #%&'"($ "21"," !" !"#$"!!"!#% !7/#8#9:#;#<#=0/0"#>7?@#;#A%!* * -#.&1.0$#;#B"''C#D#3C#EFG#!",&'0 $. 1!-() 0#" +'"-# "21","$+",-./( (%0.*!0 $*!" ,!'(!*- ,!&'!)0 +#$!)" 4!2.+ .1- $.'%3"),! + !#)" !" 0/ '" .1"4" 0' $("1! %)!*+ !". ."")$'#& ,!&'!)0 &! !0.$ 4"%-*2 #(""*,+ 2% "$2/1!#!,"1 +",-./($0 /(#$&!11 !" $%"+" *!!#"$'"( )"+"$!" !"# $-!#,%&3 '"%-#"*1-"+ #!11$5!# &""$2/!(# *"#"(-"$%)!*+ !" !"##.$"!!"!#% !" !"!#$%&'( (%-/*!- $ ".!'! */'"( "'" !,"( "(.1"4"$&1",- !". "-# /,-#& /")" .! !0%.#.&(("%$0%. $%&'(#& */0%.*$+ '!40 !'0%*,! ()""!$*+,") #" !83#9#:;#<#:#=-3-"#>?@#<#A%!* * 2#0&.0-$#<#B"''C#8#/C#DEF8#!",&'- $0 Politics & Elections 0 /(#$&!11 #%&'"($"'" !,"( '%,")$1'*"( !" !"#$4-'#"(/" %0(&1"*,! "'* 00 5&"" 5%" $!" "'" !,"( !!.#/ 3 .1"4" $"#!(!,$,1""(." "'" !,"($,% "(,3 %)!*+$/'**"(% 1'*"($/'*" *-!!$!(-4")!" ,!'(!*- $%!*" 1()#. ("%-. ,"&.+4"$1!$! (%-0*!- $*!"#-"-,$*" 0&(-%#$&-0!!+ 03#+"%4 %0(&1"*,! #(!%$+ !.&",$!#),+ !#)" !" #$%&'+&!2. (-%,- $#3"$- .'!(5$1'*"( )'5")$"-# (%0.*!0 $*!"#,! !*!!$0 3"$-%#6%!&! #'/3")$)''* 3!.)* -$" #(".$ 3)*$0#)"&.0 )*""!%+#,"* $" !" !"#$#%&'"( 1'*"($/'*"$0')1")& -"-,$"%!"#3"$- +",-./( .#!11 +)"!-&"(%-"#$&"2"%" 10% *0#.! !0%. ""( 4%""+4$" #!" -./%!"+!--!%"#""( 0! !-%0#0&(("%$-%0 -/ )!'(0)*%- !0'",%!$*,#(!%$+ )5! 2!+#$%&' $%&'( 0&(("%$-%0 0)*$-#)"&/- "() "*$+",-./($0 /(# "#!(!,$,1""(." %)!*+$2)"5 $0! ,%&2 #.! ,$%6 *"$2+#)&. 0","$&!11 "#!(!,$,1""(." ,-##"). /#-(%'( $%&'(#%0!"!" !"!$%&'( '*,)0""0#"1!'! -./%!"+(!$!--&. # ("-!2-#!-"-2!$- .!%!,1#".!'! ("1 .')3 %-*" .$0()0 #,""10%$ (%0.*!0 $#1!%!,4#"1!'! (%-/*!- $*!"#-"-,$*" #*"(#3$)/11" . !". %)!*+ 4!,#+,(.,2 2- -%!"#-"-,$*"!-'",%!$*,#(%*'!%+ 6%""($"1!7"&"#! 2"2" $'/("3 *')"$1'*"( !!)"* -("1 0/* 2#0$!$- $%&'( ,!'(!*2 -0#+"%1 &1",- */'"( .1"4" $"#!(!, !" !"#$,")"!!*"(% )"+"$/!#%!)" )*""!%+#,"* $" #$%&'+,!&'!)0 &.0* +2.""* 0"%1#,"!// 0-)!%$#"&. ,"* $" #,!'(!*- %0(&1"*,! # !$*" !" '$.-)!. #)!"+,! !)!!#. !$*" !" ," 6- $*" '&,)#'"%- $%&'( 0* # "".% 3 )"2" #'"(# ("1 .')3 #""3$"!&-' -0 0%!"#0"0,$*" '0-+ #40""+ '"-" #%&'"( "!!"!#%$1'*"( !!*"+ .)"3 #$%&'+$.!"!" !"!#$%&' 2)'/3$%!)(") .')3 %-*" .1 *"$2 .*"" ,"* $" +",-./( 0 /(# (%0.*!0 $#1!%!,4 !" !"!#$%&'( !'-%*,! #(""*$*,0 *"/! ,!&,&0 .1"4"$1"&/ 1*""#,"*$%&'(#,!'(!*- $" .23 %(!/( 2 ".!)"(#$"() "* "21","$"() "* )"+" !" !"# "!!"!#% '!%,"#%&1*" '*40#(0 ,0 .!%!,4#".!'! & *$- /$!$- &'("#& %)!*+ 1"!,-(5%'($+'!% % )#.+-#!#. !"# $)0(# /&(%-'-#,"&%$ !$*" !" ," /0 $*" $%&'(#$%&'( 2- -%!"#-"-,$*" "() "*$+",-./( )''*$%"#3 &."! )!+#$%&' $%&'(#/* '!%,"#%&.*" /!+#$%&'( #%&'"( &!11 2-##$/#-(%'( !" !"!#$%&'(#%0!"!" !"!$%&'( -./%!""*+!--!%"# 0&(("%$#$%&'( ,"* $" #,!'(!*2 -"-,$*" #!!+ #%&'"($0"," /'*"$0')1")& "#-/-"$*"/,"&' $%&'( .&(("%$0%. #$%&'+-!* '"%-#"*4-"+ -7*$#(""" -0 +"%1 $*'- .1"4" 3 2%# !" !"#$*-!/'(&!/% '0"! *!#$%&'( #$%&'+% )5.$-)#* &",-$&!11 %)!*+$/"*+"-5( (%0.*!0 $*!"#!01!$0 3.$ ).+-! !.$- &!2.+!&.$),! 2%# #%&'"( *"(.$1'*"( %0(&1"*,! # !$*" !"#," /0 $*" ""%# ,") #" +,!&'!)0""%# $)*%! (!%$+ !"$#%*2)$ .-% *-#/! !-%/ .!+#$%&'( .!'! $)!#100 -",)!"+&.!)! 4)$-#+#)&. .-% *-#0! !-%0 2% "$2/1!#!,"1$,/ ",#(".. &"(-$(/#" *"5.($3"##. 0! !-%0#,!'(!*2 (%-/*!- $*!" ,! !*!!$- "21"," "() "* +",-./( #'/3") )''* %"#3 1!%!,4#"1!'! ."!,4#3"$-%0 "!.$# *-)$ $%&'(#.!+ "!.$#3004 /74"8"9:";"<"=!4!&">7?@";"A(")#)#*"%21%!'";"B,..C"7D"$C"EF"/,-2.!#'% !75,8,9:,;,<,=(5(",>7?@,;,A$!* * 0,/%)/(#,;,B"&&C,DE,2C,FGE,!"+%&( #/ !0'",%!$*,# !$*" !"#," /0 $*" !-'",%!$*,#(!%$+ -0#)!'(/)*%- $%&'(#/!+ !" ! #!$% %)' %,-)"& .!/)" 0&"'!,(. /*,0#(%0.*!0 $*!" !" !"! #$%&' /)*$-#)"&0- ,!'(!*- #'! !-0% 3!$!,2 "3!&! &),(."". "3!&! ("(&"!%#3"$- 4"5# -0/ !'-%*,! #(""*$*,/ &!%&)!"3,#!% 0!0!"!(,* 5"6 03. !"% '$.-)!. #)!"+,!&'!)0 0"&$) ,!%""* ! !"% %,-)"& 32%')-! #!$%"%)' .*""!$- +"* #" & *$0#.$!$0 /-5&!" !//!&"$ '!%,"#%&.*" .!/)""0&"'!,(.% !0'",%!$*,# !$*" !" !(!! %0!!-+ ,#&)#!"+"("%%.!#' .!#"()*+' .(!"#.,+!$( $*0.#,2* 0 .1+*"$2+#)&. 3)""+,") #" !"% !"% 1&,* 0,%' "&'"()*+'".,4!.!#' 2"$3,+"!// "",2.$ $""& !.&",$!#), !#)" !" ," 5. #)" % *#(,/#!#( '""*#*+!",/-/#(& 2.*#(,2"$3,+"!// #$%&'+$!""* %-(&."*,! #(!%$+ .!/)""0&"'!,(. & *$-#0$!$- 2-"%2-#.&0) ."#6"0!,0&! '$.-)!. #)!"+!.3!#. 2.*#(,."%/( !"#,$*0.# 5") #4!0*,) -"-,$*" #!!+ !" !"!#$%&'( !1$*+! ,!&($*+! / &*!!"(,+"!// /"%4#,"!00 "&'"()*+' '$)''!("2%!(% (2,-"$3 !" !"!+#$%&'+$.!"!" !"!#$%&' "%$)#,!%""* ! ,"* $" #0* ,"* $" ,!'(!*2 %!"(-+ (!%2&' $+!(!"$+)'! $.'%3"),! + !#)" !"+," 5. #)" $+!(! $+)'! 0!,0&! -&)- 1")'"+!"/&)#! 0("$0(,)%/. )($ *( /! !($/ +"%1#$*'- '$(/*!( #,")!&! /%''"$#,#$%&' )72!8!9:!;!7?@!;!A(#'!'!3!&-1&",!;!B*$$C!7! C!DEF!)*/-$"!,& /&(("%$#$%&'( )*"",+"* #" %"(/"!,#3" %*'!, !#,'*!#+!%*++ 0!""#/$%--$ $%&'(#,!'(!*2 #$%&',+!&'!*0 %*++!&"* "5',!%*++ ""#$ ""#$ )*""!%+6"%-*2 #(""*,+ ,"* $" $%&'(#%-!"!" !"!$%&'( %,-)"&".!/)" &%+.,&"$( 2!"",/#$((# -"-,$"%!" ,"""-2- .!.1!(%""&' ,(-$%!/(-0 (%-/*!- $*!" !-.!$- $%&'(#/&(("%$-%/ !" !"#!$%"%',()!% 1"(!'"!&#!)"(& 064%7%89%:%8%;!4!'%<=>%:%? -"$"$#%,*2,!+%:%@"11A%B%3A%C6D%0"(*1!$+, -/ +"%4 /* #0$!$- /! !-%/#/&(("%$-%/ $+)'!"0!,0&!".!!' .!#")%%2! '* #!/#-/-& 6"1!1-&" /)*$-#3"$-%0 (""" 0)"/ !" !"!#$%&'(#%-!"!" !"!$%&'( !" !"! #$%&' '*$$#%"' -+ 2-"%2-#.&/) 1"6, (2/ '!,'!)%,+-.' $-!#,%&1 !&($*+! ,'""*#*+/ #$%&',2* "'++#(.!/+'!,*! $*('!- %3!-&"$, +"* #" +!&'!*0 -!%"+#3"$- %-(&."*,! #(!%$+ #(!,'!$#- )!$!+3 ")!&! 1#(#/4!*1#$# .*-!3 2*,"(& )"$*/(#,'/!%#(,. 2- -(/%"2-1- ,"* $" /* -"-,$"%!"#,"""-2- %("&')"!,'#+!"+"/,'*! & "1! 1"$",+! 0! !-%0#/* !"1&"- "$,.!# +)&&"(6"-&)#',#!"#!$%%"#/!(%"%200,('!(% 4'-0"1" &*+"$ $*('!- %+!,+ %"(/"!,!2*," /,#"&/"'(2.0 $('%)"*+! ,'!$#- /+'!,*! '! -!%"+ 6"$- +"'! %0!!-+ &"$( "*3("- '("'"( +"""$ &-%"(!,-"&)#. "',"!2*,"(& %*%-+#(!2*," " -5%3- 0!/)* 2$" #("/$ !-'",%!$*,#(!%$+ /"&$)#,!%""* ! ()*+'"$)#* !-+, -+".1!( ()*+'".,4!.!#' '!%" $-! #(!,+$%4 +*5*",$*0.# '$(/*!( #*!",+!&'!*0 &#!)"(& /#$%#'3! 0"$-'0%+ *1& (%-/*!- $*!"#,!'(!*2 .!.1!(% "&' ()*+' !"-!1,, ""&*,()'+. 1255!!!/"4)/!, !" !.*(4 ('"$+"$%(-1&-"#$ +"(".1! .!.! $!+*!",$(/( #&( # 1"$(*0 ,'""*+- $#(/*!(-1'* $-+""$-' ,!(* "+) ,"4"!+%*$""$ (!"&"#!$% '!!&,#," 1+#/4 2*,"(& & '!3!&,#," .)&,"6)"##,0,2&,% 0( ($!",("(+#*" 2.*#(,'("'"( !"-'"-+!- '""*#*+!" '!$#- 2.*#(,2"$3 )"!+3,'("'"( '(2.0 %200,('!(% #!$% ,(*"#)5"'),#% & !,"0!$+%"2-1- 1"00'!%!-,+ %,-)"&".!/)""0&"'!,(.% !"-'"-+!- !( "(!5(,2* 0 ,+-+*,%5*" /8-$9$:;$<$=$>#-# $?@A$<$B&(!)!).$*3+*#'$<$="11C$@$7C$DEF$/""31#)'* !(&"+$!#*+,'!$#- )*!#+)!,(-$% *$"+!%$-+""$, ('"$+"$%!"*$0-+""$ )" /%+"1! *!( #*#-,'""*#*+/ 2.*#(,5"#($/ /*!3("&&'*!#+!)'&,('/, .+'5$ (7 #!$%"%',()!% $+)'!"0!,0&! 1!$,$*0.# "+"/,*(#+ 2*," &#!)"(&!&-%%*(,"(& !#"! #$"" "& #$%&',/%''"$#($/ %*++!#2"(#3" ,"'() %#(,. #!$%"*!#)2% +!&'!*0 ,1* ! +( #$%&',/!- "#(+.!2*," $+)'!"%20(!."-)%' #!$%"!"-!1,, /! !($/,/%''"$#($/ !"")! '! !*(,"!/#(*+'!# ""% !" !"#$%&"'"() ""% *#43( $(**(3 ' 3."+!%."*,! *,,"-$%#"4! $1!$+ .3)$"")'&" "!-! $&!.6' ,(-$%)"+"3#," '! &,#4" %+")3"!)"+"3#," 57*$8$9:$;$9$<#*# $=>?$;$@&(!-!-4$0/'0#,$;$A"++B$C$1B$D7>$5""/+#-,0 !56+7+89+:+;+<-6-"+=5>?+:+@$!) ) 0+.%/.-#+:+A"&&B+CD+2B+CE5+!",%&- #. 064%7%89%:%;%?%:%@ -"$"$#%/.1/!*%:%A"22B%C%3B%C>D%0"(.2!$*/ 6*"!!4#&'/" $!3%)" / "+"/,'*!!)#. )"$*/(#,'/!%('$#(. !-&",$!#), '!$#* "/!&! .!* 2"&- +0*& !.#),. !2-"'%/! 4! &,#,"!2*," -,,-0 !#"1! +"&#$4/- !" !"#$'&/,( !,2 /0#$%"&"# ) #-$ !#)" !"+""*&'),+,"&&)##-- ,-*0" - -2"- !"#+*%"/"/ 2"00'!%!-/* ('-,,"!) "$!" 1-+""$ " -5%3- !" !"#$%&"'(" ,+-+!%0!&- +1!$+ & !,"0!$+%!'!(+ !" !"#$"%%!"#& !#"! #$3*# "&!1!)( 23*'!"# *0*'#1 ()""!$*+,") #" /*-*!%0!&- *2!$* 3"!"(#+!"+"/,'*! ("%-1+'/#!!%#(,. Societal Issues &!1-+!&-$),!+0$-!# 2()#-+("%.- 3" '0 3- 3+"*! +"./! /!(. "*)%(".$("' & "4-+!%,! 4! .3)$-!" #)"# &- 2"&- &#!)"(& '! !" !"#$"%%!"#& )" ,%*"2! $*(" +'4"+. )#7$0"&, #-!& %.! ("'0%3- /+'!,*!!/#$%#'3! ,") /(-0 $" +.%/" !- +"&#$ !)# 2 *6""'$,! $)"+""*&'),. &)!!"-+,"!.. 0"$-'0 * .2& ("4"' 3- ./0,!"#$020,#+ )('!")( $()'6#1 ""*&'), ,"&&)##-- #$%&'+.!* & "4-*! !2-"' "+"/,*(#+!/*++"3" 2-,.- %-,,-0 &("!( $'!(0 "1-2-%-02"$"/* -*""$ '!/'!) /*-+' /+'!,*!!+"#) &*-,"!/#(*+'!# + *1&%,-) 03"",$)! +"*#+#-, ' (") 0#(&$" / -//-0% !#"2! & !/"0!$*%"1-2- !"& 3" '0%3- +"*#+#-,$' (")$ !*# !"& + (",$!#"! # ('"$+"$%!1-"' $!3 )" / +"1! .-,$-#!$*+,") #" ""%& & "4-+!%!1-"'%,! 4! ""%& 2#(&$" 5 ,! $!" !"# 2! #+#(! 1 &),(-""-+"/!&! 0""!( +#5!( ) #-$ !#)" !"+""*&'), $)0(# "2 !*#$+(,,#& !" !"#$*6""' & "4-*!%/!!" 4!!"#$%&"'"() $*('!- %0!-' -#1$2"&)$"!,2 $" *+%," !-$ /*-*./%5." &("!( 5!0!(&!,!#0 (%&!"()$(1#&!"()* *$"+!%,+-+!,!(* "+)%("*$("' 4/-$*!" #-"# 4/-$ (10 )"&'6$"(&" !)( !" !"! #$%&' * .2&%/-) 3!,+%2-$/ !1-"'%,! 4! -(,!"-( $(-,3#+ !*#$+(,,#&$+"*#+#-, *3!#$""3&' ()'!$ .+'5$ (7 0$-!#+!0!) ('"$*"$%!".$0-*""$ $" *+%," !- *"% 0+'-"'"- '$-.)!- #+"/!&! & "4-*!%!2-"'%/! 4! $.('!- %0!-' & "4-+!%!1-"' .#!#-+% )" '$-.)!- #+% )#-+.#!#- ,!( !+- )%,+-+! $)" 4! -)$" !"#.+ ",", ,!( !+- )%('"$+"$ !#"!(1$#-%"&"#+#-, #$"" "& .$"*!%/*-*! 4'-0"2" %&.*"$ -" !" (! .*1-$ "#.+ 2!"" .#$--# "/'-2"(%/*-*! /-!,# 0,(,# .#)/#&$!/#)'!'0 ""*&'),+0!&- 3!/*%1-$, 4/-$""-,&" 1(** !)"(&"#&('!") .#!#-+!&-$),! ,"/43 "&!+# (%&!"()$(1#&!"() !&-$),!+0$-!# $!3 )" , *"2! ""& ""& +(,,#&$+"*#+#-, !" !"#$)! 23*'!"#$/#!(&'1#)' '$-.)!- #+% )#- /!( !*- )%/*-*! ("'0%3- /$-!1+/!$$)-$. $-*""$-'%/!(. "*) & "2!%2"$"/*! ."''- ) ('"$+"$ '(")4&"/-5 "3#") !&-$),! +'-"'"- $.('!- %*!/* .//"-$%#"4! $2!$* ,"*+.%/" !- "2-1-%-01"$",+ -+""$ &("!( ./0,!"# ,") #" +.!* 56*$7$89$:$;$<#*# $=6>?$:$@&(!-!-1$0/'0#,$:$A"++B$C$4B$DEE$5""/+#-,0 ' (")$!#"! # .)&" -+/)"-. $!3 )" , ' (")$+#- #$!,1+3)-"! +(00 !-"(&"#&(,!"- (%&!"(-$(+#&!"(- "&!1!)( $23*'!"# +)0 "#!$%1"21 /-.0"%- -1"- !" !"#$'&/,( !,3 (%&!"(-$(+#&!"(-0 &("!( $+!(* "'- +,-$-&" * 2) 0""! +.2-$% "#+* -(,!"-( (-,2#+ )! !" !"# -#1 2"&) 76!'#$!#"! # 1(**$*6""' *#43( "&!#)'('!") /-$ )-+.! !-$. $.('!- %3!-&"$/ .$"*!%$-*""$/ ()'!$ .+'5 0""!+&-!!" &!-(!%0!-' -#4$3"&) +(00$03"", *!" #-,$"&!+# /)""+,") #" '-#$"/$!. .,! !!" " -5%/) "- !*#$+(,,#& +"&# 1/- ,#&&"&!0,$(,,(") !" !"# /#!(&'1#)' (-!"#(+,!$- 0""!+&-!!". ("4"'%3- &("!( $'!(0 )! !" !"# -2 (!&'.()$- /".*+%," !- ('"$*"$%!2-"' +"*#+#-, ' (") ,96):);<)=);)>"6"*)?@A)=)B!'-(-(+)!"0!"&)=)C%##D)E)5D)F@F),%$"#"(&! 3*#$%"&"# *-!$.+!0" !" !"#$03"", !" !"#$+&3'( !'0 % )#- .#!#- ('"$*"$%(-2&-"#$ +"''- ) ('"$*"$ ""#( !" 0'()'0%!&-%( ""#( !" + (",$ !-# !" !"#$"%%!"#& *+0&1)*'5 *"&)&!'(! 13!,#$!#"! # ' (") !*# -2+*"$1 ('-//"!) "$!" 2-*""$ +(00$!-"(&"#&(,!"- /-!,#$0,(,# 0#6/( $(00(/ , ' (")$(+#&!"(-0 &!1-+!&-$),! 3)$.#+"!!*% )#-+.#!#-+!&-$),! 1-/+- %-//-0 0-!&/ $%(&!%* 3*#$+('6&""1 +"&# !)# 3 '(&-)'0%!&-%( !"2"'*)%!-"(&'&-%()+"(,"! $%"!& 8"!&-$" 1"&# !,# 0 '6!#7#89#:#;#?#:#@"%.,.,1#0&30! #:#A"22B#6C#/B#DE#'"+&2!, 0 ,75$8$9:$;$<$="5"($>7?@$;$A!)-*-*1$!".!"'$;$B&##C$D$2C$7E7$,&%"#"*'! ,"/12 "&!+# 1/-$""-,&" '(&- *+0&1 *'5 .+'5$!#"! # 1"&#$.3) *(1#$*#4 &.)#)$*&#-*)'" *)'"!)($.")! !" !"#" %&"!' 3"!'-%"! !" !"#$02"", +(00$02"", 5%#"()/"'*&/ "&!+!-( $./0,!"# !" ! "!0 ".+ .",0 ).&!'-&*$!-1+' 5"-( 5$,&/+! "&!+!-( $./0,!"#$030,#+ (%&!"(-$(+#&!"(-0 !" !"# 5#!(&,+#-, !" !"#$,! !" !"#., .2.'% .", #"!!-%4$1)!()*, $-6-*)!-+/& *'() '!"(&/%%, "&2*#+%2*%.1, *"(( /""4!"0 !5 "!2!()#"%"! %#5#&( 1"*#&-+#-, '(,"-$&"/1# "!*! $&!43, !" !#0 % ! ).&!'-&* !&5-,"!! ).&!'-&* %(-*-%! !$$!+ #!" !" 8"!&-$")'(&%(-( !"2"'*)%!-"(&'&-%( +(",.&"3)/ "6#", '(&%(-()!$'*-' /"'*&/)$'!" '&()*$!#"! # !(!+ .", '%) "&!+!-( $./0,!"# *%$7"!)!%%# !" !" '&()*.#)/#&$!#"! # !" !"#"$%&"!' (%&!"(-$(+#&!"(- !#"! #$"" "& ' (")$ !*#$+(,,#& &("!( /!*!(&!'!#* !" !" 0&**"!00.", +")('+$%)!" ),#-'$ !-5-("1" 2+&("$2&#)*$+")('+ 5!0!(&!,!#0 !" !"# !#"! #$3*#$+('6&""1 !" !#".1- 2""4$.)* 3#(& " 5 "&!+!-( $./0,!"#$020,#+ 5"-( 5$,&/+! /"')($'-!!"" +"*#+#-,$' (")$ !*# +"(,"!)-,"(&-&. %#5#&( $4"*#&-+#-, ,!/#!" !#"!0 ".+ .",0 )*'&*-*$!%)(-) '!"# $!)6 "" "" 2""!#(.4!() *"((#/% +-!"0 1/-$*!" #-"# 23*'!"#$*0*'#1 3(%+4#!" !"0 !&"$2)," %&"!'$3"!'-%"! !" !"#$5#!(&,+#-, &!'(!+"(,"! "% *" '&()*$!#"! #$3*# "!1.0 !"#!" ! !" !&,"%'-5"$!-1+' !*# +(,,#& +"*#+#-, ./0,!"#$030,#+ !%&"$& *+0&1 !" !%,"$&-6")!-+/& !-#$1(''#& +-(()!6 %(-*'&* *"((#*(%+! !" ! 2%.( !" !"#(%/ ".1#!(!+ .", 3"!'-%"$)*'&*-* 42!,#$!#"! # "% *")"!" + (",$ !-#$1(''#& (7$#)%"&"#1#)' %$".+%,#%2!".+%, !&$%+&-%" &.)#)$),#-*-!'!)'-&* ' (")$ !*# +(,,#& +"*#+#-, (%!&/)$'!%*-(' 2.,"". )#!" !"0 ).&!'-&* )%%"!! &("!( $5!0!(&!,!#0 %!-"(&'&-%( +"(,"! -,"(&-&. 3%((" #3"5 !" !"# &","& *!"0", !" !" "*-'" !')'" +()# +/&5#&#& (4 #-%"&"#+#-, "('"&/%!-4",)-##-+!'(&! %$".+%,#%2!".+%,0 +"!'.3(!#%((!1% .",0 !&$(-/" "("%'-&*$6")! ./0,!"#$5#!(&,+#-, 0""!( $+#5!( !"!-,"*'$&.)#) /0#$%"&"# (",1#(.,! 3"!'-%" )*'&*-* !%)(-) ' (") !#"! # !" !"#$)! !" !"#$"%&' !" !"#"$%&"!'$*&#-*"" *!" #-,$"&!+# 1/-$ (40 !" !"# 5#!(&,+#-,0 !" !"#" $%"!& 3"!'-%"$!%)(-) +%0 #3%((" *!"*(!#!" ! .-!'+ %&*'!&( %!-"(&'&-%()+"(,"! 0&*"!2!#+"&" ""% !" !"#")$%"!&)8"!&-$" ""% .82"9":;"<"=">!2!-"?@A"<"B%")')'!"/0*/!#"<"=& C"D",C"EFEG".&(0 !'#/ !" !"#"!1.0 "% .", ,!/ !" ! ).&!'-&*$()2! ()*$ )!"*'+&&, !% $/%-$" #"!!-$7)+'!*'(, ,"" - +%""(.,% "!*&3(.+%,0 %&"!'$*&#-*"" ' (")'(")1&"/-5$"2#") !*# +(,,#& 5%#'()/"'*&/ %&"!'$!"(" 2&#"*$+")('+ ,&/+! &( 3 02"",$)! ' (") +#- !"' !"' ("1#")'"" !%)(""()2)- /&()"-" !.)" *-"(+"1"'#$!% ).&!'-&*$!"!'!-%'-&*! '0%!&-%()!-+/& .61#7#89#:#8#;"1"!#<=>#:#?% *)*),#+&3+""#:#@'//A#B#4A#C6D#.'$&/")"+ ,"" -#+%""(.,% ).&!'-&*$.)* 065#7#89#:#;#<"5"!#=6>?#:#@% &'&'*#()3(""#:#A-..B#C#1B#DCD#0-$)."'"( !" !#".1- #%+ )*'-$).&!'-&* &!'(!) "% *" 2*) )$* !#$%*+*+ !" !#2%+-.,! &5"!'"!*$!&" ( -%'!!#" 0 ".1- !" ! !" !"#.2*!"0",% .", !(!+ .", ".1 ,&*)(,$'!"# 22-%0 3!" $ %" !'#" !"2 "% *")"!")0'&/!%%# %&"!'$3"!'-%" !"*)'"$!" ".(-%)*! 2"0"% !#%"("%5" %0'#' ',#-(-!&!'&-%( (%*/"#/*)*+""% $ %"# $" + ,#" / (-$& ! ("$)%&", %*,!"#)'4 *)+&% )$"#$'/( )- !" !"#$%&'! !" !"#"$%&"!'$3"!'-%" ""' !"2)#'!!-'+"""' -)'" "'0!- 3)%/# #) ! !%"()#!" ! 2%.(#3%((" .)*$).&!'-&* 22'%. 3!"#$ %"# $" !)- &%!"!)%-/ 3%((" #0!($.!0 2%4!#!" ! *+0&1 "% *" !%" 5'," -)##-!"*)#! ".1- #%+ "*)"'+&!-0",$-##-1!)*'! !" !"!#$ %" "% *"),-!$!-#-('&-%( ( - /'%" )*'- ).&!'-&* ()2! (&2"! %&"!' ,%('*,)&!"# +*),!"#( -"% '",%(' "&2* !" !"#(%/0 2+&("$2&#)* 22-%0 3!"#$ %"# $" $%"!& !"*" ("%$")"#("%$")" #"+!"1-"(! +%""(.,% "!*&3(.+%,0 !(!+ .",#"$$.+.%(0 !" !#+"&, 2&#)*$+")('+ $-'2!&$"#&'""%"(" &'()% '$"#$-.+ ', /&$!#/'%"

!%)*-3" Entertainment 8"!&-$")'(&%(-()!$'*-' &)*"" 5*),.'/ " 0 $&" 3)%/#"/!"/&' !-1#&'$-." "!")0'&/!%%# "$')'/*$#,%'4"! %)!$#"'&, !" !"!#+! ' &'$-."#" / !"*-+-%"! 3!"",%# 1'""#!" 1" )74#8#9:#;#<#=%4%'#>7?@#;#A" *!*!(#"0."%!#;#B&11C#7D#3C#EFG#)&-01%!!" .7'%8%9:%;%<%=!'!)%>7?@%;%A #&$&$-%+"/+!(%;%B!**C%DE%6C%DDF%.!1"*!$(+ + ,%-!!#" / 4!"%"&-. -*4*'#"*(,!#' 3" !(2! 1!"$( &!+ &!'(! "% *" "!" $!*!.#$ %" /#%"'!!%"#$)'!/ )'&"" (" "" '(&-)*+0&1 2"."% ! ,'1"%)/")" 6%)%" '#-*4*' )'&'()%"0#% "" " /#$)" /()!'(!"3)(#)&' " !%)(""()2)-","% !" !"(%!)%"#$%&$'% )!#'! !" !"!#$ $&-!) 5# #-! -3 . # 0.&'&("% "&-' !"!-,"(&)%0'#' !'#")!"2)#'!!-'+" 6%)%" '#-*4*'#"*(,! "0*-$)'$$%##%,'&-%(! &)*""#+" "" "%20 '#&"*%!! !*&!#(%!)%" )!#'!%6&$- '(.!/#"*(,! "%'*(*&0"#6"%%)&1 '!(!% !*#&$ 3 +*$#*)$'/" $! (2! $%& !)#$. / !0&(%+"""! (! + !"# 22'%. 3!"#$ %" !"# / &(#&$%)!#'! &'()%!" '$"#+! !"!#&'()% ' '$" ""# ""# / &(&+2%'!(! + *"# "'"/01!% "' (%!)%" *)%!!*!+ ("%&-)'!"3%" +! )*"-"*1*! !*&!#. "% " 1%#"%2#1 ""* (% "$&(!%,&$-.!* 1 *" * + .&'&.).#1 *" /!(&'."/!"/&' *&$&+(! %.#'&.%1#*! !$ !" !"!#$%)!#.! + ("-$4#. %4"" .&00!"#$! (( " /#+-!&$, !" !"!#*)+&% )$" #) !"#%"2!- +)3!&$ -+"&-' $%&$'%#0"% /!#(3%*1/!#(5#1! .82"9":;"<":"=!2!-">?@"<"A%")')'!"/0*/!#"<"B& C"D",C"EFEF".&(0 !'#/ +"'$5#/ %5"" !" 1"#"&%'(" ) !2!'" &%! (& ! *"(+ /01!%"*&,- '(.!/# )4&- !% " 0 $%".*" " 1%#"%2 ,&$-.!*%'!(! "%).+#+! ' " /#% "" '(.!/#' 3 1!**!$ 1" !$13 &'""%"("#% " 3)%/#"#) ! 4 ,"#, ( "%20 '#&"*%!! !*&! " &*!%*&$&+(! "%).+#" / ( '0"%(#+! ' "'(#("%$")" 3&%(!"","+!' 1&"!#"! !% $%&$'%#)*"-"*1*! !*&! $0.'*- --&11&) !*&!" 5&$#$1&#) 1 &+&+ / !0&(% !5! !$."* &(#)&#$%/#$, 4! !*3%1! /3$ /#%"'!!%"#$)'!/ '!, 4&%+ *)$'/"#" 0 " 0#(! ) )!#'!%'!(! 0-' !0#"%).+ " / +! ' 2"."% ! %"+"%1" !" 1" "&%'(" ) &)*') 3%"0*"#1'"" "+!"/!'/! / &(&+2%"!"$. "$-'-.&$#*%-1"! 1 !!#("%""" -)'" "'0!- (& )( *&&+ !&"!,#- "&'*! / !0&( '!(! ."0&$ &0 "/+ '(&-' " /#%"")%' #!--"/#&%4 +"'4!",!/# " !%)(""()2)-","% &"*%!! !*&!#(%!)%"#*)%!!*!+ )*"-"*1*! !*&!#'(.!/ &$.!"!$.!$1!%"# (3 "$&(!%+(#(! 5&'! +(# 2"0"% ! *-5"%'."'" $!&!0#$ %" *-"(+ )%%&% 1&1"(0-#0%! *!/#"1)(#0%! "-+ +"%$"'" '*(!#(%!)%"#*)%!!*!+ /! &+%4!2$+!$ #"+!"1-"(! ()2)-","% ")'"($"%"(#!% . !,"&&1#(%!)%"#*)%!!*!+ )!#'!%1#*"#&-$ "&%'(" )#&)*') '"1#,-%4 4 !!#+"%""" 0"%#. !,"&&1'&-5%"#"&&1 / &(#&$%'!(!%)!#'! / &(#&$%!0&( (&'*!"#+ ,"% #&&" 0($ !*!! %-#$!*)"') /*..!"#$! ++ !" !"#$%&$'%#0"% '(.!/#)*"-"*1*! !*&! !&!! %,#$!&'"-' + ,#.-%" %&*!" '-1 (& )("*&&+*-"(+"1"'#$!% "%&/(#(! ) "-,&&' (&4%"!1%!! .0*') 0"% . !,"&&1 (%!)%" " / &'$%" (" /0)().!"/50". !" !"!#$%"$&!$ ,&'.!%",& "' (&3!*$#'("*') ("1#")'"" !%)(" )!#'! +"""! (! + "$&(! ,&$-.!* '!(! 3 (&$#&'$-." $ %"# $" +"%$"'"#+"%$"'" 3"%'&"#( '0"%( ( ' 2% '$&($- " / $%"0&" !" !"!#(! ) *)""%"+"#% " /0)().!"/60". 3"%)*"#+ )."%+ $"%%4"1&##!% '%!#!"!" !"#$%&$'% !"#$%&$'% " &*!%*&$&+(! %.#'&. "0$"%1%#-&0"! $"&!%-!#'(.!/#$%&$'% / &(&+2%"!!")! .#'&. 1#*! !$ " 0#%""&%) +'$* !#+"$&%*"- " !%)(""()2)- *!*/! %+(#(! (73'8'9:';'<'=)3)&'>7?@';'A,%!-!-1'*."*)+';'B" C'D'$C'E7ED'("!. )-+* !"$ !"$ /#"% #%!+ "*(,! ' 3" !" !"#$%&'! $%&$'%#0"%#. !,"&&1 )!#'!%!" !"!#$%"$&!$ "+!"/!'/! *!/#"1)(#0%! #!-- /#&%4 3&"5$' -%#,&0"% &"*%!! !*&!#(%!)%" *&$&+(! %.#'&. *+,%-1),'+2!-1* ""$ ""$ . !,"&&1 (%!)%" !" !"!#$ "# )&#*!$( )#/!" %"# (3 #$(&%&**&- #$( *.!!!() *6.%( +2!,('*)%*"- +)&&'*+",4 &.#)'!%1) %!-'!2%,%!+), !" !"!#$ '3 %"/ # ./*)*+"% "*') '*(!#(%!)%" / &(&+2%!0&( ("!+",'*+,%-1) 7&' /'&, ,"-+".!". *&5"&33)(! '(.!/ $%&$'% #&.."2"'.!%,!%33 " 0 % "" -*4*' "*(,! !"$ /#%!" /!%2)(! !"$ %!"-"-)3! (&4%"!1%!!#.0*') / &(&+2%!1!$!*3 $%,-),'","* /#"%"#%!+ 1&1"(0-#0%! ,"%'!%"*%&/ #&& 0($ '%!#!" !" $"&!%-!#'(.!/ !*#&$%1#*"#&-$ '!(!%)!#'! 0!,*+ *)%*"- ( -#" 0 "%2#)*"-"*1*! !*&! .') !.#"%&/( !,"-' %- */!(),' %- -)$'4",# !!*#'(.!/ "-,&&'#(&4%"!1%!! '!(! )!#'! !" !"!#$ !'4#*)$'/" *)+&% )$"#(! ) )"4#-'%5 !!*#'(.!/#' 3 "%2 1 ""* (% 1!$+! '#(&'! "# (3 1!$( #)%/#$, !&!)*"-"*1*! !*&! ' 3" / &(#&$ '!(! *"# "'("1#")'"" !%)(""()2)- /01!% "' #&.."2"'.!%,!%33 /01!%"*&,- 0($ &%! /!50"-""//"0-# )%&'" "' .74"8"9:";"<"=!4!,">7?@";"A%"+'+'!")5/)!#";"B& C"DE"*C"FF".&35 !'#) /70#8#9:#;#<#=!0!'#>7?@#;#A")-*-*6#!"$!!,#;#B%..C#77#&C#7D7#/%3".!*,! $%&#'()%( 2).!&"!" ! !2!"%"4!' )!")&'"!" ! &!!,#()*+!&!!, %#)'*)-*) *)!"-('*)%*"- ""5'"00!!) /1).!%" "' /1).!%" "' $+!$")1"%%&* 0",!) %$%#)- *+%,'+,)# 1% )'+2,"-) /()!'(! 3)(#)&' ")'"($"%"(#!%3)%/#"/!"/&' 4).#3%"/!* ')/+#6)6) 0($" &%! "%#+'"/&&- )&'! +3! 0+%! ! -!%/!#$"-*6# !"1%".)*3! ."!-3#0-/!% *&6 &33)(! /#"%","% +"" .!2 '!, 4&%+ ."4+."/!'+&00 3%'%"# "" '! /#"%","% ,&'.!% ,& "' ,"%'!%"*%&/ "%2"")#"%- *+'."*+'#!% !" !! """# !2!' &%! /&&-")$&* 3%"*,"+#."!-3!" !2"11'!!"#$%&' )'%*6 '-1!# )$'% ")*+'/!!+.,) ()2)-","% ",!6""/".")"'#&) 0"3!,!))" !' /!(&'."/!"/&' %!"-"-)3! *!&#)'$". % ),!!%'!!3!& +"(( 3%-(),$),00 7&'"/'&, '!.")#"%- +! )'+,%3)& ""%% " !%)(""()2)- 2).!&""%% !" ! *$+#!"*",-!%) ",%-1)'-)$ *)%*"- )/!*"() -)##-! *)#! /%"'")#"%- %(' %5 )"')"")#"%- 2- #2% ,)+'%"#!&-1, "&%!# /%-+2), )"'.")'"-! ."4+. /!'+&00 *!+)) ()*+!&!!,#1!$"")"+ &!++&)'"!+) 4!"%"&-. #$%!!"!2!"%"4!' ."!-3)'#,2!),!" 6"!+# %" %* ",%-1)'-)$'"&%!#*+%,'$%,!%/+%!-'% ),!!%'!!3!& /"/'!.&+.,) ",!6 "/". ,"-+".!". +6-*7*89*:*8*;$-$"*<=>*:*?," & &'*(.2($#*:*@/!!A*B6*1A*C6D*+/!.!$&#( /&&-"%!".!%) /!'+&00"*!+)) $"%%4"1&##!% /"##,!"/")#"%.) !)- &%!"!)%-/ /!"/&'"3)'"-! 374#8#9:#;#<#="4"!#>?@#;#A% ('('+#)&-)""#;#<0**B#C#.B#7DEF#30$&*"'") '-*#.)*"!'#.-")*/) !%/+%!-'% ),!!% ("%&-)'!"3%" +! !",2/& $! (( /&( ,".2")#&'!$!"%# ,%/)+#! -!%/!#$"-*6 '),!#'),!#!2%& 2)'1,-.!#!2%& -)$ "&%!# /)"/&) *! /*"- "&%!# !,,", / " 3, !$ /" %)# +0 -+'! ,"'. %!. *!. ,%*+#)&)"/ ,%/)+ ! -!%/! 1,&#",+'! )$&*%5''!%)"."4+."/!'+&00 2%,,4 /"++), !!3!& $%, +%#)'/&%!) ""(#*0$", ."2)..)/#)'- /)0-/#$%&-! !%,"&!-)'0,% #) .!2 ",) *!!)-!)'0!!+!"- ""%$ ("1#")' " !%)("""%$ !"$"& $&$,'0 '"(# -0%"(0' !0.#('$0*" '+!$#"*"#3$ *!&#+%"( ) '!#."!-3 #) !"#%"2!- ! & !.!*1"'$ !!&%!! "%#+' )%&'" "' 3&%(!"","+!' Health & Environment #&.."4"'.!%*!%00 3)" %%'$"-*6 ()")%(! !"1%".)*3! !,#!%"*/ "%+ %-'*./), %- )&,%"*" $0&%" #$%&'!")!")&' "" '!#")-* 3!()"'1% ) 3/1$,*3""&#',$"#*2",, $,*,$$) &.!"$",*3/1$, 1 &+*(/"", %" )"2"/&,#&' '),!#'),! 1!& ",)'1!,&* %.,, !"&$*!"##%$1 '".#/0%1 % ),!!%'!!3!&'$%, 0!,*+'+! ) $"-(),'$" %- ',$$&%/.($*'"( , )" /" %) *&$! *0%" )!4!' -+'!.& ) % #!,)" 20))(!#2&"! $!(!3 $ %" "/".")"'#&) !" !"#$%&'!")!")&' ()*+!#&!!,()*+!#&!!,#()*+!&!!, " ,*3/"".# /& 2).-',%*#3)!, +6-%7%89%:%;%<$-$"%=6>?%:%@," ' '(%).2)$#%:%A/!!B%C6%1B%DEF%+/!.!$'#) &"#.,""*'"( 39+$:$;<$=$;$>&+&!$?@A$=$B,'"/"/.$-" -&($=$C122D$EE$0D$F9G$31#"2&/(- #$%!!"!2! (&'")'&* +%&'"#$%&'! /"-*(#2-!,%"+ (!*/"-3( ').)" , '%#*&/1 '.!"$",%3/1$, !/,"" ,$$* ($")*+, -$*!",( &"# /&""*3",5 )+6#$")!")&' ."'3! .%"!&') /"3-")!")&' )'$".#+!)" "")#%0$", 0$",) "(/ 1!"1"3$&!"3&2"# ,! '#, %"'"!003 220%3 -!"#$ %" )!")&'"!" !"#$%&'! /&&-")#%"'!!% % ."!-3 '-*#.)*"!' */)) "%*.$" #&/(&, 3"-&'-& .& #$*(#"#$ $ %"# $" &!!,#()*+!&!!,#1!$"")"+ 1!"1"3$!'"/*"!!&, !",2/& 3, !$ *0%"#!(1"!/ %(+!"#'0. (-.#()"/)!2-)* ).$!"#"-!! ""#% ""#% )/(( "*).$" 3%(4"#$ %) 220%3 -!"#$ %"# $" ,!!,#3)"/ ,$'$1"2"$ $'$,(0 .&%'!"1,&# "33"$%1"#!& $"$!#, !%!",) 3,".$1+&,31-& $"$!#, !*!",( !" !"#$%&'! /%!!* ")*6! .)*"!'#.-")*/) '$1 0/,5 )"*'$+","- 3/1$, 3""& $')3(#&%.!* 0', 3,". +3!"0+%! )&'! +3! %!. *& "' '),! !2%& , (&#%'/1 ("/2""%1",! !.!&%!/,$ #&/(&,$3"-&'-&$#1/(,1! '"/2"" 1",! )$&*%5''!%)"."4+. !1+)&.!"!" ! 3)'0-*#2)""-! )'!5)*/!"#2).-',%* )$"*%+, -$%!",) !"#% 1!"1"3 !'"/*"!!&, %&,1"/ ("&*),"&! (!/ !"#% !/""%3""'# ',$"#*2",, $, "33"$*1"#!% !"$"& 3/1$, $ %-0'#" 6 !7+"8"9:";"<"="+"!">7?@";"A* %'%'3"&)0&"-";"B$11C"DE",C"FE"!$.)1"'-& 39+$:$;<$=$>$?&+&!$@9AB$=$C,'"/"/0$-" -&($=$D.22E$FF$1E$GHI$3.#"2&/(- !"$"'%$'$,(0 2&3"#'!$2',"5"'/' !.!%*!/,$ 2",, $, ,$$) !" !"!#$ %" *.,#&$,%,$"+ / " (") .%-5", -"* #"*."'-".2%!!*"' 1+&,31-& 3&'(%- 1%.2%3 '"3$+ %&'!(%$#',& 2""!5 &/"$ 1 '+%)/"", !",2/'%$! )) /') 3"-&'-&$#1/(,1!$!,&+&/("1/ 4,""($+&.&(' !&- / " '"( !" !"!#(')&% '$" !0'+#""%* !" ! 0!$$! %&',($3"-&'-& ",(%$3&4&#( !", !/!3"'0 '&3&-$'&.7!(" +!0- !#. %* ) '#2% '$()$0 %"'". -!"#"'"%+/ 2 ##"'"/!%'- ($")*+, -$ '&3&- '&07!(" *"1$+""!" ! /!%'-","1"*3"'.5 1 ' 3"*-"* ,//*#/3%)/"", '"#.,"" (") )/"",%3"'$" (/"",*3"&$" ( " !/&*-"""$0 !/""*3""&# "!"+ -""0!$$! !" ! , -"* 1&"0%( !.-- %&'!(%$.44"#"'!- $"$!#, !%-$& !"$) 21,&$!"*&!7 ",(% #1/(,1! ,//)#/3 (/"", 2.,&$!&.!!& / " 3, !$ ""# ""# 2""!5 %/"$ -("37$!" !"-% 2',"5"'/'$!&.'!")'("1/ %&',( 3"-&'-& #&/(&,$3"-&'-&$#./(,.! 3/1$,%3""' 3,".$' "-& ,!" !"0!$$!&-*" -"* 1 .*%&%& ".(!$'#%,$"" #0 !&',/$2.,& 3,"0$"-& 3/1$,%3""'# *%.4"&'5!"* (,$"# 2",, $, ,$$* &'(""#)" "" 3,"0$!.!"#7 !"$"'%3/1$,%3""' !"$"' 3/1$, !" !"#$ !%"&'$ /!%'-"1%.2%3 ' ",(%$#./(,.! " ,%3/"".# /' $!" '#"'"%+/ 3$+"*%.4 #"$#!" /!%'- !"-*$%-"#)0!%. !/""%! '$ .*%&%&"/!%'-"1%.2%3 ' #./(,.!$!,&+&/("./ !" !"#$%&'!(% &$1 0/,5 ).,#%$,*,$"+ -.3'$('6 #!"2'(&$#%'/0& %&,."/$&!"3&2"# +, -$%!",) !1/.$(&,2 !",2/&*#"4 ,0."%#,! '" !'"/*"!!&, %&,."/ &!"3&2"# !",2/' #"4 (,$"#%2",, $, -("37$4"/3 -(%"!#$0'"%0! (,$$'&/.)$%(") ',"- #"# /&""*1"-$ 0- * # 3*('()"% "(0' .!"."3 &!"3&2"# !",2/' + /4 +$ !",2/'%3, !$ )"*'$-!,&'3 !/,""*,$$) ".'!$&#*,$"" #0 !"$"&*3/1$,*3""& ('""%")" % " !""'+%*$'1"'- !!"6) !%-5 "(#$&)*"!"# *-1"'- "'+%*$'1"'- ! 6) !%-5 !/""%!/!3"'0 ",(%$3&4&#( /!%'- .*%&%& /!%'- *"&%!"'-& .!"."3$!'"/*"!!&,$%&,."/ !,&0/'/($1.2&/ ,&-&',#%&,-$4"/3 %&,1"/ &!"3&2"# ,%0#$!0($" %&'!(% 144"#"'!- !&'3$!&+&! ! )"#/" % ."(+!"#!0)) *%&4"!" ! /!%'-", -"*".*%&%& !"4& &6!&#('/#7 /&1$7.,* !" !"#$%!&"'($ ,$&$1"2"$*$&$,'0 "1"*3"'.5"1 ' 3"*& 3$+ *%.4 &'5!"* 2',"5"'/' !&0'!")'("./ )/"",%3/1$, 2",, $, ,$$* &.,, !"'$%!"##&$1 ""## !"4&$&6!&#('/#7""## !"# !"# ' "&% ! + ) "4"'#*0%" .2%!!*"'"-"&- !" !"&"*+%.""!%'" !./0 (&,2 !,&./'/($012&/ 3,".$"-& "/&'*#$,! !", !/!3"&0 30' !3#"%&*, )"!2 3%(4" .+&,3.-&$3&'(%- "/'(%#$,! 21-8""(1$ &', " 6 $%"3(" 2.-8""(.$ &', 1!"1"3$!'"/*"!!&,- )(!($0' 4 !!"/ *0%"#,"0,!" "/"(&$-('(& 1', 3,"0 !'"/*"!!&,$%&,1"/ +, -$ !",( (/"",*3/1$, )$"*%+, -$ +, -$ !", 4!"/($0'(&, 1%.2%3!" '"3$+"*%.4 !"#$%&$' !",2/&*+ /4 +$ $"$!#, ! -$% !"$( , -"*"&$)*." 2&3"#'!$2',"5"'/' (,"- #"# /'"" 1"-$ .!"."3$!'"/*"!!&, !'"/*"!!&,$%&,."/ .' #$ )#"#$ !"##"/"(&$-('(& !"## !" ! #%# )"*'$-!,&'3 %&'!(%$#',& 3,".$!1!"#7 #1/(,1!$!,&+&/("1/ !"+"!"!" ! !"-*$%- #)0!%. &.2$$! 2.,& !"*&!7 $!(* ""#$! '+" !*%'4", -"* )"*'$+","- .!"."3$!'"/*"!!&,- 5(1 #4(%&) 2".#/" %) ,%"+' '"#.0*"' *0)"#,"0,!" 3,"0$.+&,3.-& -("37$4"/3 -13'$('6 !&'3 !1"-1/ ('$0*"#" 6 0!$$!"!"!"# *-1"'- "'+%*$'1"'- ! ! -(%"!#3"2"$" ,%0#!(2" , -"*"/!%'- -"0',7$3,"/* #&/(&,$3"-&'-& 0'(&, #,"-"- !" !"# %'- .*%&%&"/!%'- /&0$71,* /!%'-"*%+"* !&&,$,&+"&0 !" !"&"*+%." ""#& ""#& 3,"0$1', !" ! .*%&%& -".',7$3,"/* 3,".$0', !" !".$'- 1%' -%$' -("37 !" !"-% 0&".%($!1-- !" !"!"+"! !" !"#$%&'!(%2"#% 2.,& ,&-!($ !" !"! 4,""($+&0&(' !&- 3"-&'-&$#./(,.! 3,"0$' "-& !&'3$&6!1-",& 0!$$!"!" !"!"+"! !&&,$,&+"&1 3"-&'-& #1/(,1! )('+!"#, /"% 2"3"% ! +04"%'*"'" 3"-&'-&$#./(,.!$!,&+&/("./ 7&', .!3 !'"/*"!!&,$%&,1"/$&!"3&2"# #)0!%."&.2$$! *%+"*", -"* &"*+%.""!%'" ""#* ,&-&',#%&,-$4"/3 ""#*

Fig. 3. Multilevel Sankey diagram and wordclouds of the MS partitions of Gd2v at the three resolution levels (fine, medium, coarse). around ‘planned parenthood’. This topic is a good example of the advantage of having intrinsically quasi-hierarchical partitions instead of forced hierarchies as in Ward’s hierarchical clustering. Indeed, in a strictly hierarchical clustering, we would have either lost M6 completely engulfed by another topic, or we would have obtained less coherent topic cluster(s) on other levels. The flexibility of quasi-hierarchical structures, however, allows us to extract topics independently to the other resolutions. Figure 3 also shows that the top community in the M level (M7) is largely conformed by‘Politics & Elections’ with small involvement of ‘Entertainment’. Since this topic is related to the election campaigns, the involvement of ‘Enter- tainment’ reflects the role of media in campaigns through e.g., political speeches, interviews and debates in TV shows as well as allegations of‘fake news’ by one of the candidates who targeted the media industry (a small finer community F25). Although we only describe a few examples of interesting relationships across the three levels of resolution, all communities have distinct and high quality topics as reflected in their wordclouds. To illustrate the importance of the choice of features for text embedding, in Figure 4 we examine the MS topic clusters obtained from Gtfidf, i.e., from BoW- based features. Four of the five communities at the coarsest level of MS with tfidf are consistent with the four coarse level topics obtained by MS with d2v. The additional topic is ‘Foreign Policy’. Interestingly, a very similar cluster appears as a distinct grouping in the M level in the d2v partitions (M5 in the Figure 3) 10 Altuncu et al.

MS partitioning with geometric graph from tfidf features Fine Level Medium Level Coarse Level

-/$!/*7*89*:*;*<"+"!*=>>?*:*@) $&$&3*%(0%"-*:*A#11B*CD*,B*EC*!#.(1"&-% (5"*5$8$9:$;$<$=&7&!$>??@$;$A+'"-"-/$1" 1&($;$B0))C$DD$.C$EFF$*0#")&-(1 '0".0-9-:;-<-=->/"/,-?@@A-<-B#!"+"+2-%$&%/'-<-C*44D-@E-3D-F@F-.*)$4/+'% !"& #0+.0"7"89":"8";!4!,"<==>":"?%"+'+'!")6/)!#":"@& A"B"*A"CD".&36 !'#) '4"34)9):;)<):)=%.%!)>??@)<)A(&"+"+0)/" /%')<)B*22C)??)1C)DED)3*#"2%+'/ !"& '%!/%$8$9:$;$<$=#-# $>?@$;$A&(!)!).$*3+*#'$;$<"11B$C$7B$DEF$/""31#)'* !"-)#$-*"(0!$. 1('(& !&/'!"3& "+0/)'-!" ! )/+'/#-."%/!%/-)*+'#*, 2$",,!"+-&!##8 #%+0-"#$!'"- *!"*"3)!&"+-"!!%()$%(*"+ '&()*.#)/#& !#"! # !01&$.&"/%( )0+&$!"4&!6 ""#$%-%1#/!. !" ! #$%&'! )!")&' 3("0)1&( *+"/$.'+ ",!5""/". ()'!$ .+'4 (7 #)%"&"#1#)' 0!##!*!" !*!"+"! 10*'$('2 !/./%-!/251'" !" !-*$'&#/! ""& 1,&#",+'! ""& #*+'(*!)!(%.%+'"*+ *(1#$*#6$1(&&!(.# "!"+ -"*0!##! 3#+ )$.4 %&5!") "+0/)'"*$%-."%/!%/% 1$&,")-(/!,'(-/4/#2/+)5 )$&*%6''!%) ."4+. /!$&-*)$+") 07&+*01&$*&'(%1 5+""($7&/&(' !&1 )!%/-!" ! "+"'%)/'&'% 75!'#$!#"! # -&.$1("*6 *!"*"3)!&"+-"!!%( /'"37)4"+3 *!"*"3)!&"+-"!!%(/ /&&-"%!".!%) /"##,!"/")#"%.) /'"37)!" !"/$ !&0!!&$&'( (/!,'( *00")"!,% "!-! $&!.5' 0!"0"* !'"-4"!!&+ !/251'"-4*%6$"'*/% 2/' 1#/2+!+' ""#$%-)"#)$,!'/ *.%(3*/%)3%&'$/ !" !-)!$%/ 4*%6$"'* &"'/

!" ! "'"#%()" 3("0)& "/% + (",$ !-# Societal Issues .)$%$%*/!$&- )$+")*, -") "1")3"&.5*1 & 3") 0!"0"*$!'"-4"!!&+$%&+0"- -+'!","'. '!.")#"%- '+!$#"*"#3$ $%(*"+)%!"3%2"# "&!1!)( $23*'!"#$*0*'#1 !" !"#$,! !"" )-1"&- "&+$)#&1"&- ! %&+0"-$&!"*&)"# ,"-&)."("/ .2$!!)"&*-"%- "&+$)#&1"&- !*6( !$-5 &("!( $/!*!(&!'!#* ()'!$ .+'4$ (7 .)$%$%*/!$&-*1$.2$3 & '**$1"/'+ ""#' ",!5""/".")"'#&) ""#' "/".")"'#&) #!"2&'% #$&+0% !" ! %")+$." !"5& &2!&#('-#6 !&/'!"3& )'+","'-' $%&!'$)#&(% )#7$0"&, /!$&-*, -") #&-(&+$*"1&'1&$#0-(+0! F19 - mix of `Rio(/!,'(-*#2!+"!!'"*+!" ! ""#$% #%+'%( 3"/%&/%!"% !&"+-"!!%()$%(*"+ !"% %")+$."*!$&" 3*#,.-(/!,'( !"##"%2"")#"%- *+'."*+'#!% (%/%&(#$%(/)4"+3 !"## )$.4*%&5!") 1$/#'*-#")* 3("0)*.%(3*/% !" !*%")+$."*!$&" , -")*/!$&- !&&+$+&7"&. 1"/'+6$*+"-4 )$&*%6''!%)"."4+."/!'+&00 !-#$1(''#& *"1&'1&$#0-(+0!$!+&7&-("0- 4*%6$"'*-&/!# )&'!"+3! !*+0)'%(2 1"&#$ !,# 0 !" !*.)$%$% *+"/$!0!"#6 "-"(& 1('(& 1,!+-1!#/+'(**. )/+'/#-."%/!%/ /&&-")#%"'!!% !(%0+&+' 1*2%+ &("!( $+!(* Olympics' and%/7$!, '#!+%4"%%"*+ $%&!'$)*(0&+",&'"*+ + (",$ !-#$1(''#& 2 """&*/!$&- , -")*.)$%$% )&*"#'!$)'+","'-' ""%' ."4+.""%' /!'+&00 )!")&'"!" ! 2&("5"&+& !%0&!",&'"*+ 3*#$%"&"# +&1&'+#%&+1 5"-* 500*$#0)!'-6 )!4!'"-+'!.& ) *!"*"3)%!"3%2"# !" !*"$" /!$&-*1$.2$3 & 1#/2+!+'-3*4/+ *#6$1(&&!(.# .'+$*+"/ ,!'"+-!4/#")! 1$.2$3 & 3#+ !" !*" $&- "")."&-*.2$!!)"& $+!$")1"%%&* .&"/%($!011`Panama Papers5/,,*3-0/"/# '1/*1,/ "+0/)' ."'3!".%"!&') 1&()3("0 "('$)#*+'(*! '&()*$!#"! # !0-/$(&+) !" ! %1#/!. ."%/!%/-)*+'#*,-1#/"/+'"*+ $+"'/-%'!'/ 2$"+/! 3*#4 %!."*& "' !" !"#$*5""' 3*#$+('5&""1 %(*3-%541'*4% .&%'!"1,&# 0!"0"*$&!"*&)"# (/!,'(-/4/#2/+)5 1$&")-(!"# *+"/ 07&+*01& #!")'(&$#%'-/& !/./% !,&*1")'$% ."4+."/!'+&00"*!+)) &%3%/)&%07!'" *#63( $"&!#)'('!") (%&!"()$(1#&!"()* !" ! "#$%#& 1("*6$!" !"1% %&'+($*"1&'1& 1*(!3 $%&!'$ tax scandal )&'!"+3!"0+%! "('$)3%4%#' *5""'$,! !)$&4 , -") !"+"!*!" ! ""#& ""#&!!&%!! "%#+' 3"/%&/%)#*+'(*!)!(%.%+'"*+ 3("0)"/% /!$&- )"%$!"&-% "1")3"&.5*1 & 3")% )'+","'-'$!&/'!"3'("0- &"#'( )*+'#*, +(",.&"3)/$"5#", /!$&-*.)$%$% #$%!! !2! "'"#%()"*)$%4 !'"-4"!!&+$%&+0"- %&'!(% #'+& +%&'"#$%&'! + (",$!#"! # 0#(&$" / 1("*6 5"-* #&-(&+$*"1&'1& /!'+&00"*!+)) 4(""').%0%'&2*(% !%/ !"-%!7 '&()*$!#"! #$3*# !" !*!"+"! )"#%$)0+& &"#'(-./0/)' )!")&' !" ! #$%&'! /*3&)'&6 $%&(')3"/%&/% 1$.2$3 &*3#+*)$.4 ""#"!" /!$&- !" ! 0!##! !" !*, -") %/7$!,,5-'#!+%4"' ."%/!%/-)*+'#*, ,".2")#&'!$!"%# .3)$"")'&" %1#/!. ""#$% /"0&(7)3("+- .2$!!)"&*!" ! 4*%6$"'*-)*+'#*, *#63( (**(3 ' 3#+*)$.4 !" !"#$%&'!(% '//+-&"#'(#!'/ !%%( (%."%1 2*(%)!%*!!% !#"! # 3*# +('5&""1 !#"! #$3*# 5&*&+'! /07&+-)&-( #0-(+0!$!+&7&-("0- '#!+%4"'-""#$% )$%4*!" ! *+"/$"1& -&.$60+4 )+5#$ )!")&' (&'")'&* 1%"0$' !*// 0!##! !" ! *+"-4$'/& 2/+/-.#""/ 1$&,")-(/!,'( )!")&'"0+'",! 2*/8""'* %&( $%&!'$)*44"#"&!/ *"1&'1& #0-(+0! !" !-)!%/ ""#+ )"'.")'"-! ""#+ .$-5*, -") *+"/$' "1& 0!"0"*$!'"-4"!!&+1 )*+'#*, 1#/"/+'"*+ .+'4$ (7 /!$&- , -") .)$%$% &3+'$4!&"-$1%0. !'"-4"!!&+$%&+0"-$&!"*&)"# '//+-&"#'( %!."*!. #$%&'!")!")&' )"')"")#"%- #&.."4"'.!%*!%00 #$%!! !2! %"4!' 2%3"#&! 2&("5"&+& .#)/#&$!/#)'!'0 1(** !)"(&"#&('!") *3!#$""3&' "0+.0(7(89(:(;(>?(:(@$"+&+&!()6/)!"(:(A% B(C(*B(DE(.%36 !&") *7+37#8#9:#;#<#=!/!'#>??@#;#A")+(+(2#!"$!!*#;#B%--C#DE#&C#DFG#3%1"-!(*! 0%!5%$6$78$9$:$;#*# $<==>$9$?&(!,!,1$/.'/#0$9$@"++A$=$3A$BCD$5"".+#,0/ !"% !"% )('!")( $()'5#1 1(**$*5""' +#,0( $4#( 04 ' (")$(+#&!"(,/ !" !"#$/#!(&'1#)' ,! $!" !"# )!")%&(!" ! )#%*$6&&!$) ."4+. /!&+%00 4)! !"(+14#*!'' *!" #,0$"&!+# /0"!$%&!/) )"&)"()""$- #%+'%()3"/%&/%)#*+'(*! /%%-()#%* '%&()&%* %'0- +1#2)-! +"&#$1., !*#$+(00#& 5#(04$!#,( 02 !" !"#$%&'! !"%('*)("#* 3("0)!*!"#7 !" !"#$5#!(&0+#,0/ /"3- )!")&' +3!(0+$! /"3-()!")%& "$"+& /%%- !+'/!"#-!3)' &+( 2%'3 %" )"2"/&,#&' $')14#)*,'!*!! 34!0#$!#"! # ""#$ /&&- )$&* ""#$ +%1)7*(- -%/!-!(*#$')14 -./0!"# /2/0#+ 5!/!(&!0!#/$!" !"# +(00#&$+"*#+#,0 !" !"# '&.0( !02 /%"'")#"%- !" !"#$"%%!"#& *+&. *+&"!$ 0"3!,!))( !& 20-()!*+1!#*!)- +3!"0+%! !" !"# $%&!'$ .3)$-!" #)"# "%..(4"&.!$*!$00 )()-)#&%-!(#20-()!*+1! ) !" M2 - BLM & *(1# *#6 /%%- $!".!$) +$%&("#$%&! %'0- +1#1%--+**!! *$+#!"*",-!%) )"&'5$"(&"!#"! # "" "& !)( %'0- +1 2%'3 "&!+!,( $-./0!"# ,&%.",+!$# !"4% %6!%#'&+#7 1"&#$.3) '(")1&".,5$"4#") -./0!"#$5#!(&0+#,0 "%#+' /&&- 0"3!,!))" !' 3"/%&/%!&"+-"!!%()$%(*"+)%!"3%2"# #*+'(*! ,"-&)/!(%&3 $!.(*% "& !!%$!! "$"+& "+%#%'0- +1! 2#(&$" 5 ,%$.(,+!#" )#%*$6&&!$)(."4+. 6")"*!"$)14 1%'+( $')14 !% '! ")+!!#7+!* $')14#'+/! !" !"# 5#!(&0+#,0 1.,$"",0&" 0%!5%$6$78$9$7$:#*# $;<<=$9$>&(!,!,1$/.'/#0$9$?"++@$A$4@$BCC$5"".+#,0/ #* +*%6%78%9%7%:$-$"%;<<=%9%>," ' '(%)02)$#%9%?/!!@%?#:#@% &(&()#+*-+""#:#;.''A#B#2A#CDDE#/.$*'"("+ Health & Environment !" !"#$)! /4""0 )! &("!( 5!/!(&!0!#/ "&!+!,( $-./0!"#$/2/0#+ !" !"#$/3""0 */)) "%*0$" *0,#&$,%,$"+ ."&3!(.$"!%&) (%&!"(,$(+#&!"(, *!" #,0 "&!+# !"1+*0#2%3 !"#$%&' !"#* !"#* $ %"# $" &($.'"#" 6 $!. *!. )%&!(+3! "+%#5)(!+"% +"&#$ !)# 2 / "%4, !$ +*,%"'"#$.*%" 5&4 #1&%*+ &%-!(#! %"* (,$"#%2",, $, /%&1"#$ %+ *)4!#4(!! )7"+1)(#)-!"+1)(! 0&.+!$&( 2 /3""0$)! )$"*%+, -$%!",) ' (")$ !*#"%%!"#&$/4""0 +/'""+%#,0!4 *#+"!(*",-!$) "$2"()""$- )$3"'#5)$$)" )/"", 4"'$" !",2/'%$! )) /') "#$!!(!2! .%$&!(1,%" -%/!-!(*#$')14#'+/! !*"%$")! !1)(3)' 5",( 5 0&.+! ' (")$(+#&!"(,/ !*# +(00#& -+&!(,"&. /%%- )"$"&!!$ *!)-#"!) (3 #,%"&"#+#,0 &("!( 5!/!(&!0!#/ +220#$)(4 3+'-)#"%"!!!77 /""!( +#5!( )! $!" !"# ./#$%"&"# ,! (#, %"("!../ ."4+.(/!&+%00(*!+)) !" !("#$%&!()!")%& 7")(1+!1%#6")"*!"$)14 *).#,)/!(! 1.,$ (3/ +(// /3""0 '( 0!+"&#$!" !"# "/".()"&"%) 6")"*!"$)14#1%'+(#4)! !"(+14 ,#4$2"&) !" !"#$%!&"'($ '*$!#'.%" !&!! %3#$!&(".( !)(#7")(1+!1% '(")1&".,5$"3#") !"# '"# /'""%4",3 !"# ,%")( ("#2.'"( %&)!" (.2 #+!#()1"$$%* ' (") !#"! # &("!( $5!/!(&!0!#/$!" !"# !/""%4""'# (,$$'&/0)$%(") /!&+%00 *!+)) !,!'' 1%- )(0 ""% ""% +#,0( 3#( 03 !/,"" ,$$* +&!&$.(#1 !!"3 "#$!!(!2!($"4!& ",!5("/".()"&"%) )"&.()&"-! "(+*! !*)*! 4)! !"(+14# "%*!!* %'+1!#$""*)'+*0 +(//$/4""0 +(//$!,"(&"#&(0!", 1.,$"",0&" )!.- !#2 %' !" !"#$'&.0( !02 (4 #,%"&"#+#,0 !"$"'%4/1$, "44"$%1"#!& ! +"#3" % '3+-3$9$:;$<$=$>"4"/$?@@A$<$B!)+(+(.$!"2!"'$<$C&##D$@E$7D$@FG$-&%"#"('! !6*)6#7#89#:#;#<%4%'#=>>?#:#@" *!*!(#"0,"%!#:#A&11B#CD#3B#CEF#)&/01%!!" %,)(,"7"89":";"<$1$""=>>?":"@# )!)!4"/-*/$%":"A'..B"C"2B">>D"('+-.$!%/ !*#$+(00#&$+"*#+#,0 '"#0,""%(") '0!"$",%4/1$, $!&!/#$ %" +"!0#/%&1" , !."&&1#(%!)%"#*)%!!*!+ ,%. !&0" /&7"! %&"!' /")-"!$#+'%* 1)+$"!#$/)($!%) ""($* %$ -!(-$"*-#($! 2#(& " 5 + (#0% ($&+$. '.%"#,".,!" 6%)%" '#/*4*' !" !"#$%&$'%#0"% )*"/"*1*! !*&! , "% (.)%"!#)1)"$4$ %&"!'$4)%)(%5 !" !"#"$%&"!'$0"!'+%" $"&!%/!#'(,!- +##+.!)'+&($!"3&!# #'$"2 ($ ' (") !*# 4/1$,%4""'# !",2/'%#"5 *'#%)'! !#'1)($#/ 43!0#$!#"! # "()"'*&!+,"- +##+.!)('! /*4*'#"*(.! !#'"+&')+$ !" !"#$5#!(&0+#,0 )! !" !"# "/'( #$,! ("8'$ !"!+-"(' '(,!-#$%&$'% #$!#'(-+%)1$ #)4&% #-!". %$ 2",, $,%,$$* ,*-!&$#!" !"! !%)/+) -")'* "/.&&'#(&4%"!1%!! '%!#!" !" "!*! &!130 .- ' # /'&(&+"% "&.( '"-$%!", ,$% ""%)//-$ *'#%)'!"" 2/ ,$'$1"2"$%$'$,(. &2)#)$(&#+()'" 1)+$"!#$/)($!%/ 4$%" *'#%)'! " , 4/""0# /' %&"!'$0"!'+%" $%&$'%#0"%#, !."&&1 )# 5"2 # ' (")$!#"! # 2)!)%1 &2)#) +"&#$1., !"!+-"('$&2)#) 1&"!#"! !% '(,!-# )4&/ !% $0,'*/ //&11&) !*&!" +(//$!,"(&"#&(0!", " 6 $*" $%&$'%#)*"/"*1*! !*&! 6%)%" '#/*4*'#"*(.! !" !"! &(+*% ($" )('*&(5 1"(("-5 !%)" *'#%)'! +"*#+#,0$' (") +, -$ !",) , +" 3" %+ "(+'" !')'" '!"# +##+.!)'+&( %&"!'$!"/" 0"%#, !."&&1#(%!)%" , +%"+&$+0 (,"- #"# /'""%1"-$ *"3$%'$1) '&(&'*'#2 )" ( "*% !#) + $!" (#"("%)3 !"()'" #)0&!+'5 '(,!- ' 3 !!*#'(,!- 1)+$"!#$/)($!%) " &2)#)$)-#+(+!'!)'+&( # !$")!+$/% &"*%!! !*&!#(%!)%" ( +#$%&$'% 2$$0"* ! !%)" *'#%)'!"" 2/ ""#* ' (")$ !*#$+(00#&""#* "")#%.$", !",2/'%+ /5 +$ %&"!'$) ")/ "0$"%1%#/&0"! , !."&&1 (%!)%" "6) (+"'& #"!!+%1$.)!/)(- !"('(0",$&".1# /""!( $+#5!( +"*#+#,0$' (")$ !*# !"( "1"(#'.%" !" !"#" %&"!' (&#+("" #)0&!+'5 /")-"! !#$/)($!%) ""($* %$ 2""!3%&/"$ !"$"' $'$,(. 2"&)!"#!.++ &$ "%&"+ #$ !",2/'%4, !$ (,$"# 2",, $, ,$$* *(&"" +" "" &2)#) (&#+("" (%!)%"#*)%!!*!+ #$!#'(-+%)1$"&$ "%& )/"",%4/1$, 0"!'+%"$)('&(+($!%)/+) '*(!#(%!)%"#*)%!!*!+ /-!#$.$"+'-#% 0"/"% !#).1"%('"(" !" !"#"$%&"!'$4)%)(%5 '1$#%-#! #'$ "&!+!,( -./0!"# )/! ""%!$+ " %+!%"+'$%&"!' "%20 '#&"*%!! !*&! 5#(03$!#,( 02 -./0!"#$5#!(&0+#,0 "/"%'+&($5")! '*(! (%!)%" &)"" #3"+")!%'! " 6 $%"/&" ("& -.%"&.( +##+.!)'+&( &/+%5 "%'*(*&0"#6"%%)&1 $%&$'% 0"% * !" *'#%)'! ./# %"&"# " 1% "%2 (&4%"!1%!! ,0*') *'#%)'!" ++$// "),$".'1$.$!% 2'.$!"&$ "%& !" !"#$5#!(&0+#,0/ ("/2""%1",! 4/1$, 4""' !"()'"$!" "2/+%)(! )*"/"*1*! !*&!#'(,!- "%20 ' &"*%!! !*&! (%!)%" "*(.! ' 3" ,'6"!$2/ 1.,$ (4/ -./0!"#$/2/0#+ '!"# $)-#+(+!'!)'+&( -&()/-$'!"# !#'""),$ ,(0!",( (,03#+ 5!/!(&!0!#/$!" !"# !" !"# )! !"$"' 4/1$, 4""' "0(!$'#%,$"" #. #)0&!+'5$/")-"!$#+'%* %&"!'$(&#+("" &3($" .$!(.$!%%)."0 )!$ !" !"#$%&$'% 2'. !"&$ "%& #$!-*")+ !"! #%3 !" !"!#$ %"+.$& ! +"$*%&"3 &(""%"+"#% " )('&(+($!%)/+) #+'%*$#%%&(("// " 1%#"%2#1 ""* (% '&/5%"#"&&1 '3-,3$8$9:$;$9$<"5"*$=>>?$;$@!(-)-)+$!"/!"'$;$A&##B$C$4B$DEF$,&%"#")'! +1!(1"7"89":"8";)0)'"<==>":"?,&!-!-2"*.#*)+":"@" A"B"%A"=CBD"("!. )-+* )33+!#)'+4"$)%'+&( #"/)(+)$'!"# "%2#)*"/"*1*! !*&! "/.&&'#(&4%"!1%!!#,0*') !"()'"$#)0&!+'5$/")-"! !" !"! #$!%&''( ""%# ""%# #)*+ /!!+.,) '!-" &-.)' 1",!)"&%&$)- /.( !/#"%*', ()'"!)/$2")! "$/"$3"/% &" %#-.!"!$!+$ 2&'"$ 2'. ! &$ "%& '!()!$ "& *"$"!" *"'$'!()! !&"!.#/ "&'*!&"*%!! !*&!#(%!)%"#*)%!!*!+ ()'&)-)$!%(*-( ("2#3.%4 *+//)!5$%/+('&( '!"# $%)# )+.( !!*#'(,!-#' 3 3&"5$' /% .&0"% 00.%/ -!"#$ %" %&"!'$0"!'+%"! '(,!-#)*"/"*1*! !*&! .)0$"!$!+$ 6"!'-%"$()'&)-) !.2#&($.'" *'#%)'!"#)4&% !#'""),$".'1$.$!% +&()!"#, 3"% -)+/5 !*&7 "%2 1 ""* (% 2'.$! *'#%)'!/ 2&'"$"2'. ! 4&#")$1"(*'1 !&/+&!- & ),!!& %"("2 -!" "("%)3 !.()#""%' 0"%#, !."&&1 *'#%)'! * ! )&!'1$%(!&*-)( !&!)*"/"*1*! !*&! ' 3" '.%"#!&4"!3 -&%"! /"0"$" (&)*$!)5 !" !"#"$%&"!' 1)+$"!#$/)($!% *+,)& "*),0!!) %"-(), %" &- +)''"*+",4 $ %-.(#" 6 ('! "("%#-.! !"$*+/'0$*(4 "& *" "!" /('1!&&# 1!,*+ +! ) 0!()""2& ) !"$ 0"!'+%"$!%)/+) 0"!'+%"$)('&(+( !" !"(%!)%"#$%&$'%'%!#!" !"#$%&$'% %#-.! / 3 1"(*'1 %(!" !" !"#" %&"!' 6"!'-%" +""" .!3 /")-"!$#+'%*$#%%&(("// $"&!%/!#'(,!-#$%&$'% *'#%)'!"+")!)+/ *'#%)'!"#$/%#)+%)'!/ *&%7"!$!&&# ""% ""% %&"!'$6"!'-%" (!+0!/8/9:/;/??@/;/A)"+"+"&/$%4$!(/;/B-''C/DE/#C/DFG/0-.%'!"($ #* +*%6%78%9%:%;$-$"%<==>%9%?," ' '(%)02)$#%9%@/!!A%BC%1A%DEE%+/!0!$'#) $!&' ""#$! ()" +"),"!$-,")'-'. ,.2"%#,! (" "")#%.$", !",2/'%3, !$ !"*-+-&"!$3!"",&# .!- (+' ,-) /(+'! %&'$"()&( 1 '+%)/"", '!()!+"),"!$ "& *" '&+)"*3"% -&%"!#$.("%.! 0.++&!#0*"! *"# !"!-).!'!"( *!!)-!)"1!!+!"- *)!"-("*)&*"- &$)"*)-*) ."(.2/1- 3'-" $!").2 !"&+"! "11*!/.!-/(+' '"#0,""%(") *0,#&$,%,$"+ "/*-%$(%%&##&,('-&)! %-5-*$!-+1' (,"- #"# /'""%1"-$ )$"* +, -$ !",) "%&'!"( )!"*+(, $"$!#, !%!",) !"3%#,"+$ 2",, $, ,$$* !"2 #(!!-(+" !,"- &- (!.2"-*-&,/.-'1"", (/&!'-&) !-+1' *+/'0$ "& *" 2& )"+3,"-) .!- (+' .-- !"2"(*$&!-")'('-&)$+"),"! *+&,"%&,& ),!!& !!0!' & ),!!& !!0!' %&, (2+)0 1")(, !" !"#$%!&"'($ "%&'!"( )!"*+(,/&"'! " , 3/""0# /' ""( ""( 41&*" 4&#() "3(.3)8)9:);)<)=!2!-)>?@);)A$"(&(&!)/0*/!");)<% B)C),B)DEEF).%'0 !&"/ !"$"'%3/1$,%3""' / "%(") ()'-$*+/'0 (+'/.-- !&'"%'$*+/'0 "11*!/#"(.2 !",%!/!3"'. !& *-3" +"(("0&-(),%),11 6+0!-/&"'! #"!!-%7$+(!*(), 2+&2/!"0 !"$"'%$'$,(. (,$"#%2",, $,%,$$* 1- 3'-" &"'! !/"" ! '$ !" !"#$%3/" !. -)%"4",$ 2!' ",)"2!,'* !!0!'"%&, "1)+*/!--* #"**/$()!!( 0' #$%)#"#$ '!()! "& *" ()'-$*+/'0$*(4 ,)&'"'!1) "%..)2"&.!$,!$33 )!"*+(, &"'! (!.2 .-'1"", /"/ !.'+.,) ($%&) "& -(""-!)*("! "!#$ $+( !",2/'%#"4 "& *"$"!" 6"!'-%"$()'&)-)$!%(*-( .!3" ",) /%'("-) !.(" $,!0/!")-- 1*",/1- 3'-" (,$$'&/0)$%(") +, -$%!",) '".4-- /1)- , +$ &" " &!-")'('-&) +"),"!

0-""*0/()%'1 *()$ (!")'1&&, Entertainment $-.+"*/'!0+"0!&)!!$ $!1")"(+-" ,$'$1"2"$%$'$,(. !" !&,"%'-5"$!-+1' &!-")'('-&)$+"),"!$-,")'-'. "11*!/$", *"" "&)/01!$ "& )!"*/#-)*0 '"# /'""%3",5 ,//*#/3%)/"", 2""!5%&/"$ /1(.!$) "& !2!&) %$! /'(!&'!)3('"(%& , (&#%'/1 !" !!"#""$ 0'#) %$! !")-- /+12-"! *"!!&'" )(!& / " 3, !$ !"!-,")' &/(#( "!"$/('1!&&# !&'"%'$*+/'0$ "& *" *.!!!() *5.&( "11*! 2!*1 4&#()$1"(*'1 *+,&-2), +3!-2* 6+)(%"*/)!"*+(, 2!"012-"!/5". )$"*%+, -$ )/"",%3/1$, &!, 4%$+ '!0+"/.-'1"", $+*+.-"/6"**!, (,$"# 2",, $, !/""%!/!3"'. /"$"&!!$)"#(&!/ !".(/.2!. 7%&)/&%, !/,""%,$$* ("/2""%1",! '$1 ./,5 -(&) "&0!- $!"-)-(3! 3%$'!)","+!& "!#$ $(-)+!$ 2!*1/!4+ )"0+ %","2 " !" !"#"$%&"!' #&+ &- *./),!"$ &- !"$ )/"",%3"'$" !/""%3""'# ""+!)1-"'! (-%.2/4") '") /7%. !)4!)& 2 5$%)&",$ 3/1$,%3""' 3&,,4"/"++), /0('(.!)/50". "!#/,-) /(+'! (+'!/#")"!) ""$ ""$ )0)- ",) !"2"(*$&!-")'('-&) !(#"$!"2 1!,*+ *)&*"- '"*"&-2)')* $&-4) %)*+ "& *"$,-!%!-#-)('-&) !&/+&!-"& ),!!&"!!0!' *!/")1('"0$! /""$),"$'"1""(&)" !$('")'(2(- .%&"-. "$0 1 "11*!/.!- */)) "%*0$" !",2/'%+ /4 +$ %&"!'$!"*" '0!"$",%3/1$, &0,, !"'$ !"##&$1 *+&,"+,)$ 4)&, "'( "!#/,-) !" ! "!#$ ()'-$(/&!'-&) +&$)"/'&!) */!()," &- "!#$/!".!4-- !"$"'%3/1$, 3/1$,%3""'# !&$%1&-%" *./),"#"%' 4+(.-+" "!(#-) !(#" !"2 #(!!-(+" '"1""(&)" !$('" $""/4!)"")0+"- $-.+"*/"!(#-) !",2/' $! )) /') $"$!#, !%-$& !"$) !&"$4(," *+/'0$!-+1' &/(#($(,#-)-!'!('-&) %&,-),"#,"* #"6 "11!!) ("-&'("+,. / $(!#")&%, !(- %$! !($-/ &0(+0#7#89#:#;#<","'#=>>?#:#@/"( ( 6#%*-%"&#:#A.))B#>C#$B#>DE#+.3*)" &% +2!(2"7"89":";"<)3)'"=>>?":"@,&!-!-0"*.#*)+":"A" B"C"%B"DD>>"("!. )-+* "2'-2#6#78#9#:#;"4"!#<==>#9#?% '('()#&,3&""#9#@.//A#B#1A#CBC#-.$,/"("& %)2,)$6$78$9$7$:"5"#$;<<=$9$>*"2-2-4$!!1!"%$9$?'((@$A$+@$BCD$,'&!("-%! "2&-2#6#78#9#7#:"5"!#;<<=#9#>% &'&'*#()3(""#9#?.//@#<#1@#ABC#-.$)/"'"( '(2(- ,"$ '( #)" *"'#)(/" +" !'!! %+#$!'(".( /'('/,/#1 ) %*!(.$!"3 3.'./#2*/2'" )*%(3#,(+". *+,&-0),"+1!-0* *,3!'$ .*"'.( *"."$!" !"# &. !#"$"/+ %.3("'#)"+(" 22.%- 3!" $ %" $ %"# $" '(0"#2"-'. #"6""22!!) /"/"!.'+.,) #'&!$" !,,", )!##$)*'-%"# 2"-"% !#%"("%5" &!-"!1&,&!+), !!3!'"%&, ",&-0)"-)%"#'&!$ 1""$!"3 !" ! "#$%&! 3"',( 1"//(% 2"& %.* + "2(%.+" -/()" 6 2"/0./)" 3" *"'#)(/" +" 2(.!% !" ! -/(& "! %2""/ )*%(3"'#&1""&"/ !" !"!#$ %" !"# !"# '.$)"!&0) 0& )"+1,"-) 1!'""#!.,&" /'--!" $! && !" !"# (2!&'-,!&% &'("$)'*+"*, +&$) /'&!) !.1#'($./" !" !"!#&'()% '$" "%% 0'# /""$)"$!+ 4") 3./+" ",&-0) -)% +1!,("*)&*"- /"-&$ &-#"0+ '(&.' +"&4!),!/" 1(2#1.2 '"+!#6"6" *+&, %&, -%,) *%'$" %')!"#(.1 +'("- &'(" )'*+"*, (.$& !#("$)%&", /01!$ *%,- "'-*)#!""/ /.''#%&. " %"-(),"%" &- *)!"-("*)&*"- * +#/.%" *"." !" !"# "!!"!#% $.'2!&$" &'""%"(" ''.!+#$"--"/ 2.2#%&"/ -/( 6 2"/0./)" 3" -.( !-#"%,/* .*"!2,"-%2"#$,"1"%" 1%1)'0-"0$! ,%&.!$),% "& /!'%&.)/!"/%& !" +/(3!#'")"/ %1*00'" "'. 6 ".*#*"%$"(" "!!"!#%$+'("- &'$./" " 0 " !$('" '(2(- ")-"/#/(%" 4!"$)%-. #&+ &-"*./), &- " 0#%"",%( %&*!"#'.1 3 (&$#&'$./" " 0 $)" 2",!)"&%&$)- #)*+"/!!+.,) '($./"#"'(&,% ($" $./* (+ 0 '%/"* +'("- ,'-"#,$%*!(. 1!&""#!.)(" " 0 % "" '!++')"#!+) 1&,,4"/"++), +! )"+,&3)' &$) *)-*) 1 )"#) * 3 &'$#'($./" ("!+","*+,&-0) ""# ""#"&&!!" !" !"# "!!"!#% "+!)/!&/! '( #)"$"%&#!" !"$"%& 0"-/*"/!*"' $ %" $" " 0#$%"-'" !" !"##3$"!!"!#% #'&!$"/&-+1), &'!'$.(#4 !!"+ 2"-"% ! %"&"%4" ,('(&,%"-#% "" '% (')*%%+ ,"$&!$)*$%/ 2!,*+ *)&*"- 12##$&'!13 *"." &!#%!*" /&'&/)/#1 *" /&--!"#$! (( " !$('" '(2(- ,"$ -%/ "&!!-!/ & (#2% ($'&$. " 0 $%"-&" " 0 %"")%' *.!!!()"*5.&( 3!()""0& ) %&,-),"#,"* *''($%"#0 3""*$'#, "##"4"%2'-! %*!(. !" -#!"5"! !" -#+.&(.' &"!'./#%$(0& .!1" ",) 3"%('"#& (-"%& '(""%"&"#%" 0 % """ #"$$4)1%""!$ 2*/2'"#/"( '"&"#'"&" &".$5 / %5"" ("1#+.%5 "$#"'-*) %*2"/#-.$' "22'" )*%(3 "%,/*#*! ( " 0#+! ' *%6)%33('! !" !"#$"%&#!" !"$"%& $!'!-#$ %" * +%.!!#" 0 ,"-+).!". "'"5" +"/ 1")('&. -.' !-#"%)/+ .3 / # -/'('&"% "'.( !" !"#$"!!"!#% (&!&$.' 5 !!", !" !!"#""$ /"-'$ '-#"0* (&'.( 22.%- 3!"#$ %" 1*'&0 %!*-"* !" !"# 52'#"-&" "%)/+#+! ' "!--)/"%$4 +"(("3&-(),%),22 * + " 0 !" !"##3$/"*"!! 22.%- 3!"#$ %"# $" */!()," &- +)''"*+",4 &'()!"#* +"% #'&0"*$*''($%"#0 + ,#/.%" 3($/")/!"/%& 3($/")"( ! !"$"%&#!" !"$"%& !"#$"%& !,"-" &- 1 !!#&"%""" "%,/*#" 0 !" !"!#*! ( !"3$!"3 %*!(.$&"(."24- 1")('&. #3"%& !()#!"/+"%1(" "$.(./'$#)%.1"! !"3$%*!(. !" !"!#+! ' "$#!./! 3.* &/!#)*%(3 &"%& 3"/+ #""0$"!,2' !" !"!#$ %" !" !"! '(&,% ($" !"3$"2# .3 / # -/&'&("% "&.' + ,%.!!#" 0 3"%'&"#( '-"%( /)"/')"*!%&'$ /*"- ()&( +'("-$"&&!!" +)3!&$#.+"&.' & ),!!&"!!3!'"%&, & (-"%&#*! ( '(&,% ($"#*! ( &.$' !#&"$,%'"+ ! 3.% / %5"" ""+ 12##3$1!!/ ""+ +'("-$&'(" +",(+#-.$(" " 0#*! ( 25"-0"$%*!(. '"1#,.%4 #4)!4+7+89+:+;+<-6-"+=>?+:+@$!) ) 0+/%./-#+:+;"&&A+B+1A+CCDD+!",%&- #/ 2.2 )*%(3 !&/+&!- & ),!!& !"+ &'""%"("#% " !"+ 6/"!#2.*2. ,('"" &" "" "&&!!"$!" !"# 2"-"% !#*.5"%'/"'" !.'*#""%/ 3"/2..' !"/".!"!" !"$"%�"-/*"/! 0!' ",)"0!,'* *+&,"+,)$ " 0#$," #'&0"*$*''( 1 *"#* + /! !-$/ /%''"$#-$/ '"&"#%1.$ '"&" '"&" %1.$ & ),!!&"!!3!' ,)&'"'!2) !&,"'!-) 2,& $) 22.%- 3!"#$ %"# $" !"&'-,$.*"!2,"-%2"#

&!$,"+$%.)" Politics & Elections 52&%2(!$!" !"# %-/&-$6$78$9$:$;"0"#$<==>$9$?)"/(/(3$!!5!"%$9$@'**A$B$.A$CCD$&',!*"(%! &/.'/#6#78#9#:#;!$!!#<==>#9#? ".(.(-#1)41! #@%**A#=B#2A#C=B#'%,)*!(&1 $6*!6#8#9:#;#<#=-5-"#>??@#;#A%!* * 3#/&0/-$#;#B"''C#D#.C#EDF#!",&'- $/ -"+$3'*0 3'*0 %2(" (&'*!"#+ ,"% #$%&'+,!&'!)0 ,") #" 1) $%&'(#,!'(!*3 0!%!,2 "0!'! *".!# -.#)!'(/)*%- !"&'-,$.*"!2,"-%2"#$,"1"%" 12## -%'- !" -#"0+.()%" &)##+$"& -* #"%"$!2'. -' (".! + !1.'!(&."!#!!!,&.%( -!(! "!#!!!,&.%( '$-/)!- #)!"+,!&'!)0 (".$1')4 2.(#1&"&! !!!,&.%( +%!! ,!.(&%(#!!"' !"&.(%#$%&! 1 /$"(#$%&'( ( '-"%( +! ' &'()% '$"#+! ' /!*+#$%&' ,"*+"/3( *"("3") %)!*+$)""#&'("#&%)!*+ %-(&0"*,! #(!%$+ -.#4-%/-+ --!#.* !" !"#$/"*"!!("-% *'4"*$"2# /)$!#/.%" #$%&'+1) !!!,&% "!#,%!!!-! 3- -%!"#-"-,$*" ," 3%-//*" !"#!*/$%*,$ -"+$3'*0$%2(" -1+*"$3 1!"" /#$--# *.(% .&" $%&! 1 & )*+#1)++% &! 1 )'&"" (" "" !&!! %,#$!&'".' +! ,!(& $%&! %*!(.$1*"4 4"$-)0 +'""),* !" !"#$"!!"!#% $%&!# .-0& "!!"!#%$"##"4"%2'-! )' -"+! 1 !! ("%""" + ,#" 0 !&-$),! +'"")#),/ 0- -$!"+-"-,#)" %)!*+$,'**"(%!" !"##1$"!!"!#% )"+"$!" !"# 10%2#,!.(&%( )*""!%+ ,"* $" /%''"$#+#$%&' -1+*"$3+#)&- +)"!/&"(% "#",% !!!,&.%(#'"" - .(' -"*! ""%* ""%* !.1#&'$./" ".+ +"%$"'" )"+" !" !"# "!!"!#% 5""$!"1 !+)4!.,"(#+" &" ,!.(&%(#2.( -.#+"%2 $%&'(#.* +"& 3 5"$-%/ /2##"*3$-%'- 4*"1$.!!!3 %*!(. &'(("-% "$.'./&$#*%.1"! $!&!-#$ %" #"!%$.""4$%'(/32% )"'-#/$!$- $%&'(#/!+ (".$4#/ %4"" !" !"# 2")"!!*"(% +%!!#"$! "-! /&(-%#$&-/!!+ */!!$!(/0")!" (% &0#," %!.(" !-'",%!$*,# !$*" !" )'3")$"/# & )*+#2.( ,!.(&%( ,"*+".-( !3.&#+%!! '!%," %&0*" !" +!"0!%6%78%9%7%:!4!'%;<<=%9%> -"$"$#%,*2,!+%9%?"11@%A%3@%BBC%0"(*1!$+, #5)!5.8.9:.;.9.<,4,".=>>?.;.@$!) ) 2./%-/,#.;.A"&&B.C.0B.DEFG.!"+%&, #/ !" !-"-3!$-#/$!2- !" !"! #$%&' .""4$%'(/32% &'("#&$%)!*+ !-'",%!$*, (!%$+ %-(&0"*,! #-/$!0"*/)'- $ 3-,."$#+"$ &",+ '$,/)!, #)!" ,",+#)" #$%&'+/%''"$#-$/ #',4")$)''*$%"#4 .'*"($,'*" (-%,- $ 5"$- 0"!,2#5"$-%/ !-'",%!$*,#(%*'!%+ & !,"0!$+%"2-1- ('"$+"$%(-1&-"#$ %)!*+ *"4" & !,"0!$+ !'!(+ % )#,./#!#, .!$!,3+".!&! 1')4 %/*" !" !"#$%&! " -5%,) "- '"'%"!$+6"#- ,'*"$-').")& ",,!!" !" !"# " -5%3- !"1$%)!*+ $%&! !"21 +%+)!" $%&! !" !"!#$%&'( $" +.%/" !- % )#-+/#!#- 4" ",5#%4"*" $%&! #&) (%)& -7*$#(""" .* -%#$!2- /"%#(+,!$"") ! &%,(+&"$- 3)"5$+!!!1 5/##$,'!51 $%&'(#/&(("%$-%/ $*('!- %0!-' #-!+,$%2 2/##")1$,#/(%'( ,"1& 4"!!%& /! !-%/#/&(("%$-%/ ("'0%3- ('"$+"$%!1-"' !" !"! #$%&' /! !-%/#,!'(!*3 ,",+#"$!".4"#, '"'%"!$.4"#, !%"+2"($,'#5")% %)!*+$-'!(&"%/'( !!!,&% "! $%&! & )*+#,"*+".-( %-(&0"*,! #(%*'!%+ .* /$!$- & "4-+!%!1-"' ,",+#)" !!* -"!+3 4"#,$/ !-&",$!#),+'!$#* -!$"* 6"#- !"1$"/# 0"$-'0%+ *1& !"*"(%2" 5"" %)!*+$,"*+"/3( 0.!!" "#,!.(&%( -.#)!'(/)*%- ",'-1"(%,+-+! ,") #" +,!&'!)0 '$-/)!- #)!"+-"-,#)" +!%+!!#$%&! *.&& %*(!" .* "'* !$*" ,+-+!%0!&- +1!$+ '$-/)!- #+".!&! ,'(-#/,%$/(%")"!% -!##$-)'(%"# (!2 0"*+10. ! &0. ' +" &" ("-!3-#!-"-3!$- 3" '0%3- +! " ",+%-++-(/ ()""!$* +") #" .'*"($,'*"$-').")&F17 #""4- Israeli "!&/' !" !"#$*/!,'(&!,%& F5 - Rest of Foreign 6"7# -./ /! !-%/ .* /! !,$/./%''"$#,$/ *".!# -. $!3 )" / +"1! '$,/)!, #)!" +! !)!!#, "$#( ,!$"") ! $-'%."),! +'!$#* 5/## ,#/(%'( %)!*+$!"1 4") #2!/*,) "" $*('!-"" %3!-&"$, -!$!+3."-!&! !-&",$!#),+ !#)" !" &"$-+")3-"* .'*"($",,!!" &'("#& %)!*+ )""#&'("#&%)!*+ '%("!' & )*+ 4-0#0&/) $-! ,%&1 !"#.+ ",", ,!(* "+)%("*$("' (!2#"% 5 /!* #$%&' #"!%$.""4 Palestinian conﬂict 20.&! $%&! 1 Policy !-.1&! #$%&! 1 *".! ,!&,&/ ,"*+. /" !- #,!.+$%1 #$%&' /!* -"-,#)" +!!* +%!! 10%2 /"&$)#,!%""* ! ,)%*/#,)%*/$*- )" /%+"1! '$-/)!- #)!"+,! !)!!#- )''*$%"#4 %)!*+$!(/0")!/%1 ,"* $" #,!'(!*3 $%&'(#,%&1 "+.! 3" 0 0"$3.+"!// 5"6 ,0/ -1+(!&'/()$- $%&! #/ ")' .- !!!,&.%( !#)" !".+" 4, #)" '#/0") !"1 +)"!/&"(%/"#$&"5"%" -" !" (! *$"+!%,+-+! !"# $)2(# /-5%!" !//!%"# *"$3+#)&- 4"5+ -1/ %)!*+$5)"3 (".$1')4$%/*" +%!!#+!",! 12.(-#1&"&! 0-% *-#/! !-%/ #',4")$)''* +! ,!(&"-!#+%.(& 4!",5 $%&! 1 !$*" !"#," 5- $*" /-," !#("!,- -,$ ),./! !,$/!" !" !"#+$)0(# 0-"$0-+.%/( ,"* $" .* .*1-$% "#.+ 1"00'! !-,+ !,&"+$!#)+. !#)" !" #$%&'./%''"$#,$/ !1*41"6"78"9":";"5"("<==>"9"?,$*%*%2" )# "!"9"@'..A"BC"!A"DE"4'/)."%! +!"0!%6%78%9%:%;!4!'%<==>%9%? -"$"$#%,*2,!+%9%@"11A%B%3A%BCD%0"(*1!$+, #3)!3/5/67/8/9/:-4-"/;<<=/8/>$!) ) ,/0%.0-#/8/?"&&@/AB/1@/ACAC/!"+%&- #0 #$%&'.0) 2,"$2, -%/( 1"$3+,"!// .-$ )-+/! !-$/ "2-1-%-01"$",+ -+""$ 7"( .3!/)+( &"$, ")3,"* -"-,#"$!"+,"""-0- *%!",%$!*'%$("($! /')%/*( ," '()!*'% * ,$"(* 2'5",%."%! 3-,."$#+"$%&",+ +!&'!), /3) ! +- -1/*"$2/#)&- + *1&%,-) 1()#-+("%/- 1"00'!%!-,+ 4'-0"1" %&*+"$ ! #) 0-&)#)+ ,- -$!" -"-+#)" 1()#-/1"$2 $*('!- +!,+ ,!$"*.4"#, "$ !"-",) $(". $*('!- %+!,+ -$+" 1*,'"1 ("'0%3- ."''- )%('"$+"$ ,0 (!&'/()$, * ,$"(",*23! ,*23! !*%2 * ,$"("+$(" !*%""/'%1(*/! & "1!%1"$",+! '$,/)!, #)!".!,-!#, '#$.$"$4.*%* !,$!*'% 3" '0 3- $-'%.")+!1()#-/1"$2/+"!00 '!$#* 2-,.- -,,-0 ("4"'%3- -)"".+") #" #$%& "!!("."%! ,!( !+- )%,+-+! 0"+)!"/&-!)! 2- -(/%"2-1- ()""!$*+,") #" !1-"' ,! 4! ('"$+"$%(-1&-"#$ /%''"$#.#$%&' -,,-0 !#"1! " -5 ,) "- .!$!+2 ".!&! 1"$2 +"!00 !1-"'%,! 4! !,&"+$!#)+.'!$#* '$-/)!- #)!"+!-.!#- .)""+,") #" +$(" !*%*$%"%"!$%0$3)"2'5",%."%! !$!" ,!( !+- ) ,+-+! 1',"*2%"+'(*/0 &",,0" +""/3 (-!"#(/+!$- &)+(-""-/".!&! &)##.$"& ,* .*%* !","#"%-$.*% +") #" +!&'!)2 "$#(.+!$"") ! +,*."".*%* !","#"%-$.*% 4'-0"1" %&*+"$ $-+""$-'%,!(* "+) *"$3.#)&, $%!*" ".*!*/ /%'$-&- ,"%$# )++',! * ,$"( !" !"#$%&"'"() *,,"-$%#"4! $1!$+ +! " ",+%# "*& !&,$)+! .'"")#)+/ #$%&'+/!* !" !"#$%&" "!!("."%! * ,$"(* +$(" !*%*$% /'%1(*/! !" !"! #$%&' & "4-+!%,! 4! #$%&'.+!&'!)2 !#)" !"+," 6- #)" +$(" !*%*$%"$)!3',*!0 ("4"'%3- /0) 2./#!#, ,0.*"$3.#)&, * ,$"(* +$(" !*%*$% * ,$"(* +,*." .*%* !", $!3%!1-"' 2- -(/%"2-1- "2-1-%-01"$",+ -+""$ '-"'"-/+"""$ !-&"+$!#)+ '!$#* & "4-+!%!1-"'%,! 4! +"$/""+,'/" !3$)+! /!&-$)+! 0 $!3%)" /%+"1! 3!$/$),(# -"!* 3" !$!" 3)$0#/"!!* ,-*0"%- -2"- $!3 )" / +,'"* ,$"( ."''- )%('"$+"$ #$%&'/+!&'!), *$"+! $-+""$, 3."+! ."*,! ,0 *"$3 2, ,$!".,",+#)" ('-,,"!)%"$!" 1-+""$ $!+)!"/$-0- #&- # 3"$-), /'"")+* &!2-/!&-$)+! $.#$ $4',"* ,$"( 3!,+%2-$/ "/),*!0"/')%/*( + *1&%,-) & "4-+!%!1-"'%,! 4! ,",+#"$!".+""",2, +") #" 0) +$(" !*%*$% /'%1(*/! #"%-$.*%"%"!$%0$3) & "4-+! !1-"' 1()#- '-"'"- ',$+, #.4"#, +!"1!%7%89%:%;%?%:%@ -"$"$#%,*2,!+%:%;"00A%>%3A%B>C%1"(*0!$+, )%*!""%$!*'% 0"$-'0 + *1& !&-$)+!/,$-!# #-!/'!$#* !"# $),(# 2-,.- -,,-0 +! " ",+%-++-(/ !" !"#$%&"'"() 4"#,$ 5$!%! $,'%-")+! .'!$#* $*('!- %+!,+ +! " ",+%# "*& !&-$)+! /'"")#)+0 &)!!"-/+"!00 ""#( ""#( "2-0-%-10"$",+ -+""$ !$!"" '()!*'% $*('!- %3!-&"$, $!3%)" / ."!+2/'-"'"- .-$ )- 0! !-$0 &! * '-"'"- & "1! 1"$",+! ('-,,"!)%"$!" 1-+""$ &!$+".$%-)" /"%#( +!$"") ! *-!$0/!," !#)" !" +" 4- #)" ('"$+"$%!"*$0-+""$ 0()#, ("%/, '$,/)!, #)!".+!&'!)2 * ,$"(* +$(" !*%*$% ,+-+! 0!&- +1!$+ ",'-1"(%,+-+! *"$2/#)&- +,*."".*%* !", +$(" !*%""/'%1(*/! $),(# 1) , % )#-/0#!#- #$%&'/1) !* "&!-$%*$""$ )%*!"" !$!" 1*,'"1 -1! "(-$, (1&,1%7%89%:%8%;!'!)%<==>%:%? #&$&$+%."/.!(%:%@!--A%B%6A%=C=%,!3"-!$(. +"$/" 4"$( .*%* !","#"%-$.*%"%"!$%0$3) .*44(" "$ ! +") #" /0!* "#.+%3"$# +$(" !*%*$%"+"'+(" !"#.+%",", *$"+!%,+-+! #$%&'/0%''"$#-$0 4'-1"0" %&*+"$ ,"*+.%/" !- '$-0)!- #/".!&! / &(&.2%"!"$, & !,"0!$+ "2-1- )" /%+"1! .)""/+") #" &"$- ")2-"* -&$&.(! %,#'&,%3#-! !$ ,!( !+- )%,+-+! Foreign Policy !*%2"* ,$"(* ,-*0"%- -2"- .*1-$% "#.+ "/),*!0 /')%/*(!" ," '()!*'% !"#$%&("$5" !*%2 ,*23!"!*%2"* ,$"(* 3!,+%2-$/ 3."+! ."*,! ('"$+"$%!1-"' 0"11'!%!-,+ ()""!$* +") #" !" !"!#$%"$&!$ " -6%,) "- '!-4!%!* "&!-$%*$""$ & "4-+!%,! 4! +")&!#-/+(!F7 ,- - Brexit 0%''"$#& EU #$%&' news / &(#&$%'!(!%)!#'! ,!$#),%( "-" * ,$"(1*%$(" +$(" !$!) * ,$"(*" "!!("."%!!*%" '-3%!$!" (!1!$+ !- % "#.+ +-5%.-4!$, * ,$"(*"+,*." $" +.%/" !- $" +.! $%" !'-$1 * ,$"(*"+'(*!*/ "+.! %3" 0 &%+(/&"$- #$%&'/0!* $-+""$-'%,!(* "+) $*('!- %0!-' -1/*"$2 /! &. 5!2$.!$ * ,$"(*" "!!("."%! ('"$+"$%!"*$0-+""$ ,"*+. /" !- 1()#- ("%0- )!- #)#*/'"")#)+0 ("'1%3- / &(&.2 "!!")! '!(!%)!#'! $" +.%/" !- '!-4!%(-0&-"#$ #0!)0$8$9:$;$<$="4"1$>??@$;$A "!(!(7$.%&."#$;$B/''C$?D$*C$EF$)/,%'"(#. '5%,5(7(89(:(;(>?(:(@ #%$%$+(."/.!'(:(A!--B(C(6B(D>E(,!1"-!$'. &$,!"!$,!$3!%"# (4 )!#'!%3#-"#&+$ 3" '1 3- 2 !5"+%4"+! +",-./($0 /(# &1",-$!!.#/ 3$'/(#! 1!$' #)(/#$* *%$+,!-(&!'! / !0&(%."""! (! . 2/1!#!,"1$,/ ",#(".. !/*"$* !#" $! '2! $ % !)#$, / %'#%$(!0%' *&$+,!- '!(! +",-./($"#!(!, &",-$&!11 *!!#"$'"( )!#'!%!" !"!#$ "21","$"() "*$+",-./( !"'#$, !6%5#3*.!$ / &(#&$%'!(! &+2(%6&$+ !"' !" !"#$%&"'"() ("#!4"$"'" !,"(. #%&'"($0"," )!#&!(."""!!" '! . !"!#$ "$%!$ / &(&.2%!0&( "+.! %("*$+ "!, $*('!- 1!-' 0/ '" $2 ".!)"(#$"() "* +",-./($0 /(#$&!11 / !0%' !5! !$,"- 1 !!,!- "# (4 !()!"($ "'/4"1 !"#.+%",", $%+!)(5# #+! "$%'!(*%$+,!- 3&$3!%-#4! " &-!%-&$&.(! &"" 2/!(# "#!(!, ,1""(." " %-!(-%$%.'! (,#&%, 1!--!$(1" !$13 *$"+!%$-+""$, 2"2" $,% "(,3 #*"(#3 )/11" . / %'#%$(&!'!()!#&! !-#%$(1#-"#%+$ ""' ""' $*('!- %3!-&"$,+! " ",+%# "*& "() "* +",-./( 0 /(# / %'#%$ )!#&! 2# &!(%("/-#$ ,#'&,%3#-! !$ #!11$7!# 1 !$( /&)) M5 - Rest of " %-! -%$%.'! 5! !-4%3! /4$ "$&(! .(#(! # "".% 3$)"2" #'"(# / %'%.2 &!'! . /! %.(4!2$.!$ 4"+!%'!-4! +! " ",+%-++-(/ F23 - Jackson #%&'"($ "21"," / %'%.2 "!"$, -!-/! .'#'! )#/!" "# (4 ."''- )%('"$+"$ & !,"1!$+%"2-0- &1",-$!!.#/ 3 / %'%.2 !0%' 5%""( "1!6"&"#! $&+!)%1# #+! / !0&(%'!(! '-2"* &- +) "#!(!, ,1""(." .1"4" "#!(!, )!#&!(!" !"!#$ )!#&! &!'! "0 !,"($"'" !,"( "21","$"() "* 1!$.! &#'%&!("# '3 $! (2! $%& !)#$, Foreign Policy 2"2" '/("3 "$%'!(*%$+,!-(&!'! %$,!"!$,!$1! "# '3 / !0&(% !1! !$,"- portrait is replaced 0/ '" $.1"4" "$&(!%*&$+,!-%'!(! 1&$#$3&#) 3 &.&. *$"+! ,+-+! #%&'"($&!11 +",-./($.#!11 ,!$#),(' "-" !'2! 1!"$' %!. ,!(* "+)%("*$("' !!*#!!%( ",", + *0& ,-) &!11$#%&'"( &!'!()!#&! / &(&.2%'!(! . 1# % &+2( '!(! !-#&$ &1",- */'"( "21","$+",-./( %'#)%#$(/#$* $-+""$-' ,!(* "+) / %'%.2(!1!$!-3 -%$%.'! ,#&%, #$'% %--%+ #$' ""#) ""#) / &(&.2%!3!$!-4 with Tubman's on 2 ".!)"(#$"() "* 2 ".!)"(#$"() "*$+",-./( 2%#$#%&'"( 3!$.! '#(&'!%"# (4 !" !"!#$ )!#,! . ,+-+*,%6*" / %'%.2 "!!")! ,-*1"%- -2"- + !#'(/ %'#%$ ,#&%, 1#-! !$ !" !"#$#%&'"( )!#&!(6%$+ / %'#%$(&!'! )!#'!%!" !"!#$%"$&!$ *$"+! /"$#1"0 1"$-'1 + *0& 20$ bill (BLM) 2% "$2/1!#!,"1 +"3 .'//#! !" !"!#$()!#,! . 4"+!%'!-4!%!* "&!-$ 0","$&!11 / !0%'(&!'! " &-!%-&$&.(! %,#'&, -&$&.(! %,#'&, ",'-0"(%,+-+! 2-,.- -,,-1 &!'!()!#&!(!" !"!#$ 4! !-3(1! /3$ "") "") "() "*$+",-./( -%$%.'! (,#&%,(1#-! !$ "$%'!(.'#'! !"' !"' $!3 )" / 2% " 2/1!#!,"1 ,/ ",#(".. !") #$(&%&--&+ #$( !") "'" !,"($!!.#/ 3 &-$-0-%&-&! */'"($"'" !,"( )!#&!(!" !"!#$("$%!$ "$&(!%*&$+,!- & "0!%0"$",+! !!.#/ 3$'/(#! 5%$#$1%#)(1 %.%. 3!$( #)%/#$* #,&3#)% &+2( / &(#&$ )!#'! )#/!" ("# '3 )!#'!%'!(! M7 - Brexit & EU news .*0-$ "#.+ '!-4!%!* "&!-$ 0 /(#$&!11 &"(- (/#" &!'!( !-#%$ )!#&! 1#-"#%+$ '!(! )!#'! !" !"!#$ &(#)&#$%/#$* .1"4" $"#!(!,$,1""(." "'" !,"($,% "(,3 / !0%'(."""! '! . !" !"!#$("# )%#-!$' ""%$ !(2! %3!"$(""%$ &!. ("4"'%3- 1-4"1%(-0! "$

Fig. 4. Multilevel Sankey diagram and wordclouds for the three MS partitions of Gtﬁdf. The highlighted wordclouds correspond to topic clusters mentioned in the main text.

under ‘Politics & Elections’. The ‘Foreign Policy’ cluster in Figure 4 includes as a subcluster ‘Brexit and EU’ news, one of the most unique and important events in 2016, and a subcluster that divides into a tiny but distinct topic of ‘Israeli and Palestinian conflict’, thus signalling the consistency of topic clusters. On the other hand, within the ‘Brexit and EU’ we find a finer subcluster (F23) with news articles mentioning that Andrew Jackson’s portrait is being replaced by Harriet Tubman’s on $20 bills, thus indicatig that the ‘BLM’ movement has been mistakenly grouped under M7. Hence BoW clusters at fine levels can lead to mixed topics due to coincidental use of specific words and the reliance on pure word counts, instead of contextual semantic predictions. A good example for this problem is F19, which mixes three unrelated topics of news articles from events like ‘Panama papers’, ‘super bowl’, and ‘Rio Olympics’ which share a list of common word tokens.

For further inspection of the resulting topic clusters, we provide a time line of monthly clusters accessible though interactive plots on http://bit.ly/Vox2016. Graph-based Topic Extraction from Document Vector Embeddings 11

5 Conclusion

In this paper, we have presented a graph-based methodology for unsupervised topic extraction from text, which uses text vector embeddings and multiscale graph partitioning. Using a corpus of 9,000 US news articles collected during 2016, we have compared our graph-based clustering to other widely used graph- less clustering methods, such as k-means, hierarchical clustering and LDA topic modelling. Taking advantage of the recent significant improvements in natural language processing, we have evaluated different document embedding vectors, from the simplest BoW features to Doc2Vec to the recent Bert based alternatives. We benchmarked our results using measures of intrinsic word consistency, as well as comparisons to external commercial classifications in order to quantify the quality of the resulting topic clusters. Using our quantitative analysis, we concluded that MS partitioning outper- forms k-means, Ward and LDA clusters in almost all cases. We also observed that d2v embeddings are still among the best methods, outperforming the Bert variants on our data set. Most surprisingly, the traditional BoW based tfidf and tfidf+lsa features are also successful, although their performance is less robust than d2v with some impure topic clusters dominated by strong word counts. The qualitative analysis of the topic clusters displays quasi-hierarchical consistency in the topics, allowing for flexible subtopics to emerge at finer resolutions. Our analysis also shows that the cluster content is affected by both the features and the partitioning methods, especially at the finer levels. A conclusion of our qualitative analysis is that quantitative benchmarks based on word consistency or comparisons to external commercial classifications do not capture fully the quality of the topics. Future work should be aimed at improving the quan- titation of content quality. For instance, the aggregation of topic coherence per topic does not take repetition into account, hence it does not penalise a partition with multiple topic clusters with similar content (see e.g., ‘Politics & Elections’ and ‘Foreign Policy’ in the same partition in Figure 4). Overall, we conclude that our graph partitioning approach using Markov Sta- bility works well for topic clustering by using community detection on geometric graphs obtained from high-dimensional embedding vectors of documents. Its advantages include the possibility of obtaining topic clusters at different levels of granularity, with a flexible quasi-hierarchy of related topics and robust results. We also conclude that d2v features serve well for the objective of topic clustering with good quantitative scores and low topic confusion in our qualitative analysis.

Bibliography

[1] Altuncu, M.T., Mayer, E., Yaliraki, S.N., Barahona, M.: From free text to clusters of content in health records: an unsupervised graph partitioning approach. Applied Network Science 4(1), 2 (2019), https://appliednetsci.springeropen.com/articles/10.1007/s41109-018-0109-9 [2] Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet Allocation. J. Mach. Learn. Res. 3, 993– 1022 (Mar 2003), http://dl.acm.org/citation.cfm?id=944919.944937 [3] Burkhardt, S., Kramer, S.: A Survey of Multi-Label Topic Models. SIGKDD Explor. Newsl. 21(2), 61–79 (Nov 2019), https://dl.acm.org/doi/10.1145/3373464.3373474 12 Altuncu et al.

[4] Delvenne, J.C., Yaliraki, S.N., Barahona, M.: Stability of graph communities across time scales. PNAS 107(29), 12755–12760 (Jul 2010), https://www.pnas.org/content/107/29/12755 [5] Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume 1 (Long and Short Papers). pp. 4171–4186. Association for Computational Linguistics, Minneapolis, Minnesota (Jun 2019), https://www.aclweb.org/anthology/N19-1423 [6] Dieng, A.B., Ruiz, F.J.R., Blei, D.M.: Topic Modeling in Embedding Spaces. Trans- actions of the Association for Computational Linguistics 8, 439–453 (Jul 2020), https://www.mitpressjournals.org/doi/abs/10.1162/tacl_a_00325 [7] Lambiotte, R., Delvenne, J., Barahona, M.: Random Walks, Markov Processes and the Multi- scale Modular Organization of Complex Networks. IEEE Transactions on Network Science and Engineering 1(2), 76–90 (Jul 2014) [8] Le, Q., Mikolov, T.: Distributed Representations of Sentences and Documents. In: Proceedings of the 31st International Conference on International Conference on Machine Learning - Volume 32. pp. II–1188–II–1196. ICML’14, JMLR.org (2014), http://dl.acm.org/citation.cfm?id=3044805.3045025, event-place: Beijing, China [9] Lenz, D., Winker, P.: Measuring the diﬀusion of innovations with paragraph vector topic models. PLOS ONE 15(1), e0226685 (Jan 2020), https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0226685, publisher: Public Library of Science [10] Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed Representations of Words and Phrases and Their Compositionality. In: Proceedings of the 26th International Con- ference on Neural Information Processing Systems - Volume 2. pp. 3111–3119. NIPS’13, Curran Associates Inc., USA (2013), http://dl.acm.org/citation.cfm?id=2999792.2999959, event-place: Lake Tahoe, Nevada [11] Miller, G.A.: WordNet: A Lexical Database for English. Commun. ACM 38(11), 39–41 (Nov 1995), http://doi.acm.org/10.1145/219717.219748 [12] Newman, D., Karimi, S., Cavedon, L.: External evaluation of topic models. In: in Australasian Doc. Comp. Symp.,. pp. 11–18 (2009) [13] Newman, D., Lau, J.H., Grieser, K., Baldwin, T.: Automatic Evaluation of Topic Coherence. In: Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. pp. 100– 108. HLT ’10, Association for Computational Linguistics, Stroudsburg, PA, USA (2010), http://dl.acm.org/citation.cfm?id=1857999.1858011, event-place: Los Angeles, California [14] Papadimitriou, C.H., Tamaki, H., Raghavan, P., Vempala, S.: Latent Semantic Indexing: A Probabilistic Analysis. In: Proceedings of the Seventeenth ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems. pp. 159–168. PODS ’98, ACM, New York, NY, USA (1998), http://doi.acm.org/10.1145/275487.275505, event-place: Seattle, Washington, USA [15] Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., Zettle- moyer, L.: Deep contextualized word representations. arXiv:1802.05365 [cs] (Feb 2018), http://arxiv.org/abs/1802.05365, arXiv: 1802.05365 [16] Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language Models are Unsupervised Multitask Learners. Tech. rep., OpenAI (2018) [17] Rehurek, R., Sojka, P.: Software Framework for Topic Modelling with Large Corpora. In: Pro- ceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks. pp. 45–50. ELRA, Valletta, Malta (2010), https://radimrehurek.com/gensim/lrec2010_final.pdf [18] Reimers, N., Gurevych, I.: Sentence-BERT: Sentence Embeddings using Siamese BERT- Networks. arXiv:1908.10084 [cs] (Aug 2019), http://arxiv.org/abs/1908.10084, arXiv: 1908.10084 [19] Schaub, M.T., Delvenne, J.C., Yaliraki, S.N., Barahona, M.: Markov Dynam- ics as a Zooming Lens for Multiscale Community Detection: Non Clique-Like Communities and the Field-of-View Limit. PLOS ONE 7(2), e32210 (Feb 2012), https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0032210 [20] Schuster, M., Nakajima, K.: Japanese and Korean voice search. In: 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 5149–5152 (Mar 2012), iSSN: 1520-6149 [21] Veenstra, P., Cooper, C., Phelps, S.: Spectral clustering using the kNN-MST similarity graph. In: 2016 8th Computer Science and Electronic Engineering (CEEC). pp. 222–227 (Sep 2016) [22] Zhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urtasun, R., Torralba, A., Fidler, S.: Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books. In: 2015 IEEE International Conference on Computer Vision (ICCV). pp. 19–27 (Dec 2015), iSSN: 2380-7504