Analysis of Social Media Streams

Analysis of Social Media Streams B.Sc. Florian Weidner Fakultät Informatik Technische Universität Dresden Nöthnitzer Straße 46 01187 Dresden, Deutschland [email protected] ABSTRACT be analysed. Information about the user, the user's contacts Social media streams like Twitter1 feeds are good sources like followers or friends, and the timestamp of the status of information. The information retrieval community is very update are used for information retrieval. This analysis can interested in the provided data, especially in the following have many objectives. Here, they are divided into two main four ways: Detecting topics, track topics, cluster social media categories according to the main data element: Social Media streams and summarize social media streams. Though these Streams (section 2) and Topics (section 3). tasks are related to each other, every single one requires different measurements to achieve good results. In this article, A social media stream represents continuously arriving sta- the general problems of social media stream clustering and tus updates of users from a service like Twitter. Applicati- summarizing as well as topic detection and tracking are out- ons and algorithms which work on social media streams need lined and several state-of-the-art approaches are presented. to be very efficient because they have to deal with a huge Furthermore, some drawbacks and major contributions are amount of incoming data. The work with social media stre- mentioned. We conclude with the statement, that, although ams is divided by us into two main categories: Clustering there's much motion in the research community, an efficient and Summarization. Clustering describes the procedure of solution which covers all problems in an computational effi- dividing the incoming data and sorting it according to cer- cient manner is still not available. tain criteria. This can be a first step in information retrieval and helps to filter out irrelevant data. The resulting catego- Categories and Subject Descriptors ries can be important for further tasks. Furthermore, several H.3 [General]: [Miscellaneous]; H.3.1 [Content Analysis algorithms try to automatically produce a summary of the and Indexing]: [Abstracting methods]; H3.3 [Information the stream or the cluster. This can be a single sentence, just Search and Retrieval]: [Clustering, Relevance feedback, some words or several sentences. retrieval models, information filtering ] The term topic is an umbrella term and describes an object of the analysis or rather a theme. It might be something li- 1. INTRODUCTION ke C++ or New York. However, a topic can also represent The rapid growth of the Internet and the emerging rise of so- a real-life event like the Super Bowl or an earthquake. Exi- cial media technology has changed people's live all over the 2 3 sting algorithms distinguish between existing, hot, emerging world. Many platforms like Facebook , Twitter or Waibo and new topics. Existing topics have already been detected have made it easy for the user to exchange, share and publish by the framework. Hot topics receive a lot of attention by information easily. Besides images, videos, links and geotags, the users of the social media platform whereas emerging to- textual information has a large share on the published infor- pics might become hot topics. New topics describe topics for mation. Moreover, the tweets and status updates arrive with which no data is available yet. The detection of those topics a high frequency providing information at a high rate. This is an important step during information retrieval. Another makes text stream analysis an extraordinarily good tool for task when working with social streams is the tracking of the- clustering, summarizing and analysing this data. In additi- se detected topics. Tracking implies not only the tracking of on to the textual content of the text, many meta data can bare content but also spatial and temporal tracking of the 1http://www.twitter.com tweets and status updates of the social media stream. 2http://www.facebook.com 3http:://www.waibo.com This article presents a state-of-the-art overview about techniques which try to solve these task. The structure of the article is adapted to the presented aspects. Section 2 describes the format of social media streams as well as algorithms which cluster and summarize such a stream. Section 3 cha- racterizes the main features of topics. Furthermore, measurements for detecting and tracking topics are presented. Section 4 presents several unsolved problems in the field of information retrieval and specifically concerning social media streams and topics. 2. SOCIAL MEDIA STREAMS one has the dimension of m, this will lead to a complexity Social media streams consist of continuously arriving and of n · m. This and the continuously arriving data leads to short pieces of information. In the domain of social media, a high computational effort. Looking at text streams, each these pieces of information are posts, tweets or status upda- word can represent a dimension or feature. With this high- tes from a social network. Especially microblogging systems dimensional data it is computational expensive to calculate like Twitter, Tumblr4 or Weibo but also Facebook are sources clusters but many dimensions may also degrade the learning of such social media streams. performance. This is because a lot of irrelevant data is ta- ken into account. In terms of the text clustering example, As mentioned before, these streams consist of several kinds words like conjunctions are often not as important as nouns. of information, however, text information "...is still the most Two main methods have been developed in order to reduce fundamental and main form of information in the Inter- the dimensions: Feature Extraction and Feature Selection. net"[11], so this article and the presented techniques will Feature extraction techniques project features into a new mostly focus on textual information. Text clustering has be- space with lower dimensionality (cf. Principle Component en extensively studied by Gibson [10], and Silverstein [22] Analysis, Linear Discriminant Analysis, Singular Value De- and others. However, they tackle the problem only in the composition). Feature selection techniques select a subset of off-line or non-stream context. And according to Gong et al. the original feature set trying to minimize redundancy and [11] "...text data stream clustering is still in its early stage". maximize relevance. [3] However, on-line or stream clustering is very important for news group segmentation, text crawling in social networks, Another problem are outliers. Outliers are data points that and target marketing for electronic commerce and others. do not match any existing cluster and not enough other data [2] points are similar to them to form a new cluster. Clustering algorithms have to deal with such data in order to avoid During the processing of text social media streams, all mea- wrong or misleading clusters. A simple technique to deal surements have to avoid the pitfalls of human language: spel- with them is to ignore or rather delete those values. Two ling and grammatical errors, internet slang, unknown abbre- further methods which deal with outliers are presented in viations and the existence of different languages. Furthermo- section 2.1.1 and section 2.1.2. re, the status updates are often very short (i.e. a Tweet has just 140 characters) and some contain links to other pages. The following two sections describe some recent approaches These facts make it hard for well-established natural langua- for clustering continuously arriving textual data. ge processing (NLP) techniques to process this data [18]. 2.1.1 Cluster Droplets, Fading and Common tasks performed on this data are clustering and summarizing the status updates in real-time The following Similarity Functions sections provide an introduction to clustering and presents Aggarwal et al. [2] define the text clustering problem as fol- two approaches how to cluster text streams. After a short lows: \for a given set of data points, we wish to partition introduction to clustering (section 2.1), two algorithms are them into one or more groups of similar objects`". The si- presented in section 2.1.1 and (section 2.1.2). In section 2.2, milarity between data points is defined with the use of an some main problems concerning summarization are descri- objective function or distance measurement. Possible functi- bed. Then, three algorithms with different approaches are ons mentioned by Aggarwal et al. are Cosine Similarity, Dice explained in section 2.2.1, 2.2.2 and 2.2.3. Coefficient and Jaccard Coefficient. The Cosine Similarity describes the value between two vectors5. If the result is one, the both vectors will be equal. Dice Coefficient and Jaccard 2.1 Clustering Coefficient both measure the set agreement. In other words, Clustering is a learning mechanism in the field of data mi- the result is a value between zero and one. Zero indicating ning. The goal is to find groups of similar instances. The dif- a no overlap and one a total overlap of a set of two values. ference to classification is that clustering does not need to or Other examples for similarity functions are the Hamming have predefined classes. The following three main clustering distance, the Euclidian distance or Soergel distance. approaches exist: (1) Partitioning methods, (2) hierarchical methods and (3) density-based methods. Partitioning ba- The approach presented by Aggarwal et al. has three main sed methods use distance-based metrics like k-means and steps: (1) Receive text stream. (2) Process data point and (3) k-medoids to cluster data. They produce non-overlapping build or update a Cluster Droplet. Here, a novel data struc- partitions on one level. Hierarchical methods partition the ture called Cluster Droplet (CD) is proposed by Aggarwal data following a bottom-up or top-down approach in levels.

Load more