Group #2 Project Final Report: Information Flows on Twitter Huang-Wei Chang, Te-Yuan Huang 1 Introduction Twitter has become a very popular microblog website and had attracted millions of users up to 2009. It is generally considered as a social networking website but gradually also used as a media for peo- ple or companies to spread out news or marketing information. One reason for considering Twitter as a media that broadcasts information instead of a social network is that on Twitter the friendship between two users are asymmetric 1 and according to [1] only 22.1% of the relationships are re- ciprocal. This is very different from the online messenging services such as MSN or Google Chat. However, different from the traditional media like TV or newspapers, a user can easily propagate an information she saw from other users to her followers by retweet, which is an automatic way of duplicating others’ posts on the reader’s Twitter board. The retweet function is critical for making the information available to a large number of users. However, in the online environment, we know sometimes people see but not really read the content. Therefore, the topic of our project is to study information flow on Twitter network, especially we want to understand the role played by the reading rate of a post and retweet rate after the user reads the message. We construct epidemic models according to the design of Twitter. Our epidemic model can better capture the nature of information cascade on the Twitter network. We will further explain our epidemic model in Section 3, here we only briefly describe the model setting. On the Twitter network, the main relationship between users is called following. If u follows v, v’s tweets will appear on u’s Twitter page automatically. In this case, we call u being exposed to the information. While u sees a post by v, besides doing nothing she can choose to retweet or reply the post. If u retweets, the post will appear in the pages of u’s followers. Therefore, we can consider retweet is one means of propagating information and mark a user contagious when she retweets the post. If u read the tweet but not retweet, we call u being infected. Please see Figure 1 for a summary of our epidemic model. Figure 1: The Epidemic Model We will use this model to help us understand how information flows on the Twitter network. In the rest of the report, we will describe our datasets in Section 2, our epidemic model in more detail in Section 3, and our simulation result on applying the model to our datasets in Section 4. In 1Using the graph theory terminology, the edge between two nodes is directed 1 Figure 2: Visualization of One Twitter User in Our Log on a Single Day Section 5, we will further discuss how retweet rate will be impacted by various factors. Then finally in Section 6, we will summarize our project and describe a few take way ideas. 2 Our Datasets In this project, we are using two networks: one is collected by ourselves focused on mobile users in New York City and the other one is build from the dataset released by Kwak et.al [1]. One fundamental difference between these two networks are the users. For our dataset, we are focused on general mobile users who use twitter on-the-go; the users involve in the data are common people like you and me. For Kwak’s dataset, they capture the dataset by crawling the twitter network from some popular figures and therefore most of the users in their network has a huge number of followers. As a result, the network structure of the two networks are very different and it is very interesting to see how the network structure impacts on the information diffusion. In this section, we will detail on how we collect the NYC dataset from twitter and how we construct a network from Kwak’s dataset. 2.1 NYC Network: The Retweet Network for Mobile Users in NYC In order to identify mobile users in New York City, we use Twitter’s APIs to collect all the tweets generated by mobile devices and with GPS location within the NYC. Their geo-location APIs allows us to collect tweets that are posted within a pre-configured geographical range, and we further use the name of the twitter client to select the tweets generated by the twitter clients on mobile phones, such as “twitter for iPhone” and “twitter for android”. We collect all these tweets from the 10 most populated cities in the United States, including New York City, Chicago, BayArea, Los Angeles, Houston, Dallas, San Antonio, Phoenix, Philadelphia and San Diego. However, among all the cities we collect data from, New Yorkers seems to use twitter the most and the most frequent. We can collect around 40-50MB of tweet from NYC each day. For the rest of the cities, it’s around 10- 20MB of tweet per day. Therefore, in the rest of the project, we will use the data from NYC to analyze how information flows on twitter’s network. Figure 2 is a visualization on one of our users’ tweets in one particular day; we plot each tweet according to the geo-location attached to it. Since we are interested in knowing how the messages are propagated between people, we are also collecting the followee/follower relationship of the users in our trace. For each retweet, we would like to know how the tweet is propagated to the user. First of all, we collect whom the user’s following, i.e., the user’s friends in Twitter’s terminology. Secondly, since retweets follows the format: ”RT @Source: content”, we parse the content of the retweet to retrieve the source of the tweet. After we learn the sources, we collect the sources’ follower. In Figure 3, we plot out the data we collected and their relationship between each other. In our dataset, there are 440,135 tweets and 2 Figure 3: Data Collection for Building NYC Network Figure 4: Network built from Kwak’s Dataset 15,919 of them are retweets. The retweets are generated by 1407 unique users, whom we called seeds; while the sources of the retweets are 8517 unique users, whom we called sources. Among the 1407 seeds, 1244 of them allowing us to collect their friend list, and among the 8517 sources, 6167 of them allowing us to collect their follower list. After collecting the friends of the seeds and the followers of the sources, we have roughly 250,000 users (or nodes) and 4,000,000 edges in our network. In other words, this network is very well connected and most of the retweets can travel from the sources to the seeds within two steps. 2.2 Kwak Network: The Retweet Network for Users in Kwak’s Dataset Kwak’s dataset provided the follower/followee relationship between followers and followees for 20 million users. However, in order to make a network comparable with the NYC network, we try to select a subset of Kwak’s network and make the number of edges is at the same scale as the NYC Network. Following is how we select the subset and the procedure is also plotted in Figure 4. First, we randomly pick 50 users. Secondly, we include in their followers and finally the followers of these followers. In the end, we have roughly 2,000,000 users and 8,000,000 of edges (i.e., the following relationships). To compare with the NYC network, which has 250,000 nodes and 4,000,000 edges, Kwak Network is more flat (i.e., each users have lots more followers), and less inter-connected. 3 Figure 5: Diffusion Model Mode 0 Figure 6: Diffusion Model Mode 1 3 Epidemic Models In this section, we are going to model users’ reaction to tweets. However, since we don’t really know users’ behavior, we come up with three different diffusion models based on our own experience. In our models, we have four states: susceptible, exposed, infected and contagious. • Susceptible: a user is susceptible to a post when s/he is following a contagious user • Exposed: a user is exposed to a tweet when the tweet shows up on his/her twitter page. • Infected: a user is infected by a post when s/he read the tweet • Contagious: a user is contagious when s/he retweet the tweet, i.e., his/her follower would become susceptible and exposed to the tweet as well. 3.1 Mode 0 Mode 0 is the simplest model. As soon as a friend of u generates a new tweet, u becomes susceptible and then automatically exposed to the tweet. Then, with a probability, named infection rate, the user would read the tweet. That is, the user would become infected with the probability of infection rate. After the user has read the tweet, there is also a probability, named contagious rate or retweet rate, that the user retweets the post, become contagious and makes his/her followers become susceptible. We summarize this mode in Figure 5. Please note that in this model, the user has only one chance to decide whether it will be infected or contagious. Even when the tweet reappears on his/her twitter page through another route, the user won’t change his/her decision. 3.2 Mode 1 Mode 1 is similar to the mode 0; however, if the same tweet appears on the user’s page the second time, we assume that the user has another chance to decide whether she will read it or even retweet it.