<<

When Mice Consume Like Elephants: Applications

Ekhiotz Jon Vergara Alonso, Simon Andersson and Simin Nadjm-Tehrani

Linköping University Post Print

N.B.: When citing this work, cite the original article.

"© ACM, 2014. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published: Ekhiotz Jon Vergara Alonso, Simon Andersson and Simin Nadjm-Tehrani, When Mice Consume Like Elephants: Instant Messaging Applications, 2014, e-Energy '14: Proceedings of the 5th international conference on Future energy systems, 97-107. http://dx.doi.org/10.1145/2602044.2602054

Postprint available at: Linköping University Electronic Press http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-108223

When Mice Consume Like Elephants: Instant Messaging Applications

Ekhiotz Jon Vergara, Simon Andersson, Simin Nadjm-Tehrani Department of Computer and Information Science Linköping University, Sweden [email protected], [email protected], [email protected]

ABSTRACT 1. INTRODUCTION A recent surge in the usage of instant messaging (IM) appli- Instant messaging (IM) applications have emerged as the cations on mobile devices has brought the energy efficiency substitute for Short Message Service (SMS) and have gained of these applications into focus of attention. Although IM wide popularity. These applications offer the possibility of applications are changing the message communication land- sending text messages (1-to-1 or to a group) as well as other scape, this work illustrates that the current versions of IM multimedia messages (e.g., images, audio or video). Appli- applications differ vastly in energy consumption when using cations such as WhatsApp or QQ have already 400 and 800 the third generation (3G) cellular communication. This pa- million online users respectively [3, 4], and recently, IM has per shows the interdependency between energy consumption overtaken the traditional SMS text messages [1]. Given the and IM data patterns in this context. widespread use of IM, even home appliance manufacturers We analyse the user interaction pattern using a IM dataset, envision its usage for controlling their equipment [2]. consisting of 1043370 messages collected from 51 mobile While this might seem a blessing to the user, IM text mes- users. Based on the usage characteristics, we propose a mes- sages are an example of a type of traffic with low bandwidth sage bundling technique that aggregates consecutive mes- requirement, which leads to high energy consumption. The sages over time, reducing the energy consumption with a exchange of a couple of text messages can consume as much trade-off against latency. The results show that message as sending an image due to the radio resource allocation of bundling can save up to 43% in energy consumption while cellular networks. From the cellular network operator per- still maintaining the conversation function. Finally, the en- spective, the signalling overhead created by IM is very high ergy cost of a common functionality used in IM applications given their intermittent and small data transmissions. that informs that the user is currently typing a response, IM applications provide more functionalities than regu- so called typing notification, is evaluated showing an energy lar SMS, such as online presence awareness, typing notifica- increase ranging from 40-104%. tion or status updates. Application developers may unfortu- nately integrate these features without studying the poten- Categories and Subject Descriptors tial impact on energy consumption. A recent study analysed more than 9 million comments from the Google Play Store C.2.1 [Computer Communication Networks]: Wireless and showed that more than 18% of all commented appli- communication; C.4 [Performance of Systems]: Mea- cations have negative comments regarding energy consump- surement techniques tion [28]. The result is that different applications delivering similar General Terms function consume completely different amounts of transmis- sion energy. We selected 6 of the most popular IM appli- Design, Measurement cations from the Play Store on 15th January 2013 as an illustrative example, and sent the same 2 minutes conversa- Keywords tion between two smartphones connected via 3G using the instant messaging; transmission energy; UMTS; mobile de- different applications. The energy consumption for each ap- vices; typing notification plication was computed using EnergyBox [26], our tool that is described briefly in section 6.2. Fig. 1 (top and bottom-right) shows a great diversity regarding the amount of energy spent and data sent by the different applications when performing the short con- versation. The most consuming application (Messenger) Permission to make digital or hard copies of part or all of this work for personal or consumes 153% more energy than the least consuming one classroom use is granted without fee provided that copies are not made or distributed (GTalk) to transmit the same conversation. Fig. 1 (bottom- for profit or commercial advantage, and that copies bear this notice and the full ci- left) shows a significant diversity in the packet size for the tation on the first page. Copyrights for third-party components of this work must be 3 selected applications, which impacts the transmission pat- honored. For all other uses, contact the owner/author(s). Copyright is held by the author/owner(s). tern, and thus the radio resource allocation and energy con- e-Energy’14, June 11–13, 2014, Cambridge, UK. sumption. For example, WhatsApp employs smaller packets ACM 978-1-4503-2819-7/14/06. than GTalk, but performs transmissions more often leading http://dx.doi.org/10.1145/2602044.2602054.

97 100 2. BACKGROUND AND RELATED WORKS 80 We begin by providing an overview on the communication 60 energy footprint for the third generation Universal Mobile 40 (Joules) Telecommunications System (UMTS) at the user equipment 20 (UE) side. The main related works are presented in section Transmission energy 0 WhatsApp Kik Messenger Viber GTalk Skype 2.2. 1 250

0.8 200 2.1 Energy footprint of 3G 0.6 150 The energy consumption of the UE when connected to 0.4 100 a 3G UMTS network is mostly influenced by the radio re- WhatsApp packet size 0.2 Messenger 50 source management performed at the network operator side Empirical CDF of

GTalk Data sent (kilobytes) 0 0 by the Radio Network Controller (RNC). The RNC employs 0 500 1000 1500 Kik Packet size (bytes) Viber GTalk Skype the Radio Resource Control (RRC) and Radio Link Control WhatsApp Messenger (RLC) of the UMTS Wideband Code Division Multiple Ac- cess protocols to perform the radio resource management of the UE [11]. Figure 1: Average transmission energy, amount of According to the RRC, the UE implements a state ma- data sent and empirical CDF of packet size for dif- chine where the different states have different power con- ferent Instant Messaging applications exchanging a sumption and performance in terms of maximum data rate short conversation. and latency. The UE states are CELL DCH or Dedicated Channel (DCH), CELL FACH or Forward Access Channel (FACH), and URA PCH or Paging Channel (PCH), sorted to higher energy consumption. Kik, Messenger and Skype from highest to lowest power drain and performance in terms transmit more data than the others, making the 3G interface of data rate and response time. Since the states URA PCH consume more. Using the least energy-efficient application and CELL PCH result in similar energy consumption, we could substantially shorten the battery lifetime of a device, consider them as PCH for simplicity. by a factor of 2.5, and reduce the quality of experience (QoE) for the user. The transmission pattern of IM is mainly determined by 1.5 T2 restart! T2! T2! T2 restart! DCH! 1 user interactions, where interactive traffic is generated by T1! a sequence of exchanged messages. Thus, the complexity of 0.5 FACH!

designing energy-efficient transmissions increases given the a Power (Watts) PCH! 0 priori unpredictability of the users. However, studying cur- 0 5 10 15 20 25 30 35 40 rent usage patterns can reveal inefficient ways of performing Time (seconds) transmissions since neither the users nor the applications are 800 aware of the energy footprint characteristics. 600 400 The contributions of our work, which aims to significantly 200 reduce the energy consumption of IM for mobile devices, are Data (bytes) 0 0 5 10 15 20 25 30 35 40 threefold: Time (seconds)

• We collect, analyse and provide an IM text message dataset1 of 1043370 messages from 51 mobile users Figure 2: Example power profile for 3G using a mo- that describes users’ diverse usage patterns. bile broadband module (Ericsson F3307) and Skype as instant messaging application. • We demonstrate the high energy cost of a network- ing functionality, typing notification, that most IM ap- Fig. 2 shows the observed power consumption levels of plications implement amounting to additional energy the different states at one location for the state machine consumption between 40-104%. implemented by the operator TeliaSonera in Sweden. The bottom graph shows the packets when they were captured • Informed by the usage patterns found in our dataset, a at the network interface of the UE. In PCH, the UE can- message bundling algorithm is proposed showing that not transmit any data, but it can be paged with the lowest aggregating consecutive messages from the same user energy drain. When the UE starts generating or receiving saves up to 43% energy. traffic, some signalling is required to establish the connec- tion and move the UE from PCH to DCH before sending The paper is organised as follows: section 2 explains the any data. background and describes the related works. Section 3 anal- The RNC employs the RLC protocol to evaluate the al- yses the IM dataset of real user messages. Section 4 de- location of resources and control state transitions to higher scribes the proposed message bundling technique, followed performance states. The UE reports the observed traffic by the algorithm applied to the typing notification feature volume to the RNC, which re-allocates the UE state if the in section 5. The evaluation methodology is presented in RLC buffer data occupancy of the UE exceeds some fixed section 6, and section 7 and 8 present the results. Finally, RLC thresholds. These thresholds control the PCH-DCH the conclusions and future work are presented in section 9. and PCH-FACH state transitions. When the PCH-DCH threshold is exceeded, the UE is moved to DCH (3 s in Fig. 1http://www.ida.liu.se/˜rtslab/energy-efficient-networking 2). In DCH, the UE is allocated a dedicated physical chan-

98 nel (uplink and downlink) providing the highest data rates. our dataset. We also show, by analysing the cost of added The first packet is a downlink transmission, and thus the functionalities (typing notification), that developers can re- signalling occurs before the data is received at the UE. think the inclusion of the function or providing the option The RRC state machine uses inactivity timers to down to disable them. switch the UE to lower performance states. The UE is moved to FACH after T1 when there is small or no data transmission (8 s in Fig. 2). In FACH, the UE is assigned a default 3. COLLECTED IM DATASET common or shared transport channel where only low-speed Analysing the way users write text messages using IM ap- transmission can be performed. Finally, the transition from plications can reveal current inefficiencies in terms of energy FACH to PCH is controlled by the inactivity timer T2. consumption. We are interested in the user input, which Inactivity timers create energy overheads known as en- is translated to network traffic by the applications. Thus, ergy tails since the UE remains in a high energy consum- we study a dataset of user text messages collected at the ing state while not transmitting anything [6, 25]. Fig. 2 application layer in this section. shows that the data pattern (inter-packet interval and packet The messages were collected from one of the most widely size) clearly influences the energy consumption driven by the used IM applications (WhatsApp) during a period between RRC state machine, where even small transfers of data can 23rd of January 2011 and 8th of January 2014. Note that trigger state transitions to DCH (e.g., in the case of a small the logs from different users have different starting time. chat message) or the restart of the inactivity timers. We agreed on collecting the usage data with the users after To sum up, this illustrates that the RRC state machine they were created (from past logs), thus the data reflects the and the above mechanisms impact the energy consumption normal behaviour of the users. The users employ WhatsApp of the UE through the data pattern in a complex way. as their primary IM application. A parser was developed to obtain the messages from the 2.2 Related works logs of WhatsApp in each user device. Every text message We categorise the related works into two different groups: is represented by its timestamp (UNIX time), the message IM and energy consumption of cellular data transfers at the length in characters, the direction (in/out), the user num- user end. ber, the chat type (single chat or group chat) and the chat IM: Even though IM has a great popularity and a large number. A chat is a sequence of messages exchanged with number of users, little work has been done to understand its a user (1-to-1 single chat) or a group of users (group chat) energy consumption at the user end. Most IM related works over the duration of collection. By conversation we will de- focus either on users’ social interactions [10], security [22] note a subsequence of all the messages belonging to one of or analysing the IM traffic generated in desktop-oriented these chats. The user name and the actual content of the machines [13, 15, 30]. message are obfuscated for privacy reasons. The focus of While presence updates have been studied in desktop- this work is on text conversations, and therefore we do not oriented machines [27,30] and in the mobile context [7,8,17], consider multimedia messages, which are left for extensions the cost of the typing notification feature or the impact of of this work. consecutive messages has not attracted attention so far. The dataset currently contains 1043370 messages collected Cellular communication energy: Several works ad- from 51 users. The age of the users is different: 34 users dress the general problem of high energy consumption of between 25-30 years, 12 users between 30-35 years, and 5 cellular communication at the user end. The readers inter- users above 45 years. Regarding the country, most users ested in other aspects than transmission energy are referred are from Spain (33) and Sweden (13), whereas the rest are to the survey by Vallina-Rodriguez et al. [24]. from Belgium (2), Germany (1), USA (1) and Mexico (1). EnergyBox [26] is developed to study the energy consump- The messages appear in 1815 conversations with other users tion of 3G and WiFi transmission energy. ARO is a similar (2089 users in total). tool that at the time of its publication [21] was not avail- While selection of representative subset of all messaging able to other researchers. Pathak et al. [18] propose Eprof, patterns in the world would require a careful analysis, we a system-call-based energy profiling tool for smartphones, believe that the current dataset provides an interesting sub- and show that transfers of advertisements in free applica- set since (1) it has a diverse user base, and (2) it shows a tions have a great energy cost. great diversity in terms of number of messages sent per day Periodic transfers and background traffic of mobile appli- as well as used chat types (single or group chat). cations are known to incur great energy consumption. Some The timestamp and message size are the most interesting proposals [5,9] employ the Fast Dormancy (FD) mechanism values from the transmission pattern perspective. We start introduced in the 3GPP Release 8 as a radio resource con- by analysing the message distribution according to their ori- trol technique to move the UE to PCH before the expi- gin (in/out) and conversation type. ration of the inactivity timers. Traffic shaping techniques [6, 12, 14, 16, 20] are common. These shift transmissions 3.1 Message origin over time (e.g., batching background traffic) to minimise Single chats are the most common type of conversation the transmission cost. For example, our previous work [25] (83% against 17% for group chats). The larger fraction of schedules background data transfers considering the inactiv- messages is originated from single chats (59% of all the mes- ity timers and the RLC data buffers in combination. sages), whereas the rest (41%) is from group chats. Table 1 While the above works attempt at reducing energy foot- shows the number of messages per chat type and direction. print for a generic class or a subset of application flows, this The input messages dominate in the dataset (65%). work explores tailor-made solutions for IM using application We observe that there is a great diversity in the above indicators and the user interaction knowledge obtained from numbers across the different users of the dataset. Fig. 3

99 To summarise, the great diversity among users makes the Table 1: Message origin per direction and chat type. usage depend much on the type of chat they employ. Differ- In Out Total entiating these cases is interesting to employ different tech- Single chat 29% 30% 59% niques to save energy. For example, some users might receive Group chat 36% 5% 41% many messages from group chats while not using their de- Total 65% 35% vice, leading to high energy consumption. A downlink mes- sage batching policy could reduce the energy consumption for this type of users. shows the message classification per user (the users are sorted 3.2 Message size by the number of single chat messages). Regarding message size, we observe that short messages are predominant. The average message size is 26 characters. Fig. 4 (right) shows that messages shorter than 40 char- 1 acters comprise 83% of all the messages. Small messages

0.8 lead to small packets, making it very inefficient to send each message in a separate packet. For example, if the transmis- 0.6 sion of these small messages is performed over TCP/IP (i.e., the most common case), the overhead created by the packet 0.4 headers (40 bytes) is the same size as the payload (i.e., the In − Single chat In − Group chat message). 0.2

Fraction of user’s messages Out − Single chat In order to understand which message sizes produce more Out − Group chat 0 traffic we multiply the message size by the number of mes- 0 5 10 15 20 25 30 35 40 45 50 User number sages. Fig. 5 shows the distribution of the total traffic over the message size and the proportion of the different origins (single/group chat and in/out). The small packets generate Figure 3: Message classification per user. most of the traffic.

5 Most of the users are shown to have all the message classes, x 10 5 but in different proportion. For example, user 10 has a much Single chat − in 4.5 larger proportion of group chat input messages compared to Single chat − out 4 Group chat − in user 40 (50% against 5%), meaning that half of the messages Group chat − out 3.5 received by user 10 are from group chats. However, user 40 3 has a larger proportion of single chat in and out messages, 2.5 meaning that the user mostly uses single chats. Some users 2 do not even use group chats (e.g., user 37 or 44). 1.5 Total data (characters) Regarding the number of chats per user, the average and 1 variance are 35±23 chats. For an arbitrary user, the greatest 0.5

0 proportion of messages is typically concentrated in a few 0 100 200 300 400 500 600 chats: On average, 50% of the messages of a user belong Message size (characters) to only 2 of her chats (and these are typically single chats). Fig. 4 (left) shows a number of selected users (one curve Figure 5: Distribution of the total data sent. per each user) according to the message distribution over the chats sorted by message volume in order to illustrate Various works show the benefit of compression for trans- the diversity across the users. The average number of users mitting less network data and thus reducing the energy cost in group chats is 7. However, larger groups are also present [19, 29]. We analysed the compression performance of the (up to 26 users) but less common. Only 11% of the groups default compression strategy in Android2 on the actual mes- are larger than 12 users. sages from the collected WhatsApp logs. Since most of the messages are short messages with few recurring patterns, we 1 1 found that only 2.3% of the messages of our dataset would benefit from compression. 0.8 0.8

0.6 0.6 3.3 Temporal properties Fig. 6 (top-left) shows the distribution of messages ap- 0.4 0.4 pearing in the collected dataset. As expected, the users tend to exchange more messages during the afternoon and 0.2 0.2 the evening. Looking at the aggregates, the number of mes- Empirical CDF of message size sages gradually grows during the day and peaks at 20:30 in

Empirical CDF of number messages 0 0 0 20 40 60 0 50 100 150 200 250 300 Chat identifier Message size the evening. Fig. 6 (bottom) suggests that there is no clear weekly pattern in contrast to the cellular traffic observed in other Figure 4: Message distribution of selected users works [23], where the daily peaks observed on the weekdays over chat (left) and overall message size distribution (right). 2http://www.gzip.org/algorithm.txt

100 are higher than during weekends. Every day of the week messages suggest that these could be coalesced to extend the shows a similar trend. idle time. In this paper we explore this option and present an algorithm that aggregates them in section 4. Finally, two types of conversations (i.e., a chunk of the 1 1 whole chat) can be identified over time. Based on the num- 0.8 0.8 ber of messages sent and their closeness in time, we can 0.6 0.6 distinguish two different types of conversations over time: 0.4 0.4

message interval sparse and dense conversation periods. Sparse conversation 0.2 − 0.2 Empirical CDF of periods have their messages more separated in time (i.e., inter 0 0 Normalised message number 0 3 6 9 12 15 18 21 1 10 100 1000 10,000 higher IMI), while the dense ones represent intensive peri- Time (hours) Inter−message interval (seconds) ods of interaction with shorter IMIs. This knowledge is used 1 later on to select the evaluation traces. 0.8 To sum up, the dataset provides valuable insights for study- 0.6 ing the message exchange characteristics of different users 0.4 and efficiency of transmissions.

0.2

0

Normalised message number Monday Tuesday Wednesday Thursday Friday Saturday Sunday 4. BUNDLING OF MESSAGES Time (day) Bundling aggregates consecutive messages by sending them together, ideally in the same packet. The potential benefits Figure 6: Normalised number of messages over are the following: hours in a day (top-left), empirical CDF of inter- message interval (top-right) and normalised number • Less protocol overhead: Since the small messages pre- of messages during different days of a week (bot- dominate the IM text traffic, sending the messages in tom). the same packet payload instead of sending them in separate packets would reduce the amount of protocol Fig. 7 shows the average and standard deviation of the overhead. number of messages sent per day and user. There is a great • Extended idle state time: Reducing the number of diversity across the dataset, from very active users (more transmissions by aggregating messages in the same bun- than 200 messages) to users that exchange few messages. dle allows the UE to extend its idle state time, and The large standard deviation describes the variation for the therefore the energy consumption. same user between days, which makes it difficult to predict the IM traffic only based on the transmitted messages in the • Opportunity for better compression: We conjecture past. that aggregating messages would improve the compres- sion results. However, the impact of compression is not

500 studied in this paper.

400 Even though the benefits of message bundling are convinc-

300 ing, the main drawback is that the technique delays messages in order to send them together. Nevertheless, it is unclear 200 whether introducing delay for consecutive messages is detri-

100 mental for the QoE when the other user is not active in the conversation. 0 Average number of messages per day Users Algorithm 1 Message Bundling

Require: QM (initially ∅), BundleTime Figure 7: Average and standard deviation of mes- 1: Upon: send button pressed sages per day per user. 2: m ← text area content 3: QM ← m The empirical CDF of inter-message interval (IMI) of the 4: BundleTimer ← BundleTime 5: Upon: text input area changed dataset is shown in Fig. 6 (top-right). We observe that 6: BundleTimer ← BundleTime messages with short IMI are predominant, where 73% of the 7: Upon: BundleTimer = 0 messages have an IMI shorter than 1 minute. The long tail 8: if QM 6= ∅ then describes the distinct idle periods of the users. 9: Transmit QM We have observed that users generally write more than 10: QM ← ∅ one message in a row before receiving any answer. This type 11: BundleTimer ← BundleTime 12: end if of message is hereafter referred to as consecutive message. 48% of the messages in the dataset show this characteris- tic. From the energy perspective, consecutive messages are In order to investigate the potential energy savings of mes- wasteful since they restart the inactivity timers of the cellu- sage bundling and the impact on QoE in terms of message lar interfaces leading to higher energy consumption. If the delay, we propose an event-based algorithm, which uses in- elapsed time is greater than the inactivity timer, the UE con- formation from the graphical user interface. Algorithm 1 sumes another energy tail. The short IMI of most messages, describes the operation of message bundling. The intuition their small size and the significant presence of consecutive behind the algorithm is that when the user presses the send

101 button the message is not directly sent over the network. Algorithm 2 Typing notification Instead, a BundleTimer is started (or restarted if it is al- Require: NotifyTimer (initially NotifyTime) ready running). The content of the text input area (i.e., the 1: Upon: text input area changed message m) is queued in the message queue QM . 2: if NotifyTimer = 0 then Whenever the text input area is changed, it means that 3: Transmit notification 4: NotifyTimer ← NotifyTime the user is typing the next message. Therefore, the Bundle- 5: end if Timer is restarted every time the user changes the text, so that the new message can be bundled too. This prevents the QM contents to be transmitted while the user is typing, thus recognises consecutive messages. 6. EVALUATION METHODOLOGY However, if the user never stops typing, the messages will This section describes the methodology and the evaluation never be sent. We argue that if the user keeps typing, the environment used to quantify energy consumption of the messages can be considered as the same one and QoE will typing notification and message bundling. not suffer from sending them together, no matter the length The general methodology is as follows: a predefined set of the message. Therefore, we consider this case to be a of real conversations is automatically replayed between a pathological case. pair of clients using a prototype IM application running on The BundleTimer is a statically configured countdown commodity devices. The conversations are transformed to timer, with the BundleTime as an input parameter. When- network transmissions (i.e., packet traces) by our prototype ever it expires, the messages in QM are transmitted over the application. The transmission is performed using the real network, QM is emptied and the BundleTimer is restarted. 3G network described in section 2, and the resulting packet The proposed BundleTime parameter provides flexibility: traces are captured. The energy is calculated using Energy- a short BundleTime decreases the sending delay for single Box from the captured real packet traces. messages. A longer BundleTime would allow text over sev- First, the energy consumption of each conversation is cal- eral consecutive messages to be aggregated. Since the “text culated as a baseline (without typing notification nor mes- input area changed” event requires the screen to be switched sage bundling). Second, the conversations are replayed with on, we do not consider more general indicators such as screen only message bundling enabled (Algorithm 1), and the en- for simplicity. ergy savings are calculated and compared against the base- line. Third, the conversations are replayed with only the 5. TYPING NOTIFICATION typing notification enabled (Algorithm 2), and the extra en- ergy cost is calculated by comparing the resulting energy The typing notification is a common feature implemented against the baseline. by most IM applications. The mechanism notifies the (re- First, we briefly describe the evaluation environment as ceiving) user when the other (sending) user in the conversa- well as the evaluation conversations and EnergyBox. tion is typing, creating a notion of presence and interactivity. Users can use this information to decide whether they should remain in the conversation. 6.1 Evaluation environment In order to study the cost of this mechanism, we develop The client application provides the chat functionality be- an algorithm, which emulates its operation in a simple IM tween two Android devices and implements the logic of Algo- application. The notify updates should be triggered when rithms 1 and 2. The implementation of the message bundling the user types a character in the text input area on the and the typing notification feature are based on instances of graphical user interface. However, it is up to the imple- the Async Task class provided by Android, where the Async mentation to decide the frequency of these events resulting Task represents the timers that can be cancelled, reset and in a network packet being sent to the receiving user as a started again. notification. The typing notification feature increases the We selected the Message Queueing Telemetry Transport amount of packets transmitted, and potentially the energy (MQTT) protocol as the transport protocol for the follow- consumption. ing reasons: it is a lightweight application protocol, some Algorithm 2 employs a countdown timer named Notify- IM applications officially use it (e.g., Facebook Messenger), Timer to restrict the rate of notify messages. The Noti- and the Mosquitto open source project provides easy to set fyTimer is statically configured with a NotifyTime value up public servers that accelerate the development phase. in seconds. When the user is typing, a text input area MQTT is a publish/subscribe protocol, where the subscribers changed event is triggered. If the NotifyTimer is not run- are instantaneously notified whenever a publisher generates ning (i.e., NotifyTimer = 0), the algorithm will send a no- a new event. tification packet to the network and start the NotifyTimer. Fig. 8 shows the general architecture of the application The next time the user types, the NotifyTimer will avoid and the test environment. The IM clients are able to com- a new packet being sent to the network. Thus, if the user municate using MQTT through the server that hosts the is continuously typing the algorithm performs only a single MQTT broker. Each user subscribes to her nickname. The notification transmission every NotifyTime. conversation partners send a message by publishing the mes- When the other user receives the typing notification, the sage to the nickname (in publish/subscribe terms, the topic) receiving end of the application will show the receiving user of the recipient. The MQTT broker keeps track of the topic the notification in the screen. For example, we observe that subscriptions. WhatsApp uses approximately a 3 seconds timer for the Since the IM applications tested in section 1 are black NotifyTime. boxes, we cannot compare the results obtained from our pro- totype application against them in a fair way. However, for the interested reader, the basic energy cost of the prototype

102 Dense Android Activity! 60 Android Service! User interface! 40

Test automation! Message queue! 20 0 Typing 0 20 40 60 80 100 120 140 160 180 Notification logic! MQTT client ! Sparse

60 Bundle logic! 40 Mosquitto! 20 MQTT Broker! 0 0 50 100 150 200 250 Radio access Random Network Internet! 60 (3G)! 40 Conversation Message size (characters) 20 partner! 0 0 20 40 60 80 100 120 140 160 180 200 Short Figure 8: Architecture of the IM prototype imple- 60 User 2 mentation and the test environment. 40 User 1 20 0 0 20 40 60 80 100 120 140 application for the same conversation as shown in section 1 Time (seconds) is lower than the rest (18 Joules). 6.2 Test traces and parameter settings Figure 9: Test conversations. This section describes the parameter settings and the con- versations used to study the energy characteristics of the typing notification and the message bundling. parametrised writing speed. For each message of a given Test conversations: Four different test conversations conversation, the characters of the message are written us- are selected for the tests based on the different patterns ob- ing a constant typing speed. The automation logic starts served in section 3. Two conversations (Dense and Sparse) writing in the input text field so that the message is sent in were randomly selected from the dataset, representing inten- its correct timestamp. The time to start writing the mes- sive conversations (short IMIs) and slow-paced ones (longer sage is calculated using the message length and the average IMIs). Random is a synthetic conversation generated by ran- writing speed. domly selecting messages from the dataset not considering For simplicity, the typing speed is set to a constant of the resulting conversation’s dense/sparse characteristics. By 287 ms delay per character. This number was experimen- randomly selecting the messages we aim at obtaining mes- tally obtained by averaging the measured time to write 160 sages of different sizes. The selection follows the following characters (i.e., a SMS) for 6 different people. rules: Message pairs in the resulting conversation have an Computing the transmission energy: While replay- IMI of shorter than 30 s since larger IMIs are uninterest- ing the sequence of messages of the test conversations, the ing from the energy perspective (greater than the typical packet traces are captured in the device of the user 1 using inactivity timers). Moreover, if a selected message appears tcpdump for every test, later used for computing the energy in a sequence of consecutive messages, then the other mes- consumption. Only the traffic of the IM application is al- sages belonging to that sequence are also selected to keep lowed using a firewall to block the rest of the traffic. the realistic consecutive message relations from the dataset. Next, the transmission energy consumption for each test Finally, Short is the same conversation used in the intro- is calculated from the gathered traces using the EnergyBox. duction to compare the different IM applications. It char- EnergyBox has been evaluated against physical energy con- acterises a trace containing short messages, a single con- sumption measurements showing an accuracy of 98% [26]. secutive message, and IMIs between the Sparse and Dense Given the 3G network parameters specified at operator level, conversations. The four categories have different number of EnergyBox derives the 3G states of the UE employing trace- consecutive messages, which is interesting to test the mes- based iterative packet-driven simulation. The RRC state sage bundling. machine is captured by a finite state machine that simu- The test conversations used are shown in Fig. 9. The lates state transitions using the inactivity timers or exceed- duration of the selected conversations is below 250 s. We ing the RLC buffer thresholds. The state machine is forced believe these are representative conversations for IM. Since to go through transitions by an iterative packet-driven sim- our dataset did not delimit conversations (the users were ulation mechanism, which uses the inter-packet interval, the always logged in and no distinction of different conversations size and the direction (uplink/downlink) of the data traffic were made within the chats), we base our reasoning about trace. The energy consumption of a given packet trace for duration on earlier work [30], where 9900 of approximately the given network parameters is calculated by associating 10000 conversations were shorter than 250 s. Note that in the UE-specific power levels with the emulated intervals in Fig. 9 the conversations do not start from 0 since the time each state, and integrating them over time. to write the message is also considered, and the difference The parameters of EnergyBox are set as follows: We set in their length is not important since the results are studied the 3G network settings that correspond to the operator per conversation. TeliaSonera (the operator used for the tests) measured in Writing speed parameters: In order to simulate the our local (experiment) area. The inactivity timers T1 and user typing and automate the tests, the prototype applica- T2 are set to 4.1 s and 5.6 s respectively. The RLC buffer u u tion is instrumented to automatically write characters at a thresholds correspond to: B1 = 1000 and B2 = 294 bytes

103 d d for uplink, and B1 = 515 and B2 = 515 bytes for downlink. savings. The IMI of the consecutive messages is long for the The time to perform the different state transitions are set Sparse conversation, thus the short BundleTime values do to 1.7 s, 0.65 s and 0.435 s for PCH-DCH, FACH-DCH and not allow performing most of the possible bundles. PCH-FACH respectively. The UE power values for the dif- Regarding the Random conversation, the energy savings ferent RRC states are based on earlier measurements: DCH range from 24% to 42% for the different BundleTimes. The = 612 mW, FACH = 416 mW. We set PCH = 0 W in order shortest timer provides again significant energy savings. to quantify only the energy spent for data transmission. There is only one consecutive message in the Short conver- The evaluation methodology and environment are em- sation. No bundle is created since its inter-message interval ployed to analyse the potential energy savings of the message is longer than the BundleTimes, and the message length is bundling and the cost of the typing notification in the next short (i.e., the typing time is short). sections. Bundling results: Tables 2 and 3 show the achieved number of bundles and the messages per bundle for the dif- 7. DOES MESSAGE BUNDLING PAY OFF? ferent BundleTimes using the Dense and Sparse conversa- tions. Possible bundles refers to the number of distinct bun- Next, we quantify the energy savings that message bundling dles where each bundle is a sequence of consecutive messages can provide at the cost of an introduced delay due to aggre- with the maximum length appearing in the conversation. gating messages.

Table 2: Number of bundles and bundles per mes- 7.1 Energy savings and bundling results sage for the Dense conversation. Each test conversation is tested with the following Bundle- Times: 1, 3, 5 and 7 s. The results are normalised to the BundleTime (s) Bundles Messages per bundle base energy consumption with the message bundling dis- 1 3 3, 3, 2 abled (32.63, 26.68, 32.81 and 17.81 Joules for Dense, Sparse, 3 3 4, 3, 2 Random and Short respectively). The results are based on 5 4 5, 3, 3, 2 7 4 5, 3, 3, 2 3 repetitions of each unique test. Additional repetitions are run when large standard deviation is observed. Possible bundles 4 5, 4, 3, 2

Dense Sparse For Dense, the BundleTime of 1 s creates 3 bundles out 1 1 of the 4 possible bundles (i.e., 4 groups of consecutive mes- 0.8 0.8 sages). Only one more message is aggregated for the 3 s

0.6 0.6 BundleTime. All the possible bundles are performed when

0.4 0.4 the BundleTime is increased to 5 s achieving the maximum energy savings. Similar results are obtained for 5 and 7 s, 0.2 0.2 Normalised average energy Normalised average energy i.e., no additional messages are bundled even increasing the 0 0 None 1 3 5 7 None 1 3 5 7 BundleTime. BundleTime (seconds) BundleTime (seconds)

Random Short

1 1 Table 3: Number of bundles and bundles per mes-

0.8 0.8 sage for the Sparse conversation.

0.6 0.6 BundleTime (s) Bundles Messages per bundle 0.4 0.4 1 1 3 0.2 0.2 3 2 2, 3 Normalised average energy Normalised average energy 0 0 5 2 2, 3 None 1 3 5 7 None 1 3 5 7 BundleTime (seconds) BundleTime (seconds) 7 2 2, 3 Possible bundles 3 2, 3, 3

Figure 10: Normalised average energy and standard deviation for message bundling. For Sparse, the BundleTime of 1 s leads to a single bundle out of the 3 possible bundles. The other BundleTimes only Energy savings: Fig. 10 shows the energy savings for create an additional bundle of 2 messages. the different test conversations. The message bundling pro- Protocol overhead: The message bundling also pro- vides energy savings when the algorithm successfully per- vides less TCP/IP header overhead. The number of packets forms at least one bundle. The energy savings vary between sent in the Random conversation was reduced from 40 to 16 the conversations, and thus we describe them separately in when using a BundleTime of 7 s. Thus, the messages are this section. sent in the same packet reducing the TCP/IP overhead. The energy savings due to bundling range from 27% to To sum up, our results show that even a short Bundle- 43% for the Dense test conversation. Since it contains many Time can lead to significant energy savings. The next sec- consecutive messages and these are often close in time (i.e., tion studies the potential drawback of bundling. the user starts writing soon after a previous message is sent), even a short BundleTime of 1 s can achieve 27% savings. 7.2 Message delay The results for Sparse are different. The message bundling Even though the bundle technique is desirable from the achieves 16% energy savings with a BundleTime of 1 s. How- energy perspective, one needs to also consider its negative ever, increasing the timer value does not increase the energy impact on per-message delay. The bundle technique delays

104 each message by a minimum of the BundleTime value. The conversations with sporadic messages should not use long delay of the held messages increases when the user contin- timers. ues typing a consecutive message since the BundleTimer is Finally, the message delay does not always have a nega- restarted every time the user types a character. tive impact on the QoE. When the user is not engaged in an The per-message delay is computed for the Dense and active conversation, the reception of the non-delayed mes- Sparse test conversations for different BundleTimes. The sages or a bundle of delayed messages can be argued to be application is instrumented in order to obtain the delay be- the same. However, from the energy perspective, the latter tween the moment of pressing the sending button and the drastically reduces the energy consumption. time that the message is actually sent to the network. The minimum delay for each BundleTime is the BundleTime it- 8. COST OF TYPING NOTIFICATION self, representing the case that a message was not bundled. The bundled messages are typically delayed more than the Aggressive notification policies can lead to wasting en- minimum, depending on the IMI, the BundleTime and the ergy. This section studies the additional energy cost incurred number of characters in the next message. by the typing notification functionality for the different test conversations. The results are based on 3 repetitions of each unique test. An additional 2 repetitions are run if large stan- Dense Sparse 40 dard deviation is observed in the results. 40 35 For each conversation, we compute the energy consump- 35 30 tion with the typing notification functionality disabled as a 30 25 baseline. We select 3, 5 and 10 s as NotifyTime values for 25 the different tests. These values are representative of the 20 20 parameters used in real typing notification mechanisms. 15 15 Since the focus is on the additional cost, we normalise all 10 10 Message delay (seconds) Message delay (seconds) the values to the average energy consumed by the baseline 5 5 for each conversation (30.65, 26.99, 33.36 and 18.47 Joules 0 0 1357 1357 BundleTime (seconds) BundleTime (seconds) for the Dense, Sparse, Random and Short respectively), i.e., the energy cost with no typing notification. Figure 11: Box plot of the delay experienced by the Dense Sparse different messages for Sparse and Dense conversa- 2.5 2.5 tions. 2 2

1.5 1.5

Fig. 11 shows the delay introduced by bundling in the 1 1 Dense and Sparse conversations. The median delay for the 0.5 0.5 Dense conversation with the BundleTimes 1 and 3 s is the Normalised average energy 0 Normalised average energy 0 BundleTime since most messages are just delayed by the None 3 5 10 None 3 5 10 NotifyTimer (seconds) NotifyTimer (seconds) minimum delay. These are the last messages added to a bundle or the messages that were not bundled. As expected, Random Short 2.5 2.5 the bundled messages experience greater delay. Fig. 11 shows that the maximum delay for the Bundle- 2 2 Times 1 and 3 s is 20 and 30 seconds in the Dense scenario. 1.5 1.5

The maximum delay for 3 s BundleTime increases because 1 1 it bundles an additional message. The more messages in the 0.5 0.5

bundle, the higher is the maximum delay of the first bun- Normalised average energy Normalised average energy 0 0 dled message. However, increasing the BundleTime leads to None 3 5 10 None 3 5 10 NotifyTimer (seconds) NotifyTimer (seconds) a great increase of the per-message delay, especially the mes- sages that are bundled. In Dense, even though the Bundle- Times 5 and 7 s form exactly the same bundles, the messages Figure 12: Normalised average energy and standard experience an extra delay with no extra energy saving. deviation of the typing notification feature for the The results for the Sparse conversation show that the me- different conversations. dian delay is the minimum experienced delay. The long IMI of the Sparse conversation make the bundle technique to cre- Fig. 12 shows that in general the additional energy cost ate a single bundle of 3 messages for the 1 s BundleTime (the of the typing notification feature is large, varying from 1.4 outliers of 6 and 11 s in Fig. 11), and an additional bundle to 2 times the base energy consumption (40-104% more en- of 2 messages for the rest. Thus, the per-message delay is ergy). Since the functionality sends packets whenever the shorter due to the smaller number of bundles than for the user is typing, it keeps the 3G interface in an active state Dense conversation. for almost the whole duration of the test. The results for Comparing the energy savings and the introduced per- each conversation are explained next. message delay, we observe that a short BundleTime of 1 s For the Dense conversation, the energy consumption for leads to significant energy savings while not causing huge the different NotifyTime values is similar. Even though delays. This indicates that simply keeping track of the a longer NotifyTime implies sending fewer notify packets, user typing is enough for a simple bundle policy. Longer when the device enters a high energy RRC state, sending BundleTime values can increase the energy savings for dense more data does not incur higher energy cost. Most of the conversations at the cost of higher delay. However, Sparse energy consumption is due to reseting the inactivity timers

105 by the operator, and thus the message pattern of the con- study can also be made more extensive by creating sets of versation greatly influences the consumed energy. Since the synthetic data as well as employing further real traces, and conversation is dense, the impact of the typing notification confirming the trends observed above. Studying the user is not as high as in the Sparse conversation. interaction changes given the different functionalities (e.g., The additional energy cost for the Sparse conversation is adding bundling or removing typing notification) is left for higher than the Dense one. The Sparse conversation has pe- future works. riods of time where the device is in PCH between message More sophisticated bundling techniques are feasible lever- sendings. However, the typing notification messages greatly aging knowledge of user activity (e.g., the screen is off, the reduce the idle time. Since the 3 and 5 s NotifyTimes are user switched the focus to another application), message shorter than the inactivity timer T2, the T2 timer is reset content parsing (e.g., conversational closings such as “talk before it has expired. When the NotifyTime is 10 s, T2 ex- to you later”), context information (e.g., built-in sensors) pires, which results in higher idle time. However, the energy or presence information from other users. Distinguishing cost is still 64% more than the baseline. The Sparse conver- between periods of sparse and dense conversations is inter- sation is characterised by having slightly longer messages, esting to dynamically activate the bundle technique (e.g., and thus the time to type for the user is longer and more moderately using the typing notification). Considering mul- notify packets per message are sent. timedia messages is also a future direction. The results for the different NotifyTimes for the Random and Short are similar to the Sparse conversation. The Short Acknowledgements and Random conversations contain mostly short messages. When NotifyTime is 10 s, less notify messages are sent since This work was supported by the Swedish national graduate the user spends less time typing the messages. school in computer science (CUGS). The authors wish to To sum up, the typing notification functionality results in thank the support of Ericsson AB, and in particular B-O a high additional cost for an IM application. Even though a Hertz, P¨ar Emanuelsson and Claes Alstr¨omer for provid- longer NotifyTime avoids excessive usage of the 3G interface, ing the measurement kit which has been used in validating the cost is still high. the EnergyBox and facilitating the measurement gathering phase. We would also like to thank the instant messaging users that shared their data, and the anonymous reviewers 9. CONCLUSION AND FUTURE WORK for their many insightful comments and suggestions. When developing energy-efficient solutions for application data transmission, there is a trade-off between adding ap- 10. REFERENCES plication features that perform network transmissions and [1] BBC. Chat app messaging overtakes SMS texts, energy conservation. There is a need to quantify the extra Informa says, accessed 20th February, 2014. energy cost and the perceived functionality benefit from the http://www.bbc.co.uk/news/business-22334338. user side. [2] CNET. LG shares its plans for home appliance The typing notification functionality employed by most evolution, accessed 20th February, 2014. IM applications quantified in our work was shown to have http://www.cnet.com/news/lg-shares-its-plans-for- a tremendous energy cost. According to our results, this home-appliance-evolution/. functionality can increase the energy consumption of an IM [3] The Verge. WhatsApp now has over 400 million application by 40-104% from the basic message exchange monthly users, accessed 20th February, 2014. functionality. Quantifying the cost of a functionality can http://www.theverge.com/2013/12/19/5228656/whatsapp- allow the developers to rethink the need for the functionality now-has-over-400-million-monthly-users. or provide the option to dynamically enable/disable it (e.g., at low battery levels). [4] Tencent. About Tencent, accessed 20th February, When the traffic is directly generated by user interaction, 2014. the network transmissions can easily result in energy waste. http://www.tencent.com/en-us/at/abouttencent.shtml. In our work we collected and studied an IM dataset from [5] P. K. Athivarapu, R. Bhagwan, S. Guha, V. Navda, mobile users to create a better understanding of the user in- R. Ramjee, D. Arora, V. N. Padmanabhan, and puts that trigger network transmissions. Based on our study G. Varghese. Radiojockey: mining program execution with collected real usage data, we observe that IM appli- to optimize cellular radio usage. In Proceedings of the cation users currently tend to write consecutive messages, 18th Annual International Conference on Mobile which increases the active time of the wireless interface. Computing and Networking, Mobicom ’12, pages We show that bundling can be used to reduce transmis- 101–112. ACM, 2012. sions while the user continues typing, and send them at one [6] N. Balasubramanian, A. Balasubramanian, and go. Our results show energy savings up to 43% depending A. Venkataramani. Energy consumption in mobile on the message pattern of the conversation. Given the high phones: a measurement study and implications for percentage of the consecutive messages, this is a promising network applications. In Proceedings of the 9th ACM result. However, using longer timers for the bundle tech- SIGCOMM conference on Internet measurement nique can lead to high delays for some messages. Thus, the conference, IMC ’09, pages 280–293. ACM, 2009. bundle technique can be dynamically activated for sparse [7] Y. W. Chung. An improved energy saving scheme for conversations, or when the receiving end of a chat is offline instant messaging services. In Wireless Advanced, or away. 2011, pages 278–282. IEEE, 2011. Our work can be extended by moving message bundling [8] Y. W. Chung. Investigation of energy consumption of to the server side, which appears very interesting for group mobile station for instant messaging services. In chats that usually have denser conversations. The current Proceedings of the 2011 Tenth International

106 Symposium on Autonomous Decentralized Systems, [21] F. Qian, Z. Wang, A. Gerber, Z. Mao, S. Sen, and pages 343–346, IEEE. 2011. O. Spatscheck. Profiling resource usage for mobile [9] S. Deng and H. Balakrishnan. Traffic-aware techniques applications: A cross-layer approach. In Proceedings of to reduce 3G/LTE wireless energy consumption. In the 9th International Conference on Mobile Systems, Proceedings of the 8th International Conference on Applications, and Services, MobiSys ’11, pages Emerging Networking Experiments and Technologies, 321–334. ACM, 2011. CoNEXT ’12, pages 181–192. ACM, 2012. [22] S. Schrittwieser, P. Fruhwirt,¨ P. Kieseberg, [10] R. E. Grinter, L. Palen, and M. Eldridge. Chatting M. Leithner, M. Mulazzani, M. Huber, and E. R. with teenagers: Considering the place of chat Weippl. Guess who’s texting you? evaluating the technologies in teen life. ACM Transactions on security of smartphone messaging applications. In 19th Computer-Human Interaction (TOCHI), Annual Network and Distributed System Security 13(4):423–447, Dec. 2006. Symposium (NDSS), ISOC. 2012. [11] H. Holma and A. Toskala. WCDMA for UMTS: HSPA [23] M. Z. Shafiq, L. Ji, A. X. Liu, and J. Wang. Evolution and LTE. Wiley Online Library: Books. Characterizing and modeling internet traffic dynamics John Wiley & Sons, 2010. of cellular devices. In Proceedings of the ACM [12] J. Huang, F. Qian, Z. M. Mao, S. Sen, and SIGMETRICS Joint International Conference on O. Spatscheck. Screen-off traffic characterization and Measurement and Modeling of Computer Systems, optimization in 3G/4G networks. In Proceedings of the pages 305–316. ACM, 2011. 2012 ACM Internet Measurement Conference, IMC [24] N. Vallina-Rodriguez and J. Crowcroft. Energy ’12, pages 357–364. ACM, 2012. management techniques in modern mobile handsets. [13] E. Isaacs, A. Walendowski, S. Whittaker, D. J. Communications Surveys Tutorials, IEEE, Schiano, and C. Kamm. The character, functions, and 15(1):179–198, 2013. styles of instant messaging in the workplace. In [25] E. J. Vergara and S. Nadjm-Tehrani. Energy-aware Proceedings of the 2002 ACM Conference on cross-layer burst buffering for wireless communication. Computer Supported Cooperative Work, pages 11–20. In Proceedings of the 3rd International Conference on ACM, 2002. Future Energy Systems: Where Energy, Computing [14] H. A. Lagar-Cavilla, K. Joshi, A. Varshavsky, and Communication Meet, e-Energy ’12. ACM, 2012. J. Bickford, and D. Parra. Traffic backfilling: [26] E. J. Vergara, S. Nadjm-Tehrani, and M. Prihodko. subsidizing lunch for delay-tolerant applications in Energybox: Disclosing the wireless transmission UMTS networks. SIGOPS Operating Systems Review, energy cost for mobile devices. Sustainable Computing: 45(3):77–81, ACM. Jan. 2012. Informatics and Systems, Elsevier. Accepted for [15] J. Leskovec and E. Horvitz. Planetary-scale views on a publication. 2014. large instant-messaging network. In Proceedings of the [27] D. Wang, J. McNair, and A. George. A 17th International Conference on , smart-NIC-based power-proxy solution for reduced WWW ’08, pages 915–924. ACM, 2008. power consumption during instant messaging. In [16] H. Liu, Y. Zhang, and Y. Zhou. Tailtheft: leveraging Green Technologies Conference, IEEE, pages 1–10, the wasted time for saving energy in cellular 2010. communications. In Proceedings of the Sixth [28] C. Wilke, S. Richly, S. G¨otz, C. Piechnick, and International Workshop on MobiArch, pages 31–36. U. Aßmann. Energy Consumption and Efficiency in ACM, 2011. Mobile Applications: A User Feedback Study. In [17] L.-S. Meng, D.-S. Shiu, P.-C. Yeh, K.-C. Chen, and International Conference on Green Computing and H.-Y. Lo. Low power consumption solutions for mobile Communications, pages 134–141. IEEE, 2013. instant messaging. IEEE Transactions on Mobile [29] L. Xiang, J. Luo, and C. Rosenberg. Compressed data Computing, 11(6):896–904, June 2012. aggregation: Energy-efficient and high-fidelity data [18] A. Pathak, Y. C. Hu, and M. Zhang. Where is the collection. IEEE/ACM Transactions on Networking, energy spent inside my app?: Fine grained energy 21(6):1722–1735, 2013. accounting on smartphones with Eprof. In Proceedings [30] Z. Xiao, L. Guo, and J. Tracey. Understanding instant of the 7th ACM European Conference on Computer messaging traffic characteristics. In Proceedings of the Systems, EuroSys ’12, pages 29–42, 2012. 27th International Conference on Distributed [19] F. Qian, J. Huang, J. Erman, Z. M. Mao, S. Sen, and Computing Systems, pages 51–59. IEEE, 2007. O. Spatscheck. How to reduce smartphone traffic volume by 30%? In Proceedings of the 14th International Conference on Passive and Active Measurement, IMC ’13, pages 42–52. Springer-Verlag, 2013. [20] F. Qian, Z. Wang, Y. Gao, J. Huang, A. Gerber, Z. Mao, S. Sen, and O. Spatscheck. Periodic transfers in mobile applications: network-wide origin, impact, and optimization. In Proceedings of the 21st International Conference on World Wide Web, WWW ’12, pages 51–60. ACM, 2012.

107