Every Rose Has Its Thorn: Censorship and Surveillance on Social Video Platforms in China
Jeffrey Knockel1,2, Masashi Crete-Nishihata2, Jason Q. Ng2, Adam Senft2, and Jedidiah R. Crandall1
1Dept. of Computer Science, University of New Mexico 2Citizen Lab, Munk School of Global Affairs, University of Toronto
Abstract which suggests there may be general directives provided by the government on what content to censor, but the Social media companies operating in China face a com- companies have a degree of flexibility for how they im- plex array of regulations and are liable for content posted plement it [16, 25, 31]. Further exploring these obser- to their platforms. Through reverse engineering we pro- vations is challenging due to methodological constraints. vide a view into how keyword censorship operates on four The majority of studies on Chinese social media rely on popular social video platforms in China: YY, 9158, Sina sample testing in which researchers develop a set of con- Show, and GuaGua. We also find keyword surveillance tent suspected to be blocked by a platform, send the sam- capabilities on YY. Our findings show inconsistencies in ple to the platform, and record the results. This approach the implementation of censorship and the keyword lists introduces inherent bias as results are only as accurate as used to trigger censorship events between the platforms the overlap between the sample and the actual content fil- we analyzed. We reveal a range of targeted content in- tered. Another approach is to observe changes to a system cluding criticism of the government and collective action. (such as content deletion) over time. Studies using this These results develop a deeper understanding of Chinese method are typically limited to snapshots within a spe- social media via comparative analysis across platforms, cific period, which constrains longitudinal analysis. Ap- and provide evidence that there is no monolithic set of plications that implement censorship and surveillance on rules that govern how information controls are imple- the client-side (i.e., by the application itself rather than on mented in China. a remote server) present a unique research opportunity. Reverse engineering applications can reveal keyword lists 1 Introduction used to trigger censorship and surveillance. These lists are unbiased samples that provide comprehensive visi- The Chinese Internet is a complex ecosystem that in- bility into technical implementation and target content. cludes multiple layers of technical and regulatory infor- This study provides a broad look into keyword surveil- mation controls that are affected by actors with different lance and censorship across social video platforms positions of influence and responsibility. Extensive work (SVPs), a popular class of applications in China. SVPs has been done on China’s national-level Internet filtering combine real-time video streaming and social network- system, but this system is only one of many layers of in- ing features that enable users to broadcast content and formation control in the country. create interactive groups. One of the most popular uses Gaining a wider understanding of censorship and is broadcasting karaoke performances. SVPs are primar- surveillance in China requires analysis of its Internet plat- ily monetized through the sale of virtual goods (such as form developers and companies. These companies op- virtual roses) that users give to performers during broad- erate in a highly constrained regulatory environment in casts. While musical performances account for the ma- which they are responsible for the content on their ser- jority of revenues, SVPs are expanding to gaming, edu- vices and subject to fines and loss of operating licenses cation, financial analysis, and online dating applications. if they are found in violation. This governance model ef- Through reverse engineering, we identify client-side fectively pushes responsibility for information control to keyword censorship in four of the most popular SVPs: the private sector. YY, 9158, Sina Show, and GuaGua. In the case of YY Previous work has shown inconsistencies in how dif- we also find keyword surveillance capabilities. Our anal- ferent Chinese Internet companies conduct censorship, ysis reveals a dataset of 17,547 unique keywords, which
1 Company Product Reg. Users MAUs of Sina Show. Tian Ge reports user numbers as aggre- YY Inc. YY 861.4 mn. 117.4 mn. gates across its platforms and in 2014 had 245 million 9158 registered uses and 14.4 million MAUs [3, 4]. In July Tian Ge 245.0 mn. 14.4 mn. Sina Show 2014, Tian Ge went public on the Hong Kong Stock Ex- Jinhua not change. GuaGua 70 mn. Changfeng available Jinhua Changfeng Information Technology Co., Ltd., is a privately held company based in Zhejiang Province, Table 1: Social Video Users by Platform China that provides the GuaGua platform, which as of 2013 had 70 million registered users [9]. See Table 1 for a breakdown of user bases across SVPs. to our knowledge is the largest unbiased collection of cen- sorship keywords currently available. 1 Our main findings 2.1 Legal and Regulatory Environment in are as follows: China Inconsistencies in targeted content and implemen- The Communist Party of China (CPC) attempts to bal- tation between platforms: Comparing our SVP dataset ance the growth of information and communication tech- to previously collected chat client censorship keyword nologies in the country and limits on speech that can lists [16] allows for the first comparison of unbiased key- threaten its power [26]. In 2010 China’s State Council word samples across different industry segments. Our Information Office published what is considered the first analysis reveals limited list overlap between companies, government issued policy document on the Internet. It which substantiates previous findings that suggest com- includes a list of prohibited topics: panies are only given general directives from authorities and have a degree of flexibility in the implementation. endangering state security, divulging state Range of targeted content including criticism of the secrets, subverting state power and jeopar- government and collective action: While there is lim- dizing national unification; damaging state ited direct overlap in unique keywords, across lists we see honor and interests; instigating ethnic hatred or trends in the topics that are targeted including social is- discrimination and jeopardizing ethnic unity; sues, criticism of the government, and collective action. jeopardizing state religious policy, propagating These findings serve as a counterpoint to previous work heretical or superstitious ideas; spreading ru- from King et al. [21, 22] who posit that content related mors, disrupting social order and stability; dis- to collective action is heavily censored on Chinese social seminating obscenity, pornography, gambling, media while content critical of the government is often violence, brutality and terror or abetting crime; allowed to persist. humiliating or slandering others, trespassing Our findings provide strong evidence that there isno on the lawful rights and interests of others; and monolithic set of rules governing how information con- other contents forbidden by laws and adminis- trols are implemented in China and that developing holis- trative regulations [8]. tic understandings of the Chinese Internet requires com- Companies are held liable for content on their plat- parative analysis across platforms. forms and are expected to invest in staff and technol- ogy for ensuring compliance with government regula- 2 Background tions. Failure to comply with regulations can lead to fines or revocation of operating licenses. Complicat- The most popular SVPs in China include YY, Sina Show, ing matters is the vague language used in Chinese reg- 9158, and GuaGua. YY is developed by YY Inc. based ulatory documents. Terms like “disrupting social order in Guangzhou, China, and is the largest platform in terms and stability” are not clearly defined and punishments of user population. As of December 2014, YY had are meted out seemingly arbitrarily, thus pushing com- 861.4 million registered users and 117.4 million average panies and users to both over-censor and self-censor—a monthly active users (MAUs) [6]. In November 2012, phenomenon coined by Perry Link as “the anaconda in YY Inc. announced an initial public offering on the Nas- the chandelier” [24]. daq stock market. It is currently the only Chinese SVP In public filings for YY and Tian Ge both companies company to be traded on the US stock market. underline the risk that their businesses face from poten- Tian Ge Interactive Holdings Limited based in tial legal sanctions being brought against them for hosting Hangzhou, China owns and operates two SVPs: 9158 and prohibited content [2, 6]. The companies also highlight Sina Show. In 2010, Sina Corporation invested 10 mil- the risk of being affected by government campaigns such lion dollars (representing a 25% stake) in Tian Ge and as “Clean the Web 2014,” which was an government ef- provided the company a sole license for the operation of fort to crack down on the creation and dissemination of
2 pornographic content online. During this campaign, Sina then sent to the content monitoring team for further re- Corporation received notices regarding prohibited con- view. The company also generally describes audio mon- tent on its platforms and was subsequently fined 5.1 mil- itoring and keyword filtering systems. In addition, it pro- lion RMB, had licenses temporarily revoked, and saw its vides the same level of access that its content monitoring stock price drop as a result. This campaign demonstrates team has to the Jinhua City Municipal Public Security the dynamic nature of Internet regulations in China, and Bureau to allow the authorities means to “monitor and shows companies are subject to unpredictable enforce- supervise the activities” on the platform [2]. ment, which can impact their bottom line. As GuaGua is a private company less information Unlike most other social media platforms in China, is available on its internal operations. In a 2013 in- SVPs have an added dimension of sharing revenues with terview, co-founder Dong Guanjie claims the platform performers. This business model and the general SVP has a content management team of over 100 staff [9]. user experience encourages performers to keep audiences GuaGua’s TOS explains the company performs auto- engaged and spending on virtual goods. The popularity mated and manual inspection of content and deletes any of SVPs, the diversity of real-time media content, and the infringements [1]. Similar to YY and Tian Ge emphasis virtual goods business model puts these platforms under is placed on incentives for user self-regulation and finan- particular pressure to monitor and manage user activity. cial penalties for violations. 2.2 Content Monitoring and Censorship on Social Video Platforms 3 Related Work To comply with China’s laws and regulations SVPs man- The majority of research on Internet censorship in China age content through a combination of terms of service focuses on the Great Firewall of China, its national-level (TOS), automated content monitoring and filtering sys- Internet filtering system [12, 15, 17, 19, 29, 34, 35, 37]. tems, and dedicated review teams. In comparison to this literature, the number of studies on YY has an extensive TOS that includes descriptions of surveillance and censorship on Chinese social media is prohibited content and a five-level system for penalties limited. that range from freezing the account from 7 days (level The microblogging service, Sina Weibo, has been the 1), 30 days (level 2), 120 days (level 3), 360 days (level focus of a number of studies that show the dynamic na- 4), or permanently (level 5). Serious violations that war- ture of content filtering on the platform. Bamman, et rant a level 4 or 5 response include publishing pornogra- al. [13] conducted statistical analysis of deleted Weibo phy; publishing content that endangers national security posts and found that posts with sensitive words and from or undermines national unity, social stability, or national certain geographic locations (e.g., Tibet and Qinghai) religious policy. Offences that carry lower level punish- have a higher deletion rate. Zhu, et al. [36] measured ments include vulgar jokes, verbally abusing other users, censorship on Weibo and found that retroactive post dele- and copyright infringement [6]. Performers are expected tions occur within minutes and the censors use a variety to obey an additional list of regulations that include the of automated tools. The University of Hong Kong has same high level prohibitions and other specific guidelines developed WeiboScope, a data collection and visualiza- on inappropriate attire and performance material. Failure tions system for tracking censorship on Weibo [5]. Fu to comply can result in fines and account suspensions [7]. et al. [33] use this system to show that real name regis- To enforce these TOS, YY has a team within the tration policies on Weibo may have caused some users data security department that maintains “24-hour surveil- to self-censor. Ng [27] identified numerous ways that a lance” on content and is supported by a system that peri- Weibo message could be censored or held in review both odically “sweeps” the platform for offensive content and before and after being posted, confirming the usage of “automatically” filters keywords. The company also de- both automated and manual review processes. scribes a voice monitor system that provides “various King, et al. [21] collected posts from 1,382 Chinese alerts on sensitive words or abnormal activities of users, social media Web sites and, through statistical analy- channels, or groups” [6]. sis comparing censored and uncensored posts, contend Tian Ge has a similar combination of controls. It em- that censorship focused on content that represented, rein- ploys a team of 74 content monitors who identify TOS vi- forced, or encouraged collective action. olations and enforce internal policies. In public filings the Previous work has found inconsistencies in censored company describes an image processing system used to content and the technical implementation of censorship detect skin tone and facial features to flag nudity or “sexu- across services. MacKinnon [25] examined 15 different ally suggestive partial nudity.” Screenshots of video chat Chinese blog providers and found significant variation in rooms are randomly captured every 1-3 minutes and pro- the extent and implementation of content censorship. Vil- cessed through the detection system. Flagged content is leneuve [31] analyzed keyword filtering in search engines
3 localized for the Chinese market and found a similar lack and, if performed, the second comparison is less so, we of overlap in censored keywords. These studies suggest conclude that the censorship is server-side. If the appli- that content filtering across platforms is highly decentral- cation displays a warning message upon entering a cen- ized and affords companies a level of flexibility in imple- sored message, we also ensure that the warning message menting controls. does not display when the application has no access to the The previously outlined studies relied on testing sam- Internet. We leave it as future work to analyze the server- ples [25, 27, 31] or observing changes [13, 21, 36] (e.g. side censorship implementations used by these platforms. deletions) in a subset of content over a fixed period. Other 4.1 YY Censorship and Surveillance work focused on client-side implementations of censor- ship and surveillance have extracted unbiased keyword YY 7.1 downloads three different keywords lists with the lists used to trigger these functions and analyzed changes following names: Finance, Normal, and High. over time. Villeneuve [32] revealed keyword surveillance The Finance keyword list is downloaded from http: in TOM-Skype by discovering chat logs uploaded by the //do.yy.duowan.com/financekwordlist. These keywords client through an insecure publicly accessible Web server are downloaded in plain text in UTF8-encoded XML. hosted in China. Knockel, et al. [23] reverse engineered Keywords in this list are related to phishing scams. When multiple versions of TOM-Skype, decrypted censorship a user receives a keyword from the list the following and surveillance keyword lists and reported on updates warning message is displayed in the chat window: “YY安 over a one month period. In Crandall et al. [16], cen- 全提示：聊天中若有涉及财产的操作，请一定要先 sorship keyword lists were obtained from Sina UC (an- 核实好友身份，谨防受骗 !” (YY Security Tip: This other chat client used in China) and compared to the chat seems to involve managing assets; please be sure to TOM-Skype lists, tracking each for a period of over 22 verify the identity of a friend to avoid being cheated!). months, categorizing the keywords into granular con- The Normal keyword list is downloaded from http: tent categories, and correlating list updates with current //do.yy.duowan.com/NormalKWordlist.txt as a base64- events. Similar to [25, 31] this study revealed inconsis- encoded list of UTF16-encoded keywords each separated tencies in the implementation of keyword censorship be- by a carriage return followed by a line feed. If an out- tween the programs and found only 3% overlap in unique going or incoming message contains a keyword from this keywords within the total dataset of 4,256 keywords. list, those keywords are asterisked out in the chat window. Hardy [20] reversed engineered LINE, a mobile chat In the case of an outgoing message, those keywords are client developed by a Japanese company and marketed to also asterisked out in the message sent over the network. countries around the world including China, and revealed The High keyword list is downloaded from http://do. regionally-based keyword filtering implemented on the yy.duowan.com/HighKWordlist.txt. Like the Normal client-side that is enabled for users with accounts reg- list, it is a base64-encoded list of UTF16-encoded key- istered to mainland China phone numbers. Analysis of words each separated by a carriage return followed by a these keyword lists revealed limited overlap with TOM- line feed. If an outgoing message contains a keyword Skype and Sina UC [18]. from this list, that message is silently filtered. If an in- coming message contains a keyword from this list, it ap- pears in the chat window as a blank message. 4 Technical Analysis 4.1.1 YY Surveillance In this section, we describe the technical implementations The keywords from both the Normal and High lists are of keyword censorship and, if present, keyword surveil- also used to trigger surveillance. When attempting to lance in each of the SVPs we analyzed. send a message containing keywords from either of these We found that three Chinese SVPs not described in lists, a surveillance message is sent via an HTTP GET this paper, VV (51vv.com), Sixroom (6.cn), and BoBo request to a URL of the form: (bobo.com), perform keyword censorship on the server- side. We determine that an application censors on the http://sere.hiido.com/do.action? server-side by first ensuring that there are no obvious id=
4 of the sender or receiver of the triggering message.
5 1.0 YY Normal List Keywords Unique YY High Sina Show UC 0.9 YY Finance 48 18 Sina Show Builtin GuaGua 0.8 YY Normal 20 20 9158 Nicknames Skype 5.0 Builtin YY High 13,482 13,242 Skype 3.6 Builtin 0.7 LINE 9158 Nick 65 59 Skype 5.1 Surveil Skype 5.0 DL 0.6 9158 Chat 318 318 Skype 5.5 Surveil Skype 4.0 DL 0.5 Sina Show SinaShow.exe 1,224 910 Skype 3.6 DL Sina UC DL 3 0.4 Sina Show UCClient.dll 910 910 Sina UC Builtin 3 Sina UC DL 1 Sina Show Downloaded 3,711 3,206 Sina UC Builtin 1 0.3 Sina UC DL 2 GuaGua 58 58 Sina UC Builtin 2 0.2 Sina Show DL Sina UC Builtin 5 Sina UC Builtin 4 0.1 Table 2: List size (May 17, 2015) Sina UC DL 5 Sina UC DL 4 0.0 1.0 YY Normal LINE YY High YY High Sina UC DL 5 0.9 GuaGua
Sina UC DL 2 YY Normal Skype 3.6 DL Skype 4.0 DL Skype 5.0 DL
Sina UC Builtin 2 Sina UC DL 4 Sina UC DL 5 Sina UC DL 2 Sina UC DL 1 Sina UC DL 3 Sina Show DL
0.8 Sina Show UC Sina UC DL 1 Skype 3.6 Builtin Skype 5.0 Builtin 9158 Nicknames Sina UC Builtin 4 Sina UC Builtin 5 Sina UC Builtin 2 Sina UC Builtin 1 Sina UC Builtin 3 Sina Show Builtin Sina UC Builtin 1 Skype 5.5 Surveil Skype 5.1 Surveil Sina Show DL 0.7 GuaGua 9158 Nicknames 0.6 Skype 5.0. Builtin Figure 2: Keyword lists clustered by similarity(x, y) = Skype 3.6 Builtin LINE 0.5 max(% of x in y, % of y in x) Sina UC Builtin 5 Sina UC Builtin 4 0.4 Sina UC Builtin 3 Skype 5.1 Surveil Skype 5.0 DL 0.3 Skype 5.5 Surveil near exact duplicate of a 2004-era list built into Sina UC Skype 4.0 DL Skype 3.6 DL 0.2 that Sina Show’s built-in lists build upon. The only dif- Sina Show UC Sina Show Builtin 0.1 ference in the GuaGua list is the addition of a single key- Sina UC DL 4 Sina UC DL 3 word. Both of the founders of GuaGua formerly worked 0.0 on audio chat software at Langma UC (acquired by Sina
LINE Corporation in 2004 to become Sina UC) and Sina [9]. YY High GuaGua
YY Normal This employment history may explain why the GuaGua Skype 3.6 DL Skype 4.0 DL Skype 5.0 DL Sina UC DL 3 Sina UC DL 4 Sina UC DL 1 Sina UC DL 2 Sina UC DL 5 Sina Show DL Sina Show UC and Sina lists are so similar. Skype 3.6 Builtin Skype 5.0 Builtin 9158 Nicknames Sina UC Builtin 3 Sina UC Builtin 4 Sina UC Builtin 5 Sina UC Builtin 1 Sina UC Builtin 2 Sina Show Builtin Skype 5.5 Surveil Skype 5.1 Surveil 5.2 Keyword Content Analysis
. Figure 1: Keyword lists clustered by Jaccard similarity We used a combination of machine and human transla- tion to translate the keywords to English and analyzed the 5.1 Similarity Comparison Across Key- context behind each one. Based on these translations and word Lists contextual information three researchers coded each key- word into one of 80 content categories grouped under six In Figure 1, using the Nearest Point Algorithm, we hierar- general themes based on a code book developed in [16]. chically cluster SVP keyword lists along with the TOM- We performed interrater reliability checks throughout the Skype and Sina UC lists [16] and the latest list from categorization process. LINE [18] by Jaccard similarity coefficienti.e. ( , the size The six themes with example categories are: So- of the intersection of two sets divided by the size of their cial (prurient interests, illicit goods, gambling), Po- union). Using this method, we find very little similar- litical (Communist Party of China, religious move- ity between lists, and when lists are similar they are lists ments, ethnic groups), People (government officials, dis- within the same company. sidents), Events (scheduled events, recurring events, cur- In Figure 2, we cluster the same keyword lists using a rent events), Technology (general technical terms, URLs, different similarity metric. We compute list x’s similar- references to applications), and Miscellaneous (terms ity to y as max(% of x in y, % of y in x). The intuition without clear context). behind this metric is that it would tease out lists that in- Our content analysis is on the 17,143 out of 17,547 herit from other lists. Although using this method we see keywords currently used to trigger censorship or surveil- more lists similar to each other within companies, lists lance events. This subset includes 7,371 URLs and from different companies remain mostly dissimilar with URL fragments (such as www, http, .B32.c). These one exception: GuaGua is similar to many Sina Show URLs cover a range of websites including phishing pages, lists. Closer inspection reveals that the GuaGua list is a pornography, independent media, and competing SVPs.
6 content following attacks in the Xinjiang region. Thirty Political Chinese Internet companies signed a “letter of commit-
Social ment” to block such content [11]. People: The People theme includes five categories: People CPC officials, relatives and associates of CPC officials, YY dissidents, victims of crime, and names without clear Sina Show Events context or identities. All four SVPs included reference 9158 毛泽 GuaGua to specific CPC officials including past leaders, “ Tech 东” (Máo Zédōng), and current leaders, “习近平” (Xí Jìnpíng). Keywords often include varieties of homonyms Misc for referring to leaders (e.g.,“习尽平”, xí jǐn píng) and nick names such as Steamed Bun Xi (“习包子”), which 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 refers to a photo that circulated online of Xi Jingping or- Figure 3: Breakdown of list source by theme dering lunch at a steamed bun shop that was subsequently criticized as a political show [10]. Events: The Event theme includes 23 specific events, In the content analysis that follows we separate out URLs 14 of which are related to protests and political mobiliza- and report on 9,772 keywords. tions. Other event types include political forums (e.g., Figure 3 shows a breakdown of keywords by theme CPC National Party Congress), and rumors around inci- across the four platforms (normalized by total number of dents (e.g., the disappearance of Malaysia Airlines Flight keywords in each SVP). Keywords related to the social, 370). GuaGua had no references to events and 9158 only political, and people themes are the most common across referenced the Bo Xilai scandal and the 2009 Urumqi ri- all four platforms, which shows that despite the lack of ots. Sina Show included references to 11 events. YY overlap in unique keywords between the platforms, they referenced 18 events and had the greatest number of appear to be generally concerned about similar topics. event related keywords (2,535). Over 90% of YY’s event Social: The Social theme is divided into three cate- related keywords were references to the June 4 1989 gories: gambling and lottery (e.g., online casinos, lot- Tiananmen Square Massacre (this category accounted for tery games), illicit goods and services (e.g., narcotics, 32% of YY lists overall). We identify 8 events that are firearms, counterfeit products), and prurient interests not present in the TOM-Skype / Sina UC dataset [16] in- (e.g., sexual interests, pornography, prostitution). All cluding recent events such as the 2014 Occupy Central four SVPs included content related to prurient interests, protests in Hong Kong, the 2014 Asia-Pacific Economic whereas only 9158, Sina Show, and YY had content from Cooperation Forum, and the 2014 terrorist attack in the the other two categories. Chinese city of Kunming. The addition of these recent Political: The Political theme includes the widest events suggests sensitive events continue to be catalysts range of content with 37 categories related to issues in- for censorship as seen in [16]. cluding the CPC, religious movements, ethnic minori- Technology: The technology theme has nine cate- ties, and terrorism. Content related to the CPC covers gories including a range of identifiers (phone numbers, both general references to the party and criticism of it. QQ numbers and emails), which we suspect are primarily The only category found on all four platforms was Falun related to scams and illicit services. In other cases iden- Gong (also the only political category on GuaGua’s list). tifiers are clearly related to political issues (e.g., emails Notably 9158, Sina Show, and YY include keywords re- related to Falun Gong media websites). Other categories lated to Uyghur issues, which account for the largest per- include generic technical terms like Website (“网址”), centage of keywords within the political theme for Sina and references to software used in China. Interestingly, Show (45%) and YY (25%). YY High lists contain 0.5% three of the SVP keyword lists (YY, Sina Show, 9158) Uyghur keywords in Arabic script that include references include references to other competing SVPs, which may to terrorism and Islam. For example: be attempts to prevent users from being lured away from .the provider’s platform ﭘﺎﺭﺗﻠﯩﻘﯘﭺ ﻳﺎﺳﺎﺵ ﺩﻩﺭﺳﻠﯩﻜﻰ which is a Uyghur phrase that translates to “instructions to create explosives.” 5.3 Keyword List Updates Previously collected keyword lists do not include the Hourly data collection for the clients began on the follow- Uyghur language or Arabic script [16, 18, 20]. The in- ing days: YY on February 7, 2015; 9158 on February 24, creased focus on Uyghur related content relative to pre- 2015; and Sina Show on March 11, 2015. (GuaGua does viously available keyword lists may have been motivated not download updates to its keyword list). We collected by a June 2014 government campaign to censor terrorist data through May 17, 2015. During this time, we saw 3
7 updates to YY Normal, 21 updates to YY High, and 8 up- focused on take down of already published content in blog dates to 9158 Chat; however, we saw no updates to Sina sites and bulletin board systems. Moreover, the study re- Show Downloaded, although its HTTP last-modified date ports results as aggregates across the 1,382 sites that were would suggest that it was last updated February 9, 2015. analyzed, which may suggest greater uniformity than ex- Two major updates to YY Normal occurred when ists in practice. By contrast reverse engineering client- names of Christian Chinese songs were added on April side implementations provides an exhaustive view into 23 and subsequently removed on April 30. The effort to the exact content being targeted and the technical mech- block this content may have been in response to contro- anisms that are utilized. versy over recent church demolitions [30]. Although reli- gious songs are not explicitly prohibited by the YY TOS, Our keyword content analysis brings the generalizabil- et al. it prohibits users from violating “Chinese religious pol- ity of King ’s theoretical arguments into question. icy” [7]. Across the SVPs we studied and previous work on chat Updates to YY High were primarily in response to on- clients [16] we see keywords related to general discussion line scams. One update on February 9, however, was the and criticism of the government and collective action. addition of “bobo.com”, a link to a competing SVP. For example, all four SVPs include references to past and Updates to 9158 Chat included URLs (dropbox.com, present CPC members. Apart from ”Falun Gong” and mediafire.com) and keywords related to Islamic terror- ”prurient interests” this is the only category shared by all ism, and names of government officials. One official周 “ SVPs. We observe criticism of the CPC that can be inter- 永康” (Zhou Yongkang) was added on May 6. On April preted as collective action such as the Tuidang movement, 3, Zhou was charged with multiple corruption crimes, which urges withdrawal from the party. Keyword lists e.g. 共产 which led to a high profile trial in China [14]. also include general references to the CPC ( ,“ 党”, Communist Party) and derogatory phrases (e.g.,“共 匪”, Communist gangsters). Inclusion of such keywords 6 Discussion and Conclusion would limit both general and critical discussion of the government. In this section we discuss the implications of our findings and how they may inform future work. In the event theme we observe a high number of ref- Inconsistencies in the content and implementation erences to incidents related to collective action, but also in keyword lists across companies and platforms: those without a clear link. For example, references to Previous studies that found inconsistencies in client- the Asia-Pacific Economic Cooperation Forum include side keyword filtering (and in some instances surveil- open-ended keywords (e.g., “APEC”) and specific crit- lance) [16, 20] were limited to a small number of applica- icisms such as “一 开 盛 会 就 无 霾” (once the sum- tions that were not very representative in terms of usage mit starts, there is no haze)—a line from a poem circu- numbers. We substantiate these findings by analyzing ex- lated on social media criticizing the CPC for attempting haustive keyword lists across four of the top applications to clean up pollution in anticipation of foreign delegates in an industry segment. We found very little consistency arriving for APEC 2014 in Beijing, but not for citizens in the keywords censored by different applications (with otherwise. the exception of GuaGua, whose founders had former em- Thus, we see both collective action and government ployment with Sina). This finding is consistent with the- criticism clearly reflected in our unbiased keyword lists. ories [24, 26] that companies are under vague pressure While King et al. observed tolerance of government crit- from the government to perform censorship and surveil- icism on the platforms they studied, our analysis conclu- lance, leaving companies to decide how to implement it. sively shows that SVPs and chat clients specifically target Targeted content relates to government criticism this kind of speech. and collective action: King et al. [21, 22] acknowl- edge that social media sites in China have a flexible set This paper provides a mapping of information controls of technical options for implementing censorship. How- across industry segments and raises questions for the re- ever, they conclude that these diverse tactics lead to a search community. Diversity of tactics in implementing mostly uniform outcome—content related to collective censorship undoubtedly lead to a diversity in what con- action (both pro- and anti-government) are heavily cen- tent is ultimately restricted. Furthermore, new govern- sored while “vitriolic blog posts about even the top Chi- ment initiatives like “Clean the Web 2014” and the on- nese leaders” are often allowed to persist in social media. going anti-rumor campaign [28] may quickly render out- This theory does not appear to align with the general reg- moded previously confirmed theories. We thus offer a ulations pushed to companies (e.g., the prohibited topics cautious note about applying any comprehensive theory listed in [8]), relies on sampling methods that may not about an ecosystem as varied and fast changing as the comprehensively capture censored content, and is solely Chinese Internet.
8 Acknowledgments internet-xinjiang-idUSKBN0EV0TP20140620, June 2014. We are grateful to Jakub Dalek for research assistance and Mamatjan Juma, Greg Fay, and Zubayra Shamseden [12] Towards a comprehensive picture of the great fire- for translation. Special thanks to Greg Wiseman for de- wall’s dns censorship. In 4th USENIX Workshop velopment work on the project website (https://china- on Free and Open Communications on the Internet chats.net). Jeffrey Knockel and Jason Q. Ng were sup- (FOCI 14) (San Diego, CA, Aug. 2014), USENIX ported by the Open Technology Fund Information Con- Association. trols Fellowship Program. The research of the Citizen Lab was supported by the John D. and Catherine T. [13] Bamman, D., O’Connor, B., and Smith, N. A. MacArthur Foundation. Censorship and deletion practices in Chinese social media. First Monday 17, 3 (2012). References [14] Branigan, T. Chinese former security tsar Zhou Yongkang charged in corruption case. Available [1] GuaGua, 法 律 声 明. Available at http://www. at http://www.theguardian.com/world/2015/apr/ guagua.cn/other/1907.html. 03/china-security-tsar-zhou-yongkang-charged- corruption., April 2015. [2] Tian Ge Interactive Holdings, Global Offer- ing. Available at http://www.tiange.com/Upload/ [15] Clayton, R., Murdoch, S. J., and Watson, R. Pigeon-Cover-IPO-ENG-2d.pdf. N. M. Ignoring the Great Firewall of China. In 6th Workshop on Privacy Enhancing Technologies [3] Tian Ge Interactive Holdings, Interim Report 2014. (2006). Available at http://www.tiange.com/Upload/e101. pdf. [16] Crandall, J., Crete-Nishihata, M., Knockel, J., McKune, S., Senft, A., Tseng, D., and Wiseman, [4] Tian Ge Interactive Holdings, Tian Ge Announces G. Chat program censorship and surveillance in 2014 Third Quarter and Interim Results. Avail- China: Tracking TOM-Skype and Sina UC. First able at http://www.tiange.com/Upload/TIAN% Monday 18, 7 (2013). 20GE%20ANNOUNCES%202014%20THIRD% 20QUARTER%20RESULTS.pdf. [17] Crandall, J. R., Zinn, D., Byrd, M., Barr, E., and East, R. ConceptDoppler: A weather tracker [5] University of Hong Kong, WeiboScope. Available for Internet censorship. In 14th ACM Conference on at http://weiboscope.jmsc.hku.hk/. Computer and Communications Security, Oct.29- Nov2, 2007 [6] YY Inc, 2014 Annual Report. Available at http:// (2007), pp. 1–18. investors.yy.com/annuals.cfm. [18] Crete-Nishihata, M., Dalek, J., Hardy, [7] YY Inc, YY主播违规管理方法. Available at http: S., Ng, J. Q., and Senft, A. Asia Chats: //www.yy.com/1309/242131808402.html. LINE Censored Keywords Update. Available at https://citizenlab.org/2014/04/line-censored- [8] Information Office of the State Council of the Peo- keywords-update/, 2014. ple’s Republic of China. Available at http://english. gov.cn/2010-06/08/content_1622956.htm, 2010. [19] Ensafi, R., Winter, P., Mueen, A., and Cran- dall, J. R. Analyzing the Great Firewall of China [9] Guagua: Exploring China’s Online Video Com- over space and time. In Privacy Enhancing Tech- munity ‘Goldmine’. Knowledge@Wharton (Oct. nologies Symposium (2015), De Gruyter Open. 2013). Available at http://knowledge.wharton. upenn.edu/article/guagua-exploring-chinas- [20] Hardy, S. Asia Chats: Investigating Regionally- online-video-community-goldmine/. based Keyword Censorship in LINE. Avail- able at https://citizenlab.org/2013/11/asia- [10] China Digital Times, Steamed Bun Xi. Avail- chats-investigating-regionally-based-keyword- able at http://chinadigitaltimes.net/space/Steamed_ censorship-line/, 2013. Bun_Xi, 2014. [21] King, G., Pan, J., and Roberts, M. E. How cen- [11] China Launches Campaign to Cleanse sorship in china allows government criticism but Web of Terror Content. Available at http: silences collective expression. American Political //www.reuters.com/article/2014/06/20/us-china- Science Review 107 (2013), 1–18.
9 [22] King, G., Pan, J., and Roberts, M. E. Reverse- [33] wa Fu, K., hong Chan, C., and Chau, M. Assess- engineering censorship in china: Randomized ex- ing censorship on microblogs in china: Discrimina- perimentation and participant observation. Science tory keyword analysis and the real-name registra- 345 (2014), 1–10. tion policy. IEEE Internet Computing 17, 3 (2013), 42–50. [23] Knockel, J., Crandall, J. R., and Saia, J. Three researchers, five conjectures: An empirical analy- [34] Weaver, N., Sommer, R., and Paxson, V. Detect- sis of tom-skype censorship and surveillance. In ing forged tcp reset packets. In NDSS (2009), The FOCI 2011: Proceedings of the USENIX Workshop Internet Society. on Free and Open Communications on the Internet (2011). [35] Winter, P., and Lindskog, S. How the Great Firewall of China is Blocking Tor. In Free and [24] Link, P. China: The Anaconda in the Chandelier. Open Communications on the Internet (Bellevue, The New York Review of Books (2002). WA, USA, 2012), USENIX Association. [25] MacKinnon, R. China’s Censorship 2.0: How [36] Zhu, T., Phipps, D., Pridgen, A., Crandall, J. R., companies censor bloggers. First Monday; Volume and Wallach, D. S. The velocity of censorship: 14, Number 2 - 2 February 2009 (2009). High-fidelity detection of microblog post deletions. In Presented as part of the 22nd USENIX Secu- [26] MacKinnon, R. China’s “networked authoritarian- rity Symposium (USENIX Security 13) (Washing- ism”. Journal of Democracy 22, 2 (2011), 32–46. ton, D.C., 2013), USENIX, pp. 227–240. [27] Ng, J. Q. Tracing the Path of a Censored [37] Zittrain, J., and Edelman, B. Internet filtering in Weibo Post and Compiling Keywords that China. Internet Computing 7, 2 (2003), 70–77. Trigger Automatic Review. Available at https://citizenlab.org/2014/11/tracing-path- censored-weibo-post-compiling-keywords- Notes trigger-automatic-review/, 2014. 1The raw and processed dataset with visualizations is available at [28] Ng, J. Q. Politics, Rumors, and Ambigu- https://china-chats.net ity: Tracking Censorship on WeChat’s Public Accounts Platform. Available at https://citizenlab.org/2015/07/tracking-censorship- on-wechat-public-accounts-platform/., July 2015. [29] Park, J. C., and Crandall, J. R. Empirical study of a national-scale distributed intrusion detec- tion system: Backbone-level filtering of HTML re- sponses in China. In 30th International Conference on Distributed Computing Systems (ICDCS) 2010 (2010), pp. 1–12. [30] Tang, A. China’s ‘evil’ church demolition campaign continues, say activists. Available at http://www.telegraph.co.uk/news/worldnews/asia/ china/11537435/Chinas-evil-church-demolition- campaign-continues-say-activists.html., April 2015. [31] Villeneuve, N. Search monitor project: To- ward a measure of transparency. Available at http://citizenlab.org/wp-content/uploads/2011/08/ nartv-searchmonitor.pdf/, 2008. [32] Villeneuve, N. Breaching Trust: An analysis of surveillance and security practices on China’s TOM-Skype platform. Available at http://www. infowar-monitor.net/breachingtrust/, 2009.
10