Research Proposal and Analysis Plan

Who Believes The Peopleʼs Daily? Bias and Credibility in Authoritarian Media

Rory Truex1

Overview

The goal of this project is to understand the relationship between media bias and credibility in the authoritarian context. Are Chinese citizens naïve about the biases of different news sources, or are they sophisticated consumers of information? A sample of Chinese netizens will view a series of news stories, with the source of news story randomly assigned. Respondents will then provide an assessment of the bias of the news article. This design will allow us to see whether Chinese citizens use media outlet names as heuristics for the bias and quality of journalism in a news article. I will also be able to develop measures of the credibility of different news outlets— state- owned, foreign, and the commercial quasi-independent. The overarching hypothesis is that Chinese citizens will exhibit evidence of the hostile media phenomenon. Those sympathetic with the (CCP) will perceive foreign and quasi- independent media as biased, while more critical citizens will be dismissive of state- owned media outlets like The People's Daily.

Existing Literature

A rich literature on media bias in the U.S. has documented three important trends. First, competition has not produced more balanced reporting, but instead, a market that is fragmented among outlets that produce slanted news (Groseclose and Milyo 2005; Gentzkow & Shapiro 2010; Gilens & Hertzman 2000; Mullainathan & Shleifer 2005). A number of reasons for this fragmentation have been proposed— corporate ownership of news outlets (Gilens & Hertzman 2000); ideological leanings of journalists and editors (Baron 2006); and consumer preferences for likeminded news (Gentzkow & Shapiro 2006). Second, exposure to different sources of news can have drastic effects on political attitudes and behavior. In their study of Anti-Americanism in the Muslim world, Gentzkow and Shapiro (2004) show that misinformation about the events of September 11th is widespread. Beliefs prove directly related to news exposure; respondents watching Arab news channels prove less likely to agree that the attacks were carried about Arab terrorists. Della Vigna and Kaplan find that the entry of Fox News into media markets in the late 1990s had substantial effects on voter attitudes, shifting vote share in the Republican direction by as much as 0.4 to 0.7 percentage points in the 2000 Presidential election. Third, consumers seem to have a general preference for news that confirms their prior beliefs (Vallone et al. 1985; Mullainathan & Shleifer 2005; Baum & Gussin 2007; Gentzkow & Shapiro 2010). In their seminal study, Vallone et al. (1985) expose pro-Arab

1 Assistant Professor of Politics and Public Policy, Princeton University. [email protected].

Sample Proposal – Who Believes the Peopleʼs Daily? 1 and pro-Israeli partisans to television coverage of the Beiruit massacre. Despite viewing identical coverage, partisans rate the programs as biased against their side. Baum and Gussin (2007) extend this basic design to the 2004 presidential election. Respondents viewed identical news reports identified as originating from CNN, FOX, and a fictional station. Respondents generally perceiv the FOX story as favoring Bush, and the CNN story as favoring Kerry. Interestingly, perceptions of bias prove conditional on respondentsʼ ideological leanings— pro-Bush respondents only perceived CNN as biased towards Kerry, and viewed the FOX coverage as neutral. We have yet to test and transport these ideas to the Chinese case, despite the countryʼs status as the worldʼs largest media market. While all newspapers must receive formal government sponsorship, the past two decades have seen the rise of more commercially oriented “Evening” and subsidiary papers that attract readers with more liberal reporting (Qin et al. 2012). Lorentzen (2012) argues that the CCP regime may deliberately allow some quasi-independent journalism to combat corruption, but that more tightly controlled outlets will generally produce positive reporting to prevent too much “bad news” from getting in the system. Qin et al. (2012) document substantial variation in the level of political control and pro-government bias of Chinaʼs domestic news sources. Party “Dailies”, in comparison to Party “Evenings” and other subsidiary newspapers, are more likely to reference regime leaders, Xinhua news articles, avoid sensitive stories covered in the Epoch Times, and cover low-level corruption and natural disasters. This project will assess how Chinese citizens process and consume news. Do citizens “believe” coverage in outlets under tight political control? Qin et al. (2012) show a negative association between advertising revenues and political control. This project will further investigate media consumption through an original survey experiment of netizens, replicating and extending the work of Baum & Gussin (2007), Gentzkow and Shapiro (2004), and others. The remainder of the analysis plan outlines the proposed data collection, research design, hypothesis, and empirical specifications.

Data Collection and Research Design

The survey will be administered through China Online Marketing Research (COMR), a marketing research firm that maintains an online panel of 1.6 million Chinese citizens, the largest of its kind in China. COMR has a strong reputation and has successfully completed 200,000 samples to date. The company holds the Internet Content Provider License and a Permission Certificate for launching foreign survey projects, both of which are necessary under current Chinese government regulations. The panels also comply with the European Society for Market Research (ESOMAR) and the China Marketing Research Association (CMRA) by ensuring that panel participation is completely voluntary, privacy is maintained, and the data procedures are reliable. My previous research with COMR went very smoothly, and returned a sample that reconciled well with known characteristics of Chinaʼs online population (Truex 2014). In this instance, I will restrict my sample to the Beijing-based population, as to better assess attitudes towards local and national media outlets. Possible respondents will receive a link in their email inviting them to take part in a survey on their political attitudes. After completing an informed consent procedure, each respondent will answer a series of questions on her demographic background and media consumption patterns. The core of the survey is a news assessment procedure, similar

Sample Proposal – Who Believes the Peopleʼs Daily? 2 to that used in Baum and Gussin (2007). Each will view ten stories, with the source randomly assigned across twenty different news outlets. After each story, the respondents will assess the bias in reporting and quality of the journalism. An excerpt from the questionnaire is below:

Now you will view several different news stories. Please assess the quality and trustworthiness of the journalism:

The source for the story for questions T1-T12 will be randomly assigned from the list:

[The Peopleʼs Daily, , Xinhua.net, The Global Times, Reference Times, China News Service, , Southern Metropolitan Daily, Yangcheng Evening News, Nanyang Sin-Chew Lianhe Zaobao, The New York Times, BBC Chinese, Epoch Times, Ming Pao, Beijing Times, Beijing News, Beijing Daily Messenger, Beijing Evening News, Beijing Morning News, Beijing Youth Daily]

T3. Please read the following excerpt from a news story:

Education, employment set to boost security in Xinjiang

[SOURCE] – The central governmentʼs decision to crack down on terrorists and improve livelihoods in the Xinjiang Uygur autonomous region will help bring security and prosperity, residents and analysts said.

Urumqi resident Zhao Jiepei, 28, mother of a 3-year-old boy, is confident the measures can help.

“I think everyone like me wants their children to live in a stable and developed society,” she said.

Ma Pinyan, a senior anti-terrorism researcher in Xinjiang, praised the governmentʼs efforts, to make employment the top priority in improving livelihoods, including helping at least one person in unemployed families.

“If young people have jobs and better educational quality, they will live better lives and have the ability to distinguish what is bad,” he said.

This year, the regional government has earmarked 900 million yuan ($150 million) to support 195 programs covering ethnic handicrafts, farm produce, Uygur medince and the clothing industry.

From [SOURCE], May 28th, 2014

Original source: http://english.peopledaily.com.cn/n/2014/0528/c90882-8733656.html

T3a. How would rate the trustworthiness of the content and journalism in the above news story:

- <01> Completely trustworthy

Sample Proposal – Who Believes the Peopleʼs Daily? 3 - <02> Mostly trustworthy - <03> Neutral - <04> Mostly untrustworthy - <05> Completely untrustworthy - <99> No answer

T3b. In your opinion, the coverage of the Chinese government in the above news story seems:

- <01> Very favorable - <02> Somewhat favorable - <03> Balanced/Even handed - <04> Somewhat unfavorable - <05> Very unfavorable - <99> No answer

The survey closes with a number of questions that measure the respondent's political ideology. All questions and procedures are included in the accompanying Draft Questionnaire.

Hypothesis

There are several contending theoretical frameworks for thinking about media attitudes in authoritarian contexts. As documented above, the countryʼs state-owned media tends to paint a rosy picture of domestic and foreign events, which in turn may produce a population with distorted beliefs. This distortion would be particularly acute if citizens were unaware of the biases in their media environment.

Hnaive: Perceived trustworthiness and bias will not vary across different media outlets.

This “naïve hypothesis” holds that Chinese citizens are not sophisticated consumers of media, and place equal weight on news regardless of the source. A second view, incompatible with the first, is that Chinese citizens are generally skeptical of state-owned media, which tends to produce or spin news to favor the CCP regime.

Hskeptical: Perceived trustworthiness and bias will be lowest for media outlets most tightly under government control.

This framework assumes a savvy Chinese media consumer, one that readily detects the general pro-regime bias of the dominant state-owned outlets. As in the U.S. setting, Chinese consumers may use sources s heuristics for bias and the quality of journalism— “the messenger overwhelming the message,” to use Turnerʼs (2007) phrase. Finally, we might hypothesize that a version of the “hostile media phenomenon” might help explain citizen attitudes towards media outlets. Extending this reasoning to the Chinese setting yields the following hypothesis:

Sample Proposal – Who Believes the Peopleʼs Daily? 4 Hhostile: Citizens more supportive of the regime will perceive news from quasi- independent and foreign outlets as biased, while citizens critical of the regime will perceive state-owned outlets as biased.

Chinese citizens do not identifying with two competing parties, but existing survey research suggests there is real variance in general sympathy with the CCP regime. I expect a version of the hostile media phenomenon operates in China, with the population divided into two camps. Critical citizens will be skeptical of state-owned media, while regime supporters will be skeptical of more independent news outlets.

Empirical Analysis

Variables

The research design will allow us to evaluate these contending hypotheses. The analysis will use a combination of demographic and attitudinal variables. My primary dependent variable will be TRUSTij which corresponds to the five point Likert scale question Tja for stories j=1…8. I will also employ the variable BALANCEij, which corresponds to the five point Likert scale question Tjb for stories j=1…8, in additional robustness checks. The main treatment variable will be the OUTLETij, which corresponds to the randomly assigned source for each individual i and story j. This variable, which is categorical, will be transformed into twenty indicators for the purpose of the analysis. To simplify the analysis (and avoid estimating twenty separate treatment effects), I will also conduct specifications that group different types of media outlets together— STATEij indicating relatively conservative outlets under tight state control, INDEP ij indicating the quasi-independent, commercially oriented domestic outlets, and FOREIGNij indicating the foreign news outlets. Additionally, we will consider whether perceptions of media bias are conditional on an individualʼs sympathy with the regime. To measure ideology, I will employ two questions, P2a and P3b:

P2a. In general, how do you feel about the government and its policies?

- <01> very critical - <02> critical - <03> neutral - <04> supportive - <05> very supportive - <99> refuse to answer

P3b. And how democratically is this country being governed today? Again using a scale from 1 to 10, where 1 means that it is “not at all democratic” and 10 means that it is “completely democratic,” what position would you choose?

- <1> 1 – not at all democratic - <2> 2 - <3> 3 - <4> 4

Sample Proposal – Who Believes the Peopleʼs Daily? 5 - <5> 5 - <6> 6 - <7> 7 - <8> 8 - <9> 9 - <10> 10 – completely democratic - <98> donʼt know - <99> refuse to answer

These variables will be coded as SUPPORTi and DEMOCRATICi, respectively. As additional robustness checks, some specifications will include a vector of the covariates: RURALi, FEMALEi, MINORITYi, LOWEDi, AGEi, MARRIEDi, CCPi, HIGHINCi, FARMERi, GOVEMPi, LABORERi, CLERKi. These variables will be measured with the demographic questions D1-D11 and were used in a similar fashion in my previous research (Truex 2014).

Core Models

There will be two core models in the analysis:

TRUSTij = ! j + "1:20OUTLET1:20ij + #Xi + $ij (1)

TRUSTij = ! j + "1:20OUTLET1:20ij + # SUPPORTi +$1:20OUTLET1:20ij • SUPPORTi + %Xi + &ij (2)

The first model is a simple linear regression of the TRUST measure on the OUTLET indicator variables, including a story indicator and the vector of demographic covariates X. This model will allow us to test the relative merits of the “naïve” and “skeptical” hypotheses. If the skeptical model is true, we should observe positive coefficients on the foreign and quasi-independent news sources, and negative coefficients on the state- owned outlets. The naïve model would suggest that the assigned OUTLET has little to no relationship with TRUST, which we can assess with an F-test. The second model allows us to assess the hostile media hypothesis. It includes our measure of ideology, SUPPORT, as well as interactions between SUPPORT and OUTLET. If the hypothesis is correct, we should expect that the effects of the OUTLET on TRUST are indeed conditional on ideology. The marginal effect of outlet 1 is now

!1 +"1SUPPORTi . If this outlet were The Peopleʼs Daily, for example, the hostile media hypothesis would suggest that that !1 < 0 and !1 > 0 ; trust is generally lower, but increases with an individuals general sympathy with the regime. The core analysis will pool all nine news stories together, but I will also estimate the model for each story separately to see if there is variance in the relationships across different story types. I will also re-estimate the models substituting DEMOCRATIC for SUPPORT, excluding the demographic covariates X, and using the grouped outlet categories STATE, INDEP, and FOREIGN instead of the individual outlet dummies. All specifications will use OLS with robust standard errors clustered at the story level.

Sample Proposal – Who Believes the Peopleʼs Daily? 6