A CRITICAL REVIEW OF AGENDA-SETTING METHODOLOGY AND THE EVALUATION OF A MEDIA-CENTRIC MODEL

ANDREW LAING

A DISSERTATION SUBMITTED TO THE FACULTY OF GRADUATE STUDIES IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF

DOCTOR OF PHILOSOPHY

GRADUATE PROGRAM IN COMMUNICATION AND CULTURE

YORK UNIVERSITY, TORONTO, ONTARIO

SEPTEMBER 2009 Library and Archives Bibliothgque et 1*1 Canada Archives Canada Published Heritage Direction du Branch Patrimoine de I'gdition

395 Wellington Street 395, rue Wellington Ottawa ON K1A0N4 Ottawa ON K1A 0N4 Canada Canada

Your file Votre r6f6rence ISBN: 978-0-494-64956-5 Our file Notre reference ISBN: 978-0-494-64956-5

NOTICE: AVIS:

The author has granted a non- L'auteur a accorde une licence non exclusive exclusive license allowing Library and permettant a la Bibliotheque et Archives Archives Canada to reproduce, Canada de reproduire, publier, archiver, publish, archive, preserve, conserve, sauvegarder, conserver, transmettre au public communicate to the public by par telecommunication ou par Nnternet, preter, telecommunication or on the Internet, distribuer et vendre des theses partout dans le loan, distribute and sell theses monde, a des fins commerciales ou autres, sur worldwide, for commercial or non- support microforme, papier, electronique et/ou commercial purposes, in microform, autres formats. paper, electronic and/or any other formats.

The author retains copyright L'auteur conserve la propriete du droit d'auteur ownership and moral rights in this et des droits moraux qui protege cette these. Ni thesis. Neither the thesis nor la these ni des extraits substantiels de celle-ci substantial extracts from it may be ne doivent etre imprimes ou autrement printed or otherwise reproduced reproduits sans son autorisation. without the author's permission.

In compliance with the Canadian Conformement a la loi canadienne sur la Privacy Act some supporting forms protection de la vie privee, quelques may have been removed from this formulaires secondaires ont ete enleves de thesis. cette these.

While these forms may be included Bien que ces formulaires aient inclus dans in the document page count, their la pagination, il n'y aura aucun contenu removal does not represent any loss manquant. of content from the thesis.

M Canada iv

ABSTRACT

Agenda-setting ranks among the most studied phenomena in communications research. A factor behind its popularity has been the nature of its research design that incorporates both media content and public opinion in a manner that yields an observable and empirically testable effect. Despite the model's importance, there has been little systematic evaluation of the underlying research design and, in particular, the role of media within the media/transfer of salience/public opinion equation. How media should be conceptualized within the research design has growing importance to the broader study of media effects due to the divergence of media sources and the erosion of mainstream media influence.

The thesis addresses these issues by first conducting a meta-analysis that focuses on the research designs in agenda-setting research published in the last 25 years in major journals. The meta-analysis supports the hypothesis that media is often underconceptualized in research designs, and offers a new approach—the media-centric model—that addresses key weaknesses concerning the proximity of media to audience/respondents and other factor that undermine research design validity. The media- centric model was tested over a five-month period in the Waterloo region of Ontario. The model includes formulas for weighting of media content based on market research and other audience demographic data.

The study confirmed the presence of an agenda-setting effect, and also confirmed the importance of proximity of news content and, in particular, the importance V of local news media coverage that is often overlooked in agenda-setting research. The presence of the agenda-setting effect was not significantly strengthened by the application of audience demographic data, but other factors from the model that enhanced proximity of content did indicate significant results, but were qualified by the nature of key news events.

The results provide an important contribution to conceptualizing the role of media not only in agenda-setting research designs but also in other areas of study into primary media effects within the context of an expanding universe of mediated sources of news and information. vi

TABLE OF CONTENTS Abstract iv List of Figures viii List of Tables ix Acknowledgements xi

Introduction 1

Chapter One: Reviewing the development of the media agenda setting research model 11 A. Defining agenda setting 11 B. Historical development of the model 14 C. Key elements of the model 25 1. Media agenda 28 2. Transfer of salience 41 3. Public agenda 47 4. Time 52 D. Validity in agenda-setting research design 55 1. External validity and media sampling 56 2. Internal validity and the transfer of salience 59 Discussion 62

Chapter Two: Meta-analysis of agenda-setting research design 65 A. Outlining the meta-analysis of research designs used in agenda- setting 65 B. The consideration of time: Cross-sectional versus longitudinally designed studies. 70 C. Media sample validity and proximity 71 D. Operationalizing the transfer of salience 76 E. Controlling for contingent conditions 79 F. Evaluating media sampling, transfer of media salience and contingent factors within a single index 83 G. Defining and constructing the media-centric agenda-setting research model 91 vii

Discussion 95

Chapter Three: Implementing the media-centric model 100 A. Site selection 100 B. Media sample selection 102 C. Content analysis design and data handling 113 D. Weighting media results 114 E. Sampling public opinion 117 F. Media consumption and demographics 119 Discussion 120

Chapter Four: Analyzing results from the media-centric model 123 A. Establishing testing criteria: media-centric model versus a non- media-centric model 123 B. Overview of media and public opinion data 126 C. Cross-sectional tests of the media-centric model 133 D. Longitudinal tests of the media-centric model 144 1. Economy 147 2. Health care 152 3. Environment 154 4. War/conflict 157 5. Consumer prices 159 Discussion 162

Conclusion 168

Appendix A: Meta-analysis codebook 180 Appendix B: Media-centric model: media analysis codebook 187 Appendix C: Coding reliability tests 194 Appendix D: Media-centric model: public opinion questionnaire 196 Appendix E: Media weighting formulae 209

Bibliography 218 Vlll

LIST OF FIGURES

Figure 1-1 Basic agenda setting model 28

Figure 1-2 How some researchers explain the transfer of salience process 59

Figure 1-3 Agenda-setting process defined by stages in developing the research design 62

Figure 2-1 Proximity rankings of media samples for agenda-setting research designs, by national versus local media samples 74

Figure 2-2 Distribution of agenda-setting research articles by methodological index 85

Figure 2-3 Average number of media outlets surveyed per agenda- setting study 89

Figure 4-1 Matrix of tests of the media-centric model 125

Figure 4-2 Medimediaa, coveragby weeke, oJanuarf economiy 1—Junc storiee 6s, b200y loca8 l and national, 149

Figure 4-3 Standardized scoring of economy as a top issue by local and national media over a ten-week period 150

Figure 4-4 Tracking local and national media coverage of health care issues by week: January 1—June 6, 2008 153

Figure 4-5 Tracking local and national media coverage of environmental issues by week: January 1—June 6, 2008 155

Figure 4-6 Local media coverage and public opinion—environment identified as a top local issue, by survey week 156

Figure 4-7 Tracking media coverage of war/conflict issues, by week: January 1—June 6, 2008. 158

Figure 4-8 Public opinion and media coverage concerning war/conflict issue: 56 day time lag 159

Figure 4-9 Standardized scoring of public opinion ranking of consumer issues, media coverage, and gasoline prices in Toronto, by week 162 ix

LIST OF TABLES

Table 2-1 Results of agenda-setting meta-analysis 67

Table 3-1 Newspaper readership by respondents per week 105

Table 3-2 Television news consumption by respondents, per week 108

Table 3-3 Internet site consumption by respondents, per week 112

Table 4-1 Top national issues by respondents 128

Table 4-2 Top local issues by respondents 128

Table 4-3a Top provincial and national issues by media coverage, by

month 129

Table 4-3b Top local issues by media coverage, by month (N=4876) 129

Table 4-4 JanuarDistributioy 1—Junn of toe p7 ,topic 2008s by broadcast news channel: 135 Table 4-5 Rank-order correlation of top topics by broadcast news channels 135

Table 4-6 Breakdown of major topic by scope 137

Table 4-7 Rank-ordemedia and rorigi correlationn of conten: topt national and local issues by 139

Table 4-8 Rank order correlation of education and interpersonal discussion as control variables on local and national media coverage of issues 141

Table 4-9 Rank-order correlation by national issues—weighted versus non weighted media data 142

Table 4-10 Rank-order correlation of top issues by high/low news consumers, by media type 144

Table 4-11 Respondent's selection of top issues by week 145

Table 4-12 mediCorrelatioa n by top issue over time by media weights—all 147

Table 4-13 Correlation of economy as a top issue by time lag and media type 149 X

Table 4-14 Correlation of the economy as a top issue by lag periods, media type and level of media usage by respondents 151

Table 4-15 Correlation of health care as a top issue by lag periods, media type and level of media usage by respondents 154

Table 4-16 Correlation of the environment as a top issue by lag periods, media type and level of media usage by respondents 157

Table 4-17 Correlation of consumer issues as a top issue by lag periods, media type and level of media usage by respondents 160

Table 5-1 Percentage share of media usage per by respondents 173 xi

ACKNOWLEDGMENTS

I wish to first and foremost express my thanks to my thesis supervisor Professor Fred Fletcher for his guidance, clarity of thought, and a host of his other many qualities I've drawn upon during my doctoral studies. Also, for their assistance and patience, I wish to very gratefully recognize both the chair of the department, Professor Barbara Crow, and the program's administrator, Diane Jenner, for their encouragement and support, and extend my own appreciation to their talent and dedication in shepherding myself and so many fellow graduate students through the labyrinth of academic studies. I also wish to acknowledge the comments and insight of other members of my committee, particularly Professors Charles Davis and Robert MacDermid. Everyone at York has been very accommodating and supportive of my overly-long stay at the university - and for that they have my gratitude for years to come.

I want to thank Karen Phillips and her team at Research Strategy Group for the challenge of putting my questionnaire into the field and for taking in stride my many changes over the course of the fieldwork period in stride. Also of assistance in the project were three coders—Sandra Fonseca, Anne Dollack and Derrick Lee—who were diligent, timely and, above all, consistent in preparing the content data, with Mr. Lee deserving of particular acknowledgement for his efforts in helping me coordinate the coming and going of the very many data files required for a project of this magnitude. The author acknowledges that the thesis and all data and other materials used in its preparation will not be used for commercial purposes.

Finally, this dissertation, because of how it has taken me away from those most important to me, is dedicated to my daughters Kateryna and Kara, and to my wife Patricia, who has never wavered in her support for my decision to pursue a doctoral degree at the mid-career and mid-family point of my life, only months after completing my MBA - if that isn't love, I don't know what is. 1

INTRODUCTION

"The future of agenda setting," argued Gerald Kosicki in his 1993 critique of the

concept, "is very much the future of media effects" (108). Agenda setting, the idea that the

relative salience of objects held by one group, often the media, can be transferred to

another group, often the general public, is one of the most discussed and published areas of

study in communications research. The research tradition now spans over 35 years. It

boasts over 400 published studies (McCombs, 2004) that populate the academy's leading journals such as Journal of Communication, Public Opinion Quarterly and the American

Political Science Review. While concentrated in the United States, agenda-setting research

has been conducted around the globe, including Spain, Japan, Taiwan, Germany and

Canada. A recent citation survey (Bryant & Miron, 2004) ranked it on par with uses and

gratifications (developed years earlier) as the leading subject of mass communications

research in North America. 2

Most research activity has been directed at pushing the theoretical boundaries of agenda setting. Initial research in the early 1970s documenting the agenda-setting effect of the mass media soon gave way to examining the model's various "contingent conditions," exploring the topic along a range of issues (political and non-political), and devising more longitudinal studies. More recently, the focus has shifted from "first-level" agenda-setting effects that map issues, to "second-level" effects that focus on the attributes of those issues covered by the media, and reflected in public opinion. Second-level agenda-setting effects have also brought agenda setting closer to other areas of study involving cognitive media effects, primarily framing and priming.

At the core of this area of research is a simple hypothesis—that issues deemed important by one group are communicated to another group through a mediated transfer of salience cues. While research into agenda-setting effects has gone in several directions, at its core has been what McCombs and Shaw first studied in Chapel Hill in 1968—the

"media agenda-setting model"—which served as the catalyst for research and remains a key area of interest. The basic research design of the media agenda-setting model is composed of two parts: a media content analysis to determine the media agenda, and public opinion polling to determine the public agenda. The qualities of content analysis and polling in communications research are undoubtedly a factor behind agenda-setting's popularity in communications research. Both methods have a long history in communications research. Both methods are generally known and accessible to communications researchers and, in fact, technology may be allowing them to become even more accessible. Databases such as Lexis-Nexis or the Vanderbilt University television 3 archive allow scholars easier access to media content. Similarly, the media agenda-setting model rests primarily on public salience derived from a common question, Gallup's ubiquitous "most important problem" or MIP question that asks respondents "what is the most important problem facing the country today?" Data for this question now stretches back decades and results from this question and other public opinion data are becoming increasingly available to academic researchers through public polling archives. Equally important to agenda-setting's historical credibility and accessibility is the fact that the methodologies of both content analysis and public opinion research have known parameters for testing and establishing the two principal qualities of social science research design: validity and reliability. Ultimately, it is agenda setting's ability to marry these two methodologies and produce evidence of a significant, statistical effect that may lie at the root of the model's success and popularity in communication research. Agenda-setting is one of the very few research designs in communications research that can, and has, yielded significant statistical effects within a field experiment.

If the qualities of the research design underlying agenda-setting help to explain the theoretical growth of agenda-setting, then it is remarkable how little scrutiny the design has undergone in the last forty years. Several literature surveys of the field have been conducted (Rogers, Dearing & Bregman, 1993; McCombs, 2004), and several authors have critiqued specific elements of agenda-setting research, such as its treatment of longitudinal data (Gonzenbach & McGavin, 1997). As these examples show, methodological reviews of agenda-setting have been few and quite limited in scope. Agenda-setting research lacks a thorough methodological critique; something long overdue given its forty year history. 4

The importance of a methodological review to support the theoretical

exploration of agenda-setting is not only overdue, but also timely given the challenges

posed to many of the theory's underlying assumptions as a result of new communications

technologies and new media. If the underlying research design of agenda-setting has remained the same, what it studies—namely, the media, and the expression if not the

content of public opinion—has changed profoundly since 1968. Producing a representative

sample of the general population has traditionally been based on the time-honoured

approach of using random-digit dialing. However, recent changes in telephone technology,

particularly the advent of caller identification services, coupled with the decline in the use

of dedicated landline telephones in favour of mobile phones, has been cited as a key factor

in the reduction in response rates in recent years due to changes in telephone technology

(Curtin, Presser & Singer, 2005). The decline in response rates has undermined the

validity of using RDD and land-line telephones as a basis of determining the views and top

issues held by the public. While internet-based panel studies are growing in use in reaction

to the decline in response rates (and the ensuing rise in data collection costs), the lack of

representativeness posed by these technologies (that it skews towards segments of the

general population that are more technologically savvy) remain an issue about the validity

of this approach. These are important methodological issues, and they affect not only

agenda-setting but the broader study of how media can influence elements of public

opinion. However, the decline of response rates and the impact of new caller ID

technology and mobile phones have received notable discussion. 5

As important, but arguably less scrutinized, are the consequences for research design validity resulting from new media technologies on the media side of the research design equation. The premise of agenda setting rests on the ability of a researcher to determine a coherent agenda of issues within a representative group drawn from two sample populations: the media, and the public. As noted, random sampling is the preferred manner to establish sample validity within a human subject population. However, for the media, random sampling of outlets (in contrast to stories, statements or other units of analysis produced by the media outlets) is inappropriate. Instead, researchers have traditionally relied on logical arguments that establish the face validity of the sample: that based on known or historically-derived parameters, a sample set of outlets would logically be considered to "represent" the media. The actual representativeness of many purported media samples in agenda-setting research designs over the last forty years has been suspect: radio has almost always been excluded, and there has been a bias towards larger, elite, national print and television media outlets to the exclusion of local media. This historical weakness in agenda-setting research designs will only become more evident in the years to come (if that moment has not already arrived). The explosion of sources of news and information available to potential respondents—blog posts, RSS feeds, Google News, text- message updates, discussion forums, digital satellite radio newscasts, 24-hour cable news— has made it increasingly difficult to establish the validity of the media sample by relying on an argument of "representativeness" within the research design of agenda-setting.

This thesis is designed to address methodological questions about the role of media in agenda-setting research design and, in so doing, provide a more solid foundation 6 for researchers in using media in other areas of communications research that examine other areas of primary media effects. The dissertation has two principal objectives. The first objective is to provide a systematic critique of the research designs employed by agenda-setting. Specifically that it under-conceptualizes the role of media at two phases within the basic research design. First, at the sampling phase of experiments, agenda- setting research designs often fail basic tests for external validity. There is often little consideration as to whether the sample of media outlets chosen has any proximity to what respondents choose to read, watch or listen to as consumer of news. Second, at the phase in which the transfer of salience is modeled, media is viewed as a simple, singular construct. Its impact is often measured by simple counts of stories regardless of whether the story appears in small, local outlets or major national news chains. There is little elaboration offered as to how media transfers salience (Kiousis, 2004).

Following from this critique, the second principal objective is to present and test a research model specifically designed to address the methodological issues of how media is conceptualized. The design is referred to as the media-centric model, and would contain elements that reflect how audiences use and access different media in the context of an increasingly expanding universe of news and information choices. It would include as part of its design the key elements about how people currently consume and use news media by mapping what media respondents might be exposed to, tracking the influence of different types of media, and weighting data based on whether the outlet has a greater or lesser likelihood to influence awareness based on the size of its audience within the sample population. A practical, useful contribution to the field would be to outline the terms of a 7 media-centric research model that focuses on media consumption, proximity, channels, and audience exposure in addressing how multiple issues highlighted by news outlets affect public opinion.

Therefore, the overall goal of this thesis, to employ a metaphor, is not to build another level on to the ever-expanding theoretical framework of agenda-setting, but to work on its foundation. As McCombs (2004) himself has suggested, agenda-setting research is entering a "third phase" that will include research that will "examine key established concepts of agenda-setting theory and to draw new, more finely-detailed maps.

(144) In this context, the intent is to construct both an evaluative framework to first assess the methodology used in agenda-setting research models from the perspective of how they incorporate and conceptualize the role of media within their designs, and then use these findings to build and test a more media-centred research model. The result should yield a means of determining the relative strength of various elements within a media-centric model that might offer future researchers a better understanding of how to approach the role of media in their designs, and how different strategies might achieve stronger (or weaker) results in their efforts to expand the theoretical basis of this important area of media effects.

The dissertation is divided into four parts. The first chapter of the dissertation provides an overview of the development of the agenda-setting research model, points to major theoretical criticisms about the concept and, finally, highlights agenda-setting's main research design considerations. Since there are few published articles addressing research design questions in agenda-setting, design considerations are drawn from a review of major 8 experiments in agenda-setting undertaken over the last 35 years. The research design elements are reviewed and categorized based on the three main elements of agenda-setting research: the media, the transfer of salience, and the public. From these elements a basic analytic framework is outlined.

The second chapter applies the analytic framework to a meta-analysis of agenda- setting research published over the last 25 years in leading journals in order to objectively assess how media is viewed and subsequently constructed within the research models and design questions used in this field. The results of the meta-analysis and the analytic framework will be used in defining and establishing the basic properties of the media- centric model.

The third chapter outlines the creation of a field experiment based on a media- centric approach and implementing the model within a particular area—the Waterloo region of Ontario over a five-month period between January and early June, 2008—and discusses in detail lessons learned by researchers from the implementation of the model.

The fourth chapter reviews and discusses the actual results of the implementation of the model, looking at both a cross-sectional review as well as a longitudinal review of five key issues over the ten weeks in which public opinion was sampled. The basic design of the dissertation ensures that future researchers are provided with a clear analytic model to evaluate the sample validity surrounding the media component of their research design, as well as the strengths and weaknesses of a media- centric approach and the issues raised by implementing this model. 9

While part of the explanation behind the popularity of studying agenda-setting may lie in its methodology, the media effect is more than a subject of academic curiosity, and has important ramifications for the relationship between the news media and democratic institutions. While many journalists recognize their role in conveying important information to the public in a timely, accurate and balanced manner, media outlets and the news they cover, as an aggregate, also convey to the public what is important, and what is less important, by the emphasis and extent of attention they devote to certain issues and events. The individual exposed to a news item in a mass media outlet such as the Globe and Mail or a CBC Radio newscast is not only provided with information about the event, but at an implicit level, understands that thousands of other

Canadians are also being made aware of that same issue, in the same manner, at the same time. This social cohesion role played by the news media, partly through the agenda- setting function of the media, has been recognized by its early authors. As Donald Shaw and Bradley Hamm (1997) have written, while traditional mass media exercise a centripetal force in society, new media exercise a centrifugal force, moving people away from a core towards smaller, fragmented communities around individual interests with their own unique agendas, and away from larger communities defined more by geography.

If Gerard Kosicki is correct, and that agenda-setting is out in front as a canary in the coal mine for the broader field of media effects, then arguably one of the more pressing issues for the long-term viability of media effects research is to address the issues raised by the changing nature of media. Changes in media technologies are imposing fundamental, paradigmatic change on our understanding of a hierarchical, coherent 'media' by 10 undermining traditional sources such as newspapers while at the same time greatly expanding the selection of new sources of news and information. Before these considerations to be properly examined for their implications for theory, there should be proper allowances and adjustments made to the basic research designs employed to investigate them. The long-term viability of media effects research as a field will rest on the ability of researchers to recognize this shift and make allowances within their research design.

After almost forty years of continuous study and a small mountain of published research within its domain, agenda setting is no longer an embryonic concept, and while other topics of study in mass communications research such as framing effects may now be garnering more attention (Bryant & Miron, 2004), agenda setting remains a very large part of both a popular as well as academic understanding of how media can affect public opinion. A review of the methodological underpinnings of agenda-setting research models is, if anything, timely. It is hoped that the following dissertation will lead to improvements in the assessment of new research models intended to expand even further agenda-setting as a media effect, and the ability by which media raise issues of greater or lesser importance to society at large. 11

CHAPTER 1

Reviewing the Development of the Media Agenda-Setting Research Model

A. Defining agenda setting

Agenda setting belongs to a larger group of empirical approaches to the study of communications commonly known as media effects research. Media effects research, as

McLeod, Kosicki and Pan (1991, p. 236) explain, is distinguishable from other approaches that study communications by the fact that it focuses on audiences, in which there is evidence of influence that can be attributed to a message system derived from the media, and that models to determine influence usually involve variables and empirical testing.

Other types of cognitive media effects include priming, knowledge gap, spiral of silence and framing—approaches to researching mass communications that have emerged in the latter half of the 20th century (McQuail, 2000). Agenda-setting research is arguably among the most popular and straightforward examples of the media effects tradition. Simply put, the agenda-setting hypothesis describes the transfer of an agenda between one group and another. Critical to the model is a process by which the importance of issues or topics is transferred or communicated between groups. This process, normally referred to as a transfer of salience, is dependent on the presence of explicit and implicit cues that signify to one group the rank-order of importance of issues of another group. An "agenda" is normally comprised of issues or topics in rank-order of concern. The agenda-setting research model is an empirical means by which a researcher artificially replicates the agendas of the two groups, examines whether significant correspondences exist, and draws conclusions about the larger process from these correspondences.

While not exclusive, the agendas are normally derived from three broad social groups defined by membership (Roger, Dearing, & Bregman, 1993): the public (ranging from individuals to social or geographically-defined groups or a general population); the media (almost exclusively news media, although there have been examples of other types of media, such as social/entertainment media [Soroka, 2000b]); and, occasionally, policy- makers (governments, political actors or other organizations). Regarding the latter, policy agenda-setting boasts a relatively diverse literature generated by sociologists and political scientists investigating the core question of "how does an issue get on the policy agenda,"

(Rogers, Dearing, Bregman, 1993, p. 72) and has included research that examines the dynamics of a policy-media agenda, such as the relationship between the content of the

U.S. President's State of the Union address and news media coverage (Gilberg, et. al., 1980; Wanta, et. al., 1989). However, for the most part, policy agenda studies have developed largely outside of the field of communications studies. In one of the few

Canadian examples of agenda-setting research, Soroka examined the dynamics between all three agendas (Soroka, 2000a; Soroka, 2002). Other recent examples that have widened our understanding of agenda-setting processes at work in society have included variations on the public agenda, including religious agendas (Buddenbaum, 2001) and larger cultural agendas. Despite the investigations into the agendas of other social groups, most agenda- setting research involves the inter-relationship of the first two of the aforementioned agendas, referred to either as media agenda-setting or public agenda-setting, depending on the identification of the independent variable.

As the above illustrates, agenda-setting researchers have at times defined the concept using very broad terms (McCombs, 2004, p. 141; Rogers & Dearing, 1993, p.

69)—an important factor that has likely contributed to the ever-expanding range of research models that has in turn both enlarged and refined the original concept. However, while each approach to agenda setting offers important avenues of further research, this paper focuses more narrowly on the most common model—media agenda-setting—in which the media serve primarily (but not exclusively) as the independent variable, while public opinion serves primarily (but not exclusively) as the dependent variable. Narrowing the discussion to the media agenda-setting model is appropriate for the objective of evaluating research methodologies. Media agenda-setting is the most commonly-used research model involving agenda setting. In a literature survey conducted over the first twenty years since the model was introduced in 1972, Rogers, Dearing and Bregman (1993) identified 131 of 223 published studies focusing on this approach, with media

agenda-setting accounting for 79% of all types of agenda-setting research cited by other researchers. Therefore, media agenda-setting offers the most fertile ground to critically

examine the development and application of agenda-setting research designs.

To address another question about use of terms—whether to describe agenda

setting as "theory" —the following discussion adopts the description of "model" for agenda

setting recommended by Kosicki (1993, p. 102). By using the term "model", Kosicki

eschews both the weaker label of 'metaphor', as well as the stronger descriptor of 'theory'

in that agenda setting is simply "one type of complex media effects" that seeks to link

media content and audience effects.

B. The historical development of the agenda-setting research model

The concept that media influences the public's agenda of top issues was

explored in communications research prior to the landmark 1968 Chapel Hill study by

McCombs and Shaw. McCombs himself has repeatedly cited Lippmann's 1922 Public

Opinion and its oft-quoted descriptor of how the media construct "a picture in our heads"

as both inspiration and apt metaphor to illustrate the media agenda-setting process

(McCombs, Escobar-Lopez & Llamas, 2000; McCombs, 2004). Early systematic analysis

of mass communication research by Paul Lazarsfeld and Bernard Berelson (Lazarsfeld,

Berelson & McPhee, 1954; Lazarsfeld, Berelson & Gaudet, 1948), as well as Lang & Lang

(1959), included efforts to link media content with the rank order of issues held by the public. Noelle-Neumann (1993, pp. 151-4) suggests that the work of Niklas Luhmann

"presaged" American research into agenda setting, although it appeared independently in the United States. Most importantly, Bernard Cohen's 1963 text The Press and Foreign

Policy was a particular influence on McCombs and Shaw's later work (Rogers, Dearing &

Bregman, 1993), resulting in Cohen's often-cited passage that establishes agenda setting as a term describing how the mass media "may not be successful much of the time in telling its readers what to think, but it is stunningly successful in telling its readers what to think about" (p. 8).

Nonetheless, the study by Maxwell McCombs and Donald Shaw of U.S. media coverage and the local electorate in Chapel Hill, North Carolina, on the eve of the 1968

U.S. presidential election, later published in 1972 in Public Opinion Quarterly, was unquestionably the launch point of the empirical investigation into media agenda-setting.

The study undertook 100 interviews over a two-week period among undecided voters in the area, asking respondents a series of Gallup-style "most important problem" questions as to what were the "key issues as they saw them?" and "what are you most concerned about these days?" The responses were coded into 15 major issue categories. Over the same two-week period, the researchers also coded stories from local and national print media outlets, as well as NBC and CBS nightly newscasts. A rank-order correlation conducted between the public and the media data sets was a "stunning" 0.967 for issues coded as

"major" items in the media, and 0.979 for minor items in the media—a degree of correlation that undoubtedly was a factor behind the subsequent rush of research into this area post-1972. Equally important, the 1968 study introduced the basic elements, definitions and constructs of the research design which subsequent researchers probing agenda-setting effects would use.

Agenda-setting emerged rather quickly as the dominant area of research into media effects. The results achieved by McCombs and Shaw immediately sparked a small torrent of similar studies: 50 different empirical investigations within eight years (Carragee,

Rosenblatt, & Michaud, 1987). Over a longer period, a literature review of agenda setting by Rogers, Dearing and Bregman (1993) of twenty years of agenda-setting research also indicated substantial activity surrounding agenda-setting research throughout the sample period, with its strongest peaks occurring in 1987 and 1991. The literature review indicated that agenda setting was, up to the early 1990s, still enjoying widespread use and growth.

While now dated, Rogers, et. al. suggested as of 1997 that in terms of Kuhn's paradigmatic model (1970), agenda-setting research has reached the level of "normal science" and has not reached a stage in which scholarly interest in the model has declined. A subsequent

survey by Bryant and Miron (2004) of the top 26 theories in mass communications in six major journals indicated that agenda setting tied with uses and gratification theory as the most cited and/or applied theory in the 20th century. The same study also observed, however, that agenda setting has begun to trail frame analysis in these same journals in the first four years of the 21st century, and that agenda-setting as a topic of communications research may have peaked in the early 1990s at the time when Rogers et. al conducted their initial meta-analysis. While McCombs (2004), among others, contend that agenda- setting research continues to attract researchers, interest in media framing may be supplanting agenda setting as the most popular model for examining cognitive effects between the media and the public.

Around the same period as the development of agenda-setting was the incorporation of framing by media researchers. Frame analysis in communications is loosely based on Erving Goffman's 1974 "Frame Analysis", a primarily sociological and philosophical treatise in which he described frames as "definitions of a situation [which] are built up in accordance with principals of organization which govern events [...] and our subjective involvement in them; frame is the word I use to refer to such of these basic elements as I am able to identify" (lOf). The concept of framing in communications studies, according to Iyengar (1987) involves the subtle alterations of a statement or presentation of a judgment or problem, and "framing effects" as the changes in decisions by individuals that occur as a result of these differing statements (11). Another definition is offered by Tuchman (1978): that the media actively set the frames of reference that readers or viewers use to interpret and discuss public events" (ix).

The research approach has often been to examine the effects of alternative news frames (either through laboratory work or field experiments) on the (usually) political choices of a public, such as the impact on viewers subject to TV news that employs

'episodic' (event-oriented reports on concrete issues with pictures, or 'thematic' (more general, background items and often using 'talking heads' to delve into an issue (Iyengar,

1987,11-3). As Scheufele (1999) describes, an examination of frames either tends to address media frames (how journalists employ central organizing themes, categories or structures to present stories in a way that will be recognizable and have resonance with a broad audience), and/or individual frames (the heuristic devices that audience members use to categorize and slot news and information).

Similar to agenda-setting, a notable if disparate body of research has been conducted into media framing effects in the last twenty to thirty years. Of particular note has been the use of media frame analysis in work in the U.S. and Canada in examining the gendering of political campaign coverage by both the U.S. media (Kahn 1994; Kahn and

Goldenberg, 1997; Devitt, 1999; Bystrom, Robertson & Banwart 2001) and in Canadian federal election campaigns, particularly televised leadership debate coverage (Gidengil &

Everitt, 2000; Everitt & Gidengil, 2003; Trimble & Arscott, 2003). For example, the political media tend to focus on 'horse race' aspects of the campaign - strategies, poll results, and other evidence to indicate who might be leading in the campaign - rather than on issues and platforms. This 'horse race' frame raises a number of problems for women, primarily if women are portrayed as less viable candidates due to a lack of experience or incumbency, which can lead to more negative coverage (Kahn, 1994, p. 164). For example, the media may employ a "first woman" frame (recognition of the candidate as the first to be elected to that particular office) that, while raising the volume of coverage, often sets the candidate apart as a novelty and farther from the norm (Heldman, Carroll & Olson,

2000, p. 8).

A third approach is priming. As Iyengar and Kinder define it, priming involves the process by which the media call attention to some matters while ignoring others, and thus influence the standards by which government and political actors are judged (1987, 63).

Priming draws its theoretical basis from a psychological understanding of individuals' cognitive capacity and how they process information - that a limited number of themes and heuristic factors, particularly accessibility to information. While considered by some as an extension to agenda-setting (Zaller, 1997; Iyengar & Kinder, 1987), it should be differentiated from agenda-setting, which establishes what issues are (or are not) recognized by the public as more or less important, with priming examining how a candidate is judged based on which issues the media give inordinate weight). For example, Iyengar and Kinder's research indicated that respondents would judge a

President's performance differently based on the type of issues shown to them in repeated newscasts. Miller and Krosnick (1997) examined media coverage of a number of issues

(1990 Gulf War, 1992 presidential election, the Iran-Contra affair) and how it changed the electorate's basis of judgment of the President's performance as respondents based more of their evaluation on foreign versus domestic policy considerations.

Agenda-setting, framing and priming and the research conducted using these

approaches marked an important development in understanding media influence,

principally by identifying the presence of significant media effects beyond the issue of persuasion and, as a consequence, reinvigorating mass communications research by

offering new methods and models for understanding such influence. Moreover, despite the

critics of 'significant media effects' in general, they have been useful for some researchers for the purpose of critically evaluating certain roles played by the media, as illustrated by

some of the research noted above. In particular, agenda-setting, priming and framing provided an effective basis: a) to provide an empirical, measurable basis by which to highlight and criticize the content and format of news; b) to underline areas where political and business elites may exert undue influence on the practice of journalism, news and current affairs; and c) to link media content and the formation of public policy in a way that is complementary to democratic ideals.

Nonetheless, these approaches to media influence do possess a number of weaknesses in how useful they are for critically evaluating the role that media plays in society. Some have argued (Iyengar) that approaches such as agenda-setting are more of a metaphor than a theory. Much of the research that has fallen under all three approaches suffers from what earlier limited effects research encountered: too much is devoted to proving that an effect exists, and too little towards linking that effect with either theory or with broader questions about the implications of the influence of media on audiences, or on the processes by which such news is determined. Furthermore, much of agenda-setting, priming and framing research addresses a rather specific set of questions arising from political communications; chiefly, how effective is the news media in providing information for citizens to make informed choices within a democracy.

There have been a number of factors cited in the literature for the rapid growth of agenda setting and other approaches to understanding primary media effects in communications research subsequent to the Chapel Hill study. Among the most influential and often discussed factors is its historical context within the evolution of media effects literature. Prior to the Chapel Hill study, the dominant explanatory paradigm was that the media had a "limited effect" on the public—the end product of a number of landmark studies by Paul Lazarsfeld, Bernard Berelson and Elihu Katz, among others, which found limited, indirect effects when they focused on the mass media's potentially more persuasive 21 influences on individual behavior, opinions and attitudes. Summarized by Klapper (1960), the perspective of limited effects and the concept of selective perception (that locates the influence of media primarily within the individual, such as his/her personal preferences, more so than external factors such as media salience) were sources of frustration for researchers who were convinced that media effects were significant, but that the design of research models at the time simply failed to uncover them (Carragee, Rosenblatt, &

Michaud, 1987; Protess & McCombs, 1991; Tankard, 1990). The approach taken by

McCombs and Shaw in 1972, by eschewing persuasive, behavioral effects and focusing instead on proving mass media's more elementary cognitive influence, provided a fresh approach relying on a research model that could demonstrate a significant, direct impact by the media on a more basic set of effects such as the public's awareness and expressed levels of concern over certain issues (Kosicki, 1993).

A second factor behind the model's growth has been that the exploration of agenda setting provided fertile ground for researchers to move relatively quickly beyond the original domain of demonstrating the media's impact on public salience of issues into other areas that expanded the understanding of this effect. The expansion of the research model progressed in several directions: a) as noted above, researchers explored other agendas other than simply the relationship between the media and public agenda; b) contingent conditions, such as the impact of media exposure and credibility, demographics, political affiliations and interest, were examined to determine their impact in either sharpening or limiting the agenda-setting effect; c) the various attributes of an issue and how they are covered by different media sources came under more frequent scrutiny as a contributing factor behind the salience of issues; d) intermedia agenda-setting effects were determined, examining potentially different levels of influence between print and television

(McClure & Patterson, 1976; Wanta, 1997) and, most recently, internet news coverage on public salience of issues; e) the range of specific issues has moved beyond its original focus on simply political, electoral issues to include civil rights (Winter & Eyal, 1981); the environment (Salwen, 1988; Schoenbach & Semetko, 1992), and drug abuse (Reese &

Danielian, 1989); and finally f) geographically, agenda-setting effects have been compared at local, regional and national levels, as well as in other countries, most notably in Spain

(McCombs, Lopes Escobar & Llamas, 2000); Japan (Takeshita, 1993); Taiwan (King,

1997) and Canada (Soroka, 2000a; Soroka, 2002). As discussed later, these various directions in the expansion of agenda-setting research has, in turn, contributed several new elements to the original research design, as well as raising a host of new questions about the methods employed as the body of research has expanded.

Despite its growth, agenda setting has been subject to criticism on both theoretical and methodological grounds since its inception, although broader critiques of the model within communications effects literature are surprisingly few, given the popularity of the model. Theoretically, agenda setting has been subject to a number of criticisms, a full catalogue of which is beyond the scope of this paper. Nonetheless, three broad directions serve to capture the most frequent and visible. First, criticism has been directed at the perspective that agenda-setting presupposes in modeling the relationship between media and public opinion formation, portraying it as mechanistic and oversimplified, particularly when compared with other approaches to the study of media effects. These other approaches, authors have argued, may provide more fruitful areas of

research, particularly framing (Entman, 1993), which establishes a better connection

between the cognitive effects of content and opinion formation than offered by the agenda-

setting model (Kosicki, 1993; Scheufele, 2000). Similarly, agenda-setting's focus on a

transfer of issue salience ignores other possible explanations involving how individuals

process information when determining issues of importance (for example, accessibility—

that respondents choose from a limited range of information most accessible to them), and

other important cognitive responses that are generally ignored in agenda-setting research

and the primary elements of its research designs. Testing for cognitive media effects tends

to involve research design elements that are not typically associated with agenda-setting,

such as response latency measures using various cues to determine attitude accessibility

among respondents, post-hoc coding of open-ended questions, or post-test questionnaires

(Bassili, 1995; Scheufele, 2000). Moreover, they typically involve laboratory experiments

rather than field research designs.

A second general theoretical criticism is that agenda setting focuses too narrowly

on the effects of media. The concentration on effects has distracted communications researchers from addressing more crucial social and cultural questions about the role of

media and society. In particular, authors have focused on the model's treatment of media

as a neutral, value-depleted independent variable (Westley, 1976; Gandy, 1991; Kosicki,

1993; Carragee, Rosenblatt & Michaud, 1987; Lang & Lang 1981), thereby overlooking

inherent and more important questions that would shed light on what factors and

institutional frameworks lead the news media to give salience only to a certain narrow 24 range of issues chosen among all possible issues of public concern. As an alternative, authors such as Shoemaker and Reese (1996) have called for a more critical examination that focuses on the factors and structure that ultimately contribute to how news media organizations identify, create and disseminate news. Other critics have expressed similar concerns that the agenda-setting model and its popularity among many researchers has led to a suggestion of "powerful" media effects which they reject, arguing that "issue salience" does not necessarily result in any demonstrable cognitive behavior, or that evidence of such behavior would need to be highly qualified (Burd, 1991; McLeod, Kosicki, & McLeod,

2002).

A third frequent source of criticism is that alternate explanations exist to explain the strong correlations achieved by many media agenda-setting studies. Alternative explanations have included whether media agenda-setting in particular understates the influence that the public exerts on the media, or whether the media in fact contribute to some combination of media, public and political policy agendas simultaneously (McQuail,

2000).1 Similarly, others have argued that"real-world events" have a greater influence on public perceptions than media-generated events. However, this latter critique is often incorporated within agenda-setting research, and several researchers have controlled for real-world events ranging from energy prices, drug issues, crime and employment, and still

1 While the issue of public influence on the media agenda is a frequent question raised about the agenda-setting model, many researchers in this field have studied the issue using cross-lagged correlations along longitudinal data. The results are not conclusive and can vary by contingent conditions as well as media and issue type, but results tend to indicate that more variance is explained by a media-public influence than a reverse relationship. Cf. (Iyengar & Kinder, 1987; Shaw & McCombs, 1977) 25 produced results that support the media agenda-setting hypothesis (Funkhouser, 1973;

Kepplinger & Roth, 1979; McCombs, Einsiedel & Weaver, 1991; Ader, 1995).

C. Understanding the key elements of the agenda-setting research model

Researchers involved in agenda setting have argued that the methodology has, in fact, become quite advanced. In coming to this conclusion, authors have cited several factors observed in the development of the model over time, particularly the use of more longitudinal data (Gonzenbach & McGavin 1997; Kosicki, 1993) accompanied by more sophisticated statistical tests. McCombs (2004) also suggests that in terms of questionnaire design, the reliance on the 'most important problem" question asked by the Gallup organization since the 1930s is now often supplemented with other questions in order to further probe issue salience.

Despite these declarations about the advanced state of agenda setting's underlying methodology, very few researchers have conducted a comprehensive evaluation of the subject. Gonzenbach and McGavin's 1997 methodological analysis of agenda setting looked rather narrowly at the research model, focusing exclusively on the treatment of time and what statistical techniques were applied to account for factors such as stationarity (the consistency of results over time in a time-series or panel studies using cross-lagged correlations) and autocorrelation (the correlation of a variable with itself over time), and how approaching agenda-setting using ARIMA modeling can correct for these problems. Accounting for stationarity and autocorrelation are important in any analysis of time-series data, but as Soroka (2002) points out, the use of ARIMA modeling in removing autocorrelation and stationarity can potentially affect other dynamics, and proposes other techniques for handling time-series data that involve using the original series. Kosicki

(1993, p. 107) also devoted much of his evaluation to the issue of time, which he described as an "insufficiently theorized and underspecified" concept given the causal language used in agenda-setting research. However, Kosicki went further by pitting agenda setting against the broader demands of media effects literature. In this regard, Kosicki concluded that "many individual agenda-setting studies are deficient methodologically" (107), and that "to conclusively demonstrate and document the existence of a media effect such as agenda setting, researchers must assemble a variety of evidence—including content, exposure, effect, and conditions" (109). Agenda setting, Kosicki went on to add, is no longer an "embryonic" idea within media effects, and must be structured to consider "more comprehensive and well-ordered evidence." In general, however, criticism of the designs of actual research models outside these few examples has been scant, and the field still lacks a comprehensive, systematic analysis and critique of the methodologies that have been applied in the name of agenda-setting research over the last 40 years.

Given this apparent lack of a systematic evaluation of the methodology and research designs employed in agenda setting, a two-step approach was adopted to systematically catalogue and critique the core elements of the research designs used historically in agenda-setting research. The first step involved a critical review of the three main elements of agenda-setting research design—the media, the transfer of salience process, and the public—and commentary on these elements from among agenda-setting's most prolific authors as well as its critics. These elements were aggregated and synthesized into a set of core conditions that serve as a form of Weberian ideal type, against which agenda-setting measurement models can be assessed more objectively in terms of their

success in maximizing internal and external validity. The second step involved a meta-

analysis of agenda-setting research published in major communications and public opinion research journals, isolating and focusing on the methodologies used, and evaluating the designs in order to observe broader patterns and trends in methodological development

within agenda-setting. The first step is the subject of this chapter, while second step is the

subject of the following chapter.

The critical review is organized around agenda-setting's core definition: that of

a process by which the media agenda of issues, through a transfer of salience, influences

the public agenda of issues. McCombs (2004, p.5) suggests that the fundamental

distillation of the agenda-setting is this three-part process, outlined in Figure 1-1. This

model is a useful starting point in providing basic categories from which to examine how

agenda-setting researchers set out to model key terms such as "media agenda", "transfer of

salience" and "public agenda" within their design. Because agenda setting is purportedly a

dynamic, causal relationship, time is the critical, additional fourth element that is evaluated.

2 Similarly, Weiss (1992) also suggests that the "theoretical prerequisites to the agenda-setting concept" can be reduced to three questions: a) "What is defined and measured as media information? b) What are defined and measured as audience cognitions? [and] c) What type of influence is to be seen as an agenda-setting effect of the mass media?" (p. 376) 28

Figure 1-1: Basic Agenda Setting Model

U

As noted earlier, agenda-setting's focus on public opinion as the dependent

variable has arguably been at the expense of adequately conceptualizing the true functions

of the independent variable (i.e., the media): how it operates, the nature of media content,

the relationship between news producers and consumers, and how all relate to the processes

by which public opinion is formed. A reason for this underconceptualization of the media

as an independent variable, as Brosius and Kepplinger (1990) observed, is that most

agenda-setting studies must make use of existing poll data in which questions cannot be

altered to suit the needs of the media content. And yet these core assumptions are

fundamental if, as the general hypothesis of agenda setting states, media exert a

determinable influence on the public salience of issues and concerns.

One core assumption is that of an audience's potential exposure to media. If a transfer of salience from the media to the audience is to be determined, then the population from which the sample of respondents is to be drawn must have the opportunity to be exposed to the content. The question of potential exposure has been raised previously by both developers and critics of agenda-setting models. McCombs has argued that media is

so pervasive that exposure is not a critical variable for understanding agenda-setting effects (1981b), but Shoemaker and Reese (1990) argue that determining exposure is fundamental to linking media content with media effect, and that much more effort must be made by researchers to identify factors relating to exposure. Winter (1979) highlighted that early studies failed to identify the specific source of media information and whether the respondent had actually been exposed to the content. As Kosicki (1993, p. 107) argued,

"researchers should present evidence that the people alleged to be affected have, in fact, been exposed to the content." McQuail (2000, p. 455) also argued that agenda setting must provide "some indication of relevant media use by the public concerned. Such data have rarely if ever, been produced at the same time in support of the hypothesis of agenda setting." McCombs and Shaw (1980), addressing this and other criticisms of agenda- setting research, also suggested that media use and level of exposure be examined among a set of contingent conditions.

Ensuring a respondent's potential exposure to the media sample would appear to be a self-evident element given the agenda-setting hypothesis. An example would be the design of the initial 1968 study, in which Maxwell McCombs and Donald Shaw a priori determined that the news media that respondents in Chapel Hill, North Carolina, would view would be four local/regional newspapers, as well as the New York Times, Time,

Newsweek, and two of the three U.S. national evening television news broadcasts.

However, many agenda-setting studies eschew this test, and instead employ a sampling strategy that selects a proxy media sample believed to either represent or influence other news media; for example, selecting the Globe and Mail as representing the "national media" in Canada. Research models employing this sampling strategy do not test for what 30 media respondents are exposed to, but rather rely on an often untested assumption that the

agenda of issues drawn from a media proxy represents the type, selection and agenda of

issues from all potential news outlets.

An inherent problem with this sampling strategy is the representativeness of a

set media sample to an audience diminishes as the geographic scope of the study increases.

Within a study of limited geographic scope (such as the Chapel Hill study), the number of

local, regional and media outlets that respondents can be potentially exposed to is

comparatively limited. As the geographic scope increases, so too does the number of local

media that respondents typically consume. Such a gap almost unavoidably occurs when the public being surveyed is drawn from a national sample—indeed, the use of national population samples in agenda-setting studies raises important methodological issues. In

U.S.-based studies (that comprise the bulk of research into agenda setting, see Chapter 2 below), the national media proxy often selected by researchers has been the New York

Times, and/or nationally televised newscasts in the United States obtained from Vanderbilt

University's national television archives. Evidence in support of selecting the New York

Times has come from a 1971 study by Les Brown that indicated a relationship at that time between the New York Times and U.S. network newscasts (Winter & Eyal, 1981;

Gonzenbach, 1992, Winter, Eyal & Rogers, 1982).3 In Winter and Eyal's 1981

3 A similar finding was made a decade later by Massing (1984), cited by Ader (1995) for selecting the New York Times. Eleven of the 32 U.S.-based studies used the New York Times within their sample. Wanta and Hu (1993) offer a common variant on the selection of New York Times for such studies chosen for the study because, "besides having a national circulation, elite media such as the Times may have an agenda-setting effect on smaller media that could show up in survey data of a cross section of the population. Thus, while the vast majority of respondents in the Gallup polls were unlikely to have been exposed to the Times, they may have been exposed to other media that had taken salience cues from coverage in the Times." (p. 255) longitudinal study of the changing importance of civil rights issues over time relative to media coverage that relied on New York Times, researchers succinctly explained the arguments behind the use of such proxies, stating that the Times "is considered the most prestigious national newspaper, it was thought that the Times coverage would be indicative of national media coverage on this issue, and that it would be an adequate estimate of media content for comparison with national opinion polls." (103) While recognizing it as an "imprecise if relatively workable" approach, Winter and Eyal argued in support of using elite media as proxies by citing much earlier newspaper research that indicated that: a) top newspapers (including the Times) provided the same basic account of a story (Breen,

1968); b) the prevalence of wire-copy in major newspapers tends to restrict variation in news subjects and content, particularly in non-elite news outlets; and c) the "arterial effect" that news tends to flow down from elite newspapers to non-elite news media (Breen, 1968;

Crouse, 1972). Similarly, a number of studies have chosen to focus exclusively on broadcast coverage to the exclusion of print, often citing two similar rationales: that historically public opinion polls have suggested that respondents choose television news as their primary source of information (Robinson & Levy, 1986); and that citizens may view television as more credible than print (Watts, Mazza & Snyder, 1993). Another form of proxy sampling has been the use of only the first or lead item offered by a broadcast news outlet or only front-page stories from newspapers. The use of the first news item on broadcast outlets has been adopted by several researchers based on the conclusion by Behr and Iyengar (1985) that the lead item had the strongest impact. In short, agenda-setting research designs often employ a media sampling strategy that selects a media proxy based 32 on an argument of simple face validity, by citing past (and often very dated) media research, or citing past agenda-setting research practices. All these practices raise questions about research design validity in the context of the true exposure of the media sample to respondents.

In designing media samples for agenda-setting research, another (and arguably stronger) strategy of selecting or excluding certain media outlets in the experiment can be through the use of media usage questions designed to determine what constitutes the respondents' primary sources of information. For example, Palmgreen and Clarke (1977) asked respondents which of four types of media (newspapers, television, magazines, radio) was "the most useful." Consumption questions allow the researcher to delve further into how much respondents use different sources of media, and which types of information they may be seeking out. In this manner, media usage questions allow a level of refinement as to the correct selection of information sources, as well as providing an understanding of the relative degree of importance respondents place on different outlets to determine intermedia effects at an individual level, as discussed below.

A number of researchers have argued that while important, media exposure cannot be equated with actual attention or reception to media. The exposure of an audience to media content is normally produced through self-reporting questions about patterns of personal media consumption. Typically, these questions probe about the general frequency by which the respondents gathers news ("How often would you read a newspaper each week?"), or the source of information ("Which television news programs would you be most likely to watch during a typical week?"). Such questions designed to determine audience exposure may be important, but questions designed to determine audience attention have shown to account for more variance than survey questions about exposure

(Chaffee & Schleuder, 1986). In order to differentiate exposure from actual attention, media effects researchers have included questions that probe a respondent's actual awareness and knowledge of recent media stories. Questions designed to probe audience attention tend to probe the respondent's actual awareness of a media-generated event or knowledge of an issue prevalent in the news. Price and Zaller (1993) argued that researchers who rely only on media exposure understate the true impact of news on public opinion by not providing alternative survey measures that explore who in fact understands or 'gets' the news. One approach used by Price and Zaller was to insert questions about a current news topic in the survey and ask open-ended questions concerning basic knowledge about the topic to confirm their exposure and understanding of the topic. Of course, these types of questions about knowledge of and/or attention to current events require that they be inserted into the questionnaire prior to the survey being submitted to respondents.

While media consumption and attention questions are useful, determining the media sample requires a further exercise of unpacking the concept into its primary dimensions that frame or outline the potential population of media, just as establishing a representative sample of respondents requires an a priori understanding of the demographic dimensions of the general population to be sampled. The internal validity of the media 34 sample relative to the total population of media available to be sampled can be assessed

along three dimensions: temporal, geographic and type.4

The temporal dimension recognizes that news is continuous: that in any given

24-hour period there is an ebb and flow of news and information and the size and

distribution of audiences consuming it. News organizations issue continual updates as to the top stories and issues of the day, and that rate of update varies depending on the media

outlet. A daily newspaper is issued once a day, mainstream television outlets normally broadcast news four times a day, radio often broadcasts news reports hourly, while web- based news sources are updated on an irregular but nonetheless continuous basis. At the

same time, the size of each news outlet's audience varies at each interval in time. The total

population of potential media to sample for a chosen media outlet therefore would include

all potential updates. From this flow, agenda-setting researchers typically choose to sample

at intervals that are deemed to have the greatest impact due to the relative size of known

audiences, such as selecting only the supperhour newscast of a national or local television

station, and exclude other broadcasts. In the meta-analysis discussed below of agenda-

setting studies published over the last 25 years, 33 of 50 studies included national evening

newscasts in their sample.

As it concerns television, selecting the late-evening national newscasts for major

television outlets may still be a preferred sampling approach, but it is becoming more

4 Time lag and the cumulative effects of news and information can be considered a fourth dimension, but time lag effects in agenda-setting research design concern both the public and media sampling procedures, and are discussed below as a separate element of consideration within the entire research design. problematic due to the influence of other sources of news in which the stream of news is more constant. Historically, choosing major evening national newscasts for major national television networks ignored the influence of other newscasts (morning, noonhour) that may also exert influence, and in Canada, often have significant audiences for programs such as

CTV's Canada AM, or CBC News' Today program at noon.5 Moreover, relative to the more oft-used national evening newscasts, morning and noonhour newscasts can vary considerably by what issues they select, or even within the same issue, how they may treat it, given the nature of the news format and the perceived composition and interests of their audiences. Morning newscasts often include more human interest content and attention to personal health and entertainment stories that often do not correspond with the current events that dominate later evening newscasts.

A more important, recent development, however, has been the growing prevalence of television channels that provide updates to the news agenda at any time of the day. The emergence of 24-hour cable news shows, such as Fox News, MSNBC and

CNN in the United States, and CTV Newsnet and CBC Newsworld in Canada, introduce a unique format that provides a constantly varying stream of news and information delivered in different formats. Sampling at a specific hour or half-hour time frame may capture the

"top stories" of the day, but unlike a mainstream outlet's flagship newscast, may not reflect the news events that receive the most overall attention from the 24-hour news station and,

5 Indeed, even restricting the sample to include only regular news broadcasts ignores the issues-related content on news magazine programs such as CTV's W5 or CBC's the fifth estate, daytime programming that often includes issue commentary, such as radio call-in shows or factual entertainment programs such as CBC's The Gill Deacon Show. Soroka 36 as a consequence, what viewers (more broadly distributed across the news cycle) are seeing as top issues. Similarly, the top stories on web-based platforms of mainstream news organizations are also constantly updated—sampling at a specific time frame may not capture the true rank-order prominence of stories across a 24-hour news cycle in the same definitive way that examining a newspaper or an evening national newscast can have.

There are many potential consequences to the understanding of news media effects introduced by the advent of 24-hour news sources that are beyond the scope of this paper, but in terms of sampling procedures for agenda-setting research designs, the chosen interval for sampling within the known flow of news must be considered. However, to date, and for whatever reason, the 24-hour news channels are largely absent from the study of agenda-setting effects. In a survey of over 110 agenda-setting studies published in major journals between 1983 (three years after the launch of CNN) and 2008, only three (Roberts,

1992; Wanta, Golan & Lee, 2004; Craft & Wanta, 2004) included a 24-hour news channel

(CNN).

The geographic dimension recognizes that the various news media outlets accessible to any one respondent will range in terms of the proximity and the scope of news they cover. Normally, a respondent would have access simultaneously to news sources that, while possibly offering information that is a mix of local, regional, national and international in origin, would tend to give greater weight to one geographic area (e.g., a community newspaper's focus on local issues) relative to other areas. While the type of

(2000b) suggests that popular entertainment programming, including films, can radically alter the importance that the public may view certain issues. questions and issues under investigation by a researcher might warrant isolating the news sample to a certain area (e.g., an agenda-setting study that questions respondents about top local issues, and therefore only focuses on local media), if no such limit is warranted, then geographical limits placed on the research model may pose a weakness to the design.

While there has been little research into the varying effects of the temporal flow of news on the media agenda-setting effect, several researchers have probed the difference in local compared to national news media in examining agenda-setting effects. However, the area that has received the most attention has been the final dimension: media type.

Historically, agenda-setting studies have probed two media: print (primarily newspapers, but also news magazines, particularly in earlier U.S. studies when the impact of national magazines such as Time and Newsweek was believed to be more important), and television.6 Throughout the history of the paradigm, researchers have explored the relative

impact of television versus print sources on the agenda-setting function (McClure &

Patterson, 1976; McCombs, 1977; Salwen, 1988; Wanta & Hu, 1994; Winter & Eyal,

1981). Wanta (1997) did observe two differences in comparing newspaper and television

coverage over time along two of three dimensions: (a) regarding time lags, national newscasts tended to produce agenda-setting effects much earlier than newspapers and that national media generally have shorter time-lags than local/regional news media; and that

(b) accumulated learning and decay of effects tended to be stronger (more learning, less

6 Few if any of the agenda-setting studies published in leading journals have examined the impact of radio. A keyword search of Communication Abstracts. Communications Studies and Communication & Mass Media Index revealed only five published studies in which "radio" and "agenda setting" were cited in the abstract, compared to 75 for television, 60 for newspapers and ten for the Internet. decay) for respondents exposed to print news than television. On whether there is a difference in the magnitude of effects between print and television, however, Wanta observed in his review of intermedia effects (1997, p. 144) that "previous research ... is inconclusive regarding the processing of information among print and broadcast consumers." Similarly, drawing on over thirty-five years of experience in reviewing and participating in agenda-setting studies, McCombs' own reflection (2004, p. 48) was that half the time there is no difference, and in the other half, there appears to be a slight edge to newspapers. McCombs credited this to the potentially greater "channel capacity" of newspapers, offering more information and probing more deeply into a wider range of issues, to perform an initiating role in the public opinion process. Television, on the other hand, was more akin to a newspaper's front page in that it identifies issues already deemed important. Given the mixed results from past research, excluding either print or broadcast media in any agenda-setting study would be a weakness, not only as it relates to determining important questions about relative intermedia effects (as has been raised repeatedly in previous studies), but as importantly, the potential multiplier effects that may occur when both print and broadcast give equally high prominence to a single story at the same time.

Recently, agenda-setting studies have begun to explore the implications of the rising use of the internet as a conduit for news and information. The Internet currently offers news through three primary sources: (a) formal news portals, operated by mainstream news organizations (e.g., CBC's cbc.ca, or the Globe and Mail's globeandmail.com), (b) news portals operated by aggregators (Google, Yahoo) or major Internet service providers, such as Quebecor's CANOE site or Bell Canada's Sympatico portal that depend on content primarily from mainstream or traditional news sources; or (c) blogs, podcasts, and other informal, social media sources maintained by individuals or non- news media organizations made available only through the Internet. Individuals access internet news sources either directly through the portal, or indirectly through a customized composite news feed, often using RSS (Really Simple Syndication) programs that deliver new news from Internet sources that have been previously identified and tagged as sites of

interest.

The Internet as a source of news is undoubtedly growing in importance, and its emergence has equally important methodological as well as theoretical implications for agenda-setting research designs. One critical implication is whether the comparatively transient and fragmentary nature of Internet news sources relative to traditional news

outlets erodes or undermines the agenda-setting function of the media. Althaus and

Tewksbury (2002) conducted laboratory experiments that found a weaker correlation regarding agenda-setting effects among groups exposed to Internet news versions of the

New York Times compared to the print versions, although it was higher than among the control group that lacked exposure to either platform. Takeshita (2005, pp. 287-8), however, theorized that several characteristics of current Internet usage do not support the

argument that the Internet erodes the media's agenda-setting function, particularly: (a) that media businesses seeking economies of scale will simply combine media entities, providing the same prominence and news/issue selection, but through different platforms;

(b) the media imperative of seeking as wide an audience as possible will remain intact across traditional and non-traditional platforms, which will tend to restrict the news selected by the media towards similar stories and events; and (c) net users will tend to access only a few sites—a fact borne out by net rating surveys. As such, Takeshita recommended that a few 'hub' sites that research demonstrates are popular sources of current news and information should be included in future research. Market research in this area in Canada (NADBank Winter 2005) supports this position, in that rising readership of online versions or a combination of online and printed versions (from 9% in

2001 to 14% in 2004) has mostly compensated for declining readership among print versions (from 74% in 2001 to 66% in 2004), such that total weekly readership has remained relatively constant. A similar finding was observed by the Canadian Internet

Project in 2007, which found that the increase in Canadian internet usage was not at the

expense of use of traditional media, even among younger Canadians, and that behavior is

evolving towards a more concurrent use of traditional and new media sources. (115) While

empirical research into the impact of the Internet within agenda-setting scholarship is

indeed limited (and is much less relevant to research prior to the Millennium), the

Internet's rising significance to both audiences and to media organizations cannot be

ignored, and must be considered in future agenda-setting studies.

A final assumption of agenda setting is, of course, that a definable "agenda"

exists. Given that respondents are likely to be exposed to several media sources, it raises

the question as to the level of consistency or intermedia agreement that is present and observable within a sample of media outlets. Wanta (1997) remarked that national news outlets are more likely to focus on international and national news issues compared to local papers. Regional variations can also make the determination of a media agenda problematic. Soroka (2000a) questioned whether in Canada, given the presence of French and English-language media, a truly 'national' media agenda can be determined.

Nonetheless, he found a high level of consistency among disparate major regional papers in issue selection, allowing him to use an aggregate of leading newspapers that would constitute a 'national' media agenda suitable for agenda-setting research purposes. The original McCombs and Shaw study in 1968 also determined a high level of consistency in the selection and priority of issues between different media chosen, as did the subsequent

study conducted in 1972 (Shaw and McCombs, 1977). a

The transfer of salience is the core dynamic element of agenda setting: the process by which cues surrounding issues and/or attributes are identified, selected and finally communicated by one group to another. Agenda-setting research has articulated a theory as to why such a transfer of salience occurs, primarily focusing on psychological

factors surrounding the central concept of the individual's need for orientation in order to develop cognitive reference points about more and less important issues in order to function in a public, civil society (Weaver, 1977; Roessler & Schenk, 2000). According to

Weaver (1991, p. 132), the need for orientation assumes that each individual feels a need to understand and be familiar with their physical and mental environment. It is based on

Tolman's idea (1948) of cognitive mapping: individuals strive to map their world in order to orient themselves within it. Operating elements behind this central idea of need for orientation include relevance (the feeling that the issue has more or less direct personal relevance to the individual), and uncertainty (the relative feeling of whether the individual possesses all the information they require on an issue).

While the psychological explanation of the need for orientation is important to the broader understanding as to why a transfer of salience occurs, it sheds little light as to how it occurs, particularly in explicating the role of the media within the salience process.

This question of the relative role of the media compared to the individual respondent in examining salience is another weakness in the literature that has been recognized by other authors. Kiousis (2004) commented on the lack of research examining how to operationalize media salience. While salience is usually determined by 'cues' offered by the media, the determination of what constitutes media cues is often conceptualized in a very rudimentary manner in many media agenda-setting research models. At its most basic, these models employ a simple frequency of stories: counting the number of times the object (i.e., issue) appears in the media sample. While frequency is important, it would

appear to be a minimum standard of establishing salience.

A more refined approach of establishing media salience would be to further examine the influence of various external 'cues' inherent in the presentation of an issue or object. These cues would include such basic elements as prominence (the position of

stories in newspapers, or the rank order of items in newscasts), the length of the story, and the use of pictures and other graphics to draw greater attention to a news item. A combination of frequency of coverage weighted by the presence or absence of these external factors might better differentiate the media content in a manner that parallels the emphasis placed on different news stories by news organizations, and thereby better approximates the probability that an audience member was exposed to and/or influenced by a story (Watts, Mazza & Snyder, 1993). Manheim (1996) labeled this combination of prominence and frequency as "visibility".

Several studies have concluded that including news prominence factors can contribute to higher recorded salience with the public (Behr & Iyengar, 1985; Dearing &

Rogers, 1996; Graber, 1984). Wanta (1997) indicated that including pictures and then- relative size affected salience. Hill (1985) found evidence that prior print media exposure to a story enhanced agenda-setting effects among television viewers, suggesting that another salience cue is the presence of intermedia coverage of the same issue (Reese & Danielian,

1989). Similarly, Kiousis (2004) argued that the presence of stories in certain 'elite' media may also constitute a salience cue of media prominence. Kiousis (2004) further suggested that prominence may not account for significant variance relative to the use of frequency or other contingent conditions related to media salience, but nonetheless argued that since his results were exploratory, prominence cues "should not be dismissed as a key dimension of media salience." (p. 83)

The above discussion focusing on the salience of issues and objects is described by researchers in this field as the "first level" of agenda setting (Ghanem, 1997;

McCombs & Estrada, 1997) that comprised the bulk of early research into media agenda- setting effects. First-level agenda setting is distinguished from "second level" agenda setting that has emerged more recently as a subject of academic research, and which focuses on the agenda of issue attributes rather than the actual issues themselves.7 In second-level agenda setting, issues reviewed from the perspective of their attributes, and public opinion as the dependent variable is reviewed not from the agenda of issues, but rather as it corresponds to a set of pre-determined attributes that parallel the issues. Issue

attributes, according to McCombs (2004, p. 70), are to be viewed broadly as

"encompassing the entire range of properties and traits that characterize an object." An

attribute can be viewed as a characteristic, property or quality that describes an object

(McCombs & Ghanem, 2001). Attribute agenda-setting is viewed as a natural extension of

Lippmann's suggestion of the media's ability to create "pictures in our heads" (McCombs

& Estrada, 1997), and has become more common in the study of media agenda-setting

effects, particularly involving political campaigns (Becker & McCombs, 1978; Weaver,

Graber, McCombs & Eyal, 1981; McCombs, Lopez-Escobar & Llamas, 2001; Golan &

Wanta, 2001; Scheafer, 2007). Moreover, Ghanem (1996; 1997) contended that second-

level affective attributes also contribute to higher salience of objects examined at the first

level of agenda setting by generating compelling arguments that affect public accessibility

to the object, regardless of the frequency or other salience cues examined at a first level of

agenda setting.

7 There has been notable discussion about the convergence between frame analysis and second-level, attribute agenda-setting approach. McCombs and Ghanem (2001) argued that frame analysis and agenda-setting are theoretically linked, suggesting that attributes can be viewed as existing along a micro-macro scale, in which 'frames' focus at the macro level concerning a special class of attributes that serve as central thematic elements of the text, while attribute agenda-setting tended to focus more at the micro level of individual attributes. Takeshita (2005, p. 280) also argued that frame analysis focused on the substance of a particular issue, while attribute agenda-setting looked at a set of issues, and "rarely looks into how an individual issue is constructed." Other authors, however, have been critical of the suggestion of a strong theoretical link between frame analysis and agenda-setting (Scheufele, 2000) and have highlighted important differences to argue that the two should not be used interchangeably (Maher, 2001). See also Aday (2006). 45

How research models are designed in order to probe for a relationship concerning second-level attribute agenda-setting effects has varied, and have been greatly influenced by the type of object or issue under scrutiny. Ghanem (1996) relied on a four- part questionnaire examining local media coverage of crime issues that looked at four dimensions of attributes: sub-topics, framing mechanisms, affective elements and cognitive elements. Subsequent research has tended to focus on the latter two in discussing agenda-setting attributes.8 Cognitive and affective elements were also the primary dimensions in research conducted by McCombs, Llamas Lopez-Escobar and Rey (1997) that examined how newspapers presented local Spanish political candidates, and in a subsequent evaluation by McCombs, Lopez-Escobar and Llamas (2000) that analyzed both newspaper and television coverage, subdividing it into 'substantive' elements (issue positions, biographical data, perceived qualifications, personality and integrity) and

'affective' dimensions.

The affective element has drawn particular interest and is generally viewed as the tone of the object's portrayal by the media, categorized simply as positive, neutral or negative. Kiousis (2004, p. 83) described it broadly as the object's "valence," but nonetheless cited it as an important element which in his experiment accounted for a significant portion of the variance in the salience of an object, albeit behind factors relating

8 In her discussion, Ghanem equates sub-topics as similar to sub-issues of the main topic, and is preceded within agenda-setting by the work conducted earlier by Benton and Frazier (1976), Atwater, Salwen and Anderson (1985), Salwen (1988). By framing mechanisms, Ghanem draws on framing literature (Entman, 1993; Gamson & Modigliani (1989) to refer to the devices used by the media to draw attention to the news item, such as pictures, pull quotes and other elements - elements that, discussed earlier, are external (what makes a stimuli stand out from other stimuli, as discussed earlier as an element of media prominence), rather than internal (stimuli comes from association with personal relevance prompting a person of how to think about a problem). Cf. Kiousis (2004) for further explanation. to its external visibility. Scheafer (2007) also concluded that the affective dimension concerning the tone of an issue presented in the media contributed significantly not only at the second level of agenda setting, but also reinforced first-level agenda-setting effects.

The transfer of salience in media agenda-setting involves two elements, the media, which (primarily) transmit salience, discussed above, and the public, which are exposed to and assumed to respond to salience cues. Methodologically, how agenda setting determines the salience of issues or attributes among the public has been to use the "most important problem" (MIP) question. The MIP question is a top-of-mind, normally open- ended survey question that originated with the Gallup organization in 1939 that asks respondents the following: "What do you think is the most important problem facing this country today?" Evatt and Ghanem (2001) argued that the initial use of a variation of this

question by McCombs and Shaw in 1972 and the strong results it received established it as a cornerstone for future agenda-setting research in determining public salience. A number of the most oft-cited articles in agenda-setting research (Funkhouser, 1973; McCombs &

Shaw, 1977; Palmgreen & Clarke, 1977; Winter & Eyal, 1981) have used the MIP question or variations as the basis for determining the public's issue salience. Generally, a variation

of open-ended questions and ratings scales has been used repeatedly in agenda-setting

studies.

The reliance on traditional public opinion polling methods and the construction

of the criterion variable has been a subject of criticism by authors who view it as a

simplification of the concept of what is an "issue" held by the public (Edelstein, 1993).

This criticism has in turn triggered efforts for greater refinement and expansion of the 47 measures of public salience beyond the MIP format. One issue, raised by McLeod, Becker and Byrnes (1974) on questionnaire design, was how to measure the difference in perspectives held by respondents between individual or interpersonal salience (measured in the original Chapel Hill study and in the MIP question), community or intrapersonal

salience (salience derived from direct discussion between individuals), and a third perspective that lies in between by communicating through the media what respondents perceive externally as being the most important issue to the larger public—what the authors referred to as the perceived issue salience. Similarly, Evatt and Ghanem (2001) called for a refinement that included tests for different types of salience: personal salience, social

salience, and situational salience. Carter, Stamm and Heinz-Knowles (1992) added a series of questions concerning 'consequentially' in order to determine why respondents ranked

certain issues as either a low or high priority. Kiousis (2000) introduced a further set of

questions utilizing the elaboration-likelihood model (ELM) to suggest that when personal

motivation and information processing are high, the relationship between media coverage

and public opinion should be strong (reflective of how ELM's "central processing"

message creates a strong, positive attitudinal change in a respondent). In summary, as

McCombs (2004, p. 19) noted, further refinements to the question establishing public

salience beyond the MIP has been a characteristic of the methodological advancement of

agenda setting generally. A quality of a stronger research design would be questions that better extrapolate public salience beyond this single, open-ended question.

V 48

By its nature, media agenda-setting is concerned with the link between the media and the formation and expression of public opinion. As such, the public as envisioned within the research design has two roles to play: to serve as audience for the mass media, and to express 'public opinion'. This positivist view of the public's role in agenda-setting research has been subject of much of the same criticism associated with media effects research generally: the metaphorical tightrope that researchers walk in attempting to document significant effects, while at the same time acknowledging that audience members are not, as Neuman (1990, p. 162) colorfully described it, "strapped to the chair in front of the television set with their eyes held open like the anti-hero of A Clockwork Orange."

Proponents of agenda setting, however, have been careful to avoid any prescription that the audience acts automatically to media messages. Audience agency is incorporated within agenda-setting by testing for the limits of media effects in public opinion formation. Media agenda-setting research has approached the question of limits by exploring a wide range of contingent conditions that circumscribe or qualify media effects.

A wide variety of contingent conditions have been explored and, in fact, soon after the initial findings were declared by McCombs and Shaw, much of what McCombs (2004) described as the "second phase" of agenda-setting research was devoted to exploring conditions that would refine and/or proscribe the effect of the media.

Despite the level of research activity in exploring contingent conditions, there has been little effort to provide a concise definition of the concept, let alone categorize this research or place it in an analytical framework useful in order to design consistent tests for 49 the impact of contingent conditions. Nonetheless, several broad categories of contingent conditions have been explored. These include the following:

a) demographics Wanta (1997) contended that despite the long and rich history of studying demographic variables in mass communications research, there has been little attention to demographics within agenda-setting studies, and little is known about their impact on agenda setting despite the potential impact of age and other elements on level of media usage. Shaw and Martin (1992) argued that the media play a role in improving dialogue and consensus among various demographic groups (gender, race and, to a lesser extent, wealth), but did not suggest the converse: that demographics form a contingent condition for understanding agenda-setting effects. The few studies that have examined demographic variables have pointed to one—education—as a contributing and/or filtering factor to the presence of an agenda-setting effect (Wanta, 1997; Hill, 1985).

b) cognitive As discussed earlier as an element that drives the agenda-setting process, the need for orientation is an important cognitive contingent condition. In theory, the individual's need for orientation leads to increased media use, which in turn leads to increased agenda-setting effects (Weaver, 1991, p. 132). A number of studies in agenda setting have tested this core concept as a contingent condition (Huegel, Degenhardt &

Weiss, 1989; Roessler & Schenk, 2000).

A second major cognitive type of contingent condition explored in agenda- setting research models has been issue capacity. Many studies in agenda setting have indicated that respondents as a group can generally identify only a small number of issues

(usually five to seven) that would consistently rank high in importance. Cognitive psychology has been employed by agenda-setting researchers (Iyengar & Kinder, 1987) to explain this restriction: that individuals have limited capacities of perception, memory, recall and, most notably for audiences, attention to external events, and that this will either impose restrictions on the amount of information an individual can process, and/or lead the individual to construct heuristic devices to circumscribe information (Zhu, 1992; McCombs and Zhu, 1995).

c) interpersonal communication Recognizing that other social processes may affect an individual's salience of issues, several studies have investigated the role of interpersonal communication as a contingent condition, albeit often with mixed results.

Wanta and Wu (1992) suggested that interpersonal communication can enhance issue salience among topics that receive significant media exposure, but that it can also increase salience for issues that receive relatively little media attention, with frequency (rather than intensity) of discussion among the best determinants of the effects of interpersonal communication. However, Wanta and Wu's review of the literature (1992, p. 847-8) in which researchers tested for the impact of interpersonal communication found studies where it enhanced the agenda-setting effect, inhibited the effect, or had no measurable impact.

d) prior knowledge Price and Zaller (1993) suggested that prior knowledge, within the context of political communication, is the strongest indicator for news story recall among a variety of contingent factors, including interpersonal communication, education and self-reported media use. However, prior knowledge is seldom considered among various contingent conditions. Part of the problem may be in the difficulties inherent in determining prior knowledge within a research design. In several cases

(Battels, 1993), prior knowledge was ascertained using panel data—an approach only occasionally used in agenda-setting studies.

e) issue obtrusiveness Another area of early investigation was the relationship of personal experience and issue identification. The idea was discussed as early as 1975

(Becker, McCombs, & McLeod, 1975), and in 1978, Zucker observed that the magnitude of the media's agenda-setting effects varied depending on the issue's obtrusiveness. Issues that have a high component of personal experience are viewed as obtrusive, such as taxes; whereas issues that the public would have low direct personal experience, such as foreign affairs or stem cell technology, would be unobtrusive. Researchers usually determine the obtrusiveness of an issue a priori by using deductive reasoning, but several researchers have applied audience-based responses to determine obtrusiveness (Blood, 1982; Einsiedel,

1984). Most studies into this area have held that issues with low obtrusiveness demonstrate a greater reliance on media as the primary source of awareness and understanding of an issue (Zucker, 1978; Eyal, 1979; Winter & Eyal, 1981; Atwater,

Salwen, & Anderson, 1985b; Smith, 1987), although issue obtrusiveness should be viewed as a continuum rather than as a strict dichotomy (Blood, 1981). Of note: not all studies have agreed that obtrusiveness is a sound explanatory factor for agenda-setting. Other authors (Erbring, Goldenberg, & Miller, 1980; Demers, Craff, Choi & Pessin, 1989) have found evidence to support a contrary theory— cognitive priming—positing that personal experience enhances rather than diminishes media effects. In either case, however, the obtrusiveness of an issue has been deemed as an important element to agenda-setting research designs.

A concept related to issue obtrusiveness often examined as a type of contingent condition is the tracking of "real-world conditions:" prevailing circumstances or events that can have a direct impact on respondents and, as a result, an indirect impact on the assessment of a media effect. Real-world conditions have generally been included as a control variable in determining both the effect of media coverage on responses, as well as on the level of media attention devoted to specific issues, and have been demonstrated to provide an independent impetus to public concerns, as well as driving the media agenda

(Behr & Iyengar, 1985; p. 53). One of the most common approaches is to compare the rank importance and/or perceptions of certain economic issues (e.g., employment, overall state of the economy) with fluctuations among nationally-produced indicators such as the gross domestic product or consumer price index (Demers, Craff, Choi, & Pessin, 1989;

Eaton, 1989; Hester & Gibson, 2003; Scheafer, 2007), or public perceptions about crime rates relative to actual changes in the prevalence of crime (Lowry, Nio & Leitner, 2003).

Most agenda-setting research can be divided based on how it approaches the question of time: whether it focuses on a single period of time (cross-sectional), or looks at changes over time (longitudinal). Cross-sectional studies (such as the initial Chapel Hill study) have been criticized for their lack of consideration of time. As Gonzenbach and

Gavin (1997, p. 115) remarked, "agenda setting is by definition a time-related process."

Citing Iyengar and Behr (1985) and Brosius and Kepplinger (1990), Gonzenbach and Gavin rejected a cross-sectional approach because it focuses on a short time frame, and

"cannot account fully for the dynamic nature of the agenda-setting process, nor can it account at all for the time lags of possible effects" (116). An analysis conducted by Brosius and Kepplinger (1990) specifically comparing static and dynamic models indicated that static models were "unsatisfactory" (203).

Even within longitudinal studies, there has been criticism as to how agenda- setting research has specified and theorized time (Kosicki, 1993; McCombs, 2004). Eyal remarked in 1981 that a key problem has been that up to that point, agenda-setting research had failed to determine an appropriate time frame for agenda-setting effects to occur.

However, by the early 1990s, many researchers were beginning to make advancements in using time-series data in agenda-setting research (Gonzenbach & McGavin, 1997).

Researchers agree that a time lag exists, but have reached little consensus as to what would constitute an optimal lag or the conditions under which it would occur. Early research experimented with many different time frames ranging from a six months to a few weeks

(Stone, 1975; Mullins, 1977; Zucker, 1978). Winter & Eyal's 1981 study of civil rights pointed to an optimal time frame of approximately four to six weeks prior to field testing, but that examining the impact of content beyond two months accounted for little of the variance. The conclusion of a one-month to two-month optimal range was also supported in early research (Zucker, 1978) and in subsequent research (Salwen, 1988; Wanta & Hu,

1994). While agenda-setting studies may never determine an exact time frame, previous research has pointed to a period of at least four to six weeks as a minimum to assess agenda-setting effects. Moreover, previous research has underscored the need for researchers that conduct longitudinal studies to assess the impact of various time-lags in determining if agenda-setting effects are or are not occurring. This would require that the evaluation of media content be conducted as close as possible to the exact time that the survey was in the field in order to determine when a time lag would peak and then decline.

A third criticism to emerge about how agenda setting theorizes time arises from how time-series data is handled. In some cases, correlation has been simply a product of

'eye-balling' a graphic representation of the media and public opinion data trended over time. A notable example is Funkhouser (1973) who compared real-world statistics with public opinion during the 1960s on such issues as crime and drug use. Increasingly, however, researchers have incorporated more advanced statistical procedures for handling longitudinal data to identify relationships between media and public agendas. The most common approach is to apply cross-lagged correlations that determine the relative influence of media to public opinion or public opinion to media.9 However, in their review of the use of longitudinal data in agenda-setting methods, Gonzenbach and McGavin

(1997) argued that cross-lagged correlations do not control for three elements important to provide evidence of causality: a) "temporal erosion": when differences arise between the exact point in time that the independent and dependent variables are measured; b) autocorrelation: examining the level of variance in changes over time explained within the variable itself; and c) stationarity, the degree to which the variable would remain unchanged. Certain statistical procedures can be applied to control for one or all of these

9 If prior media coverage (XI) was a stronger cause of subsequent public opinion (Y2) than subsequent media coverage (X2) was of prior public opinion (Yl), then the correlation r(XiY2) would be larger than r(x2yd. 55 elements, including Granger causality (Soroka, 2000a); ARIMA modeling or partial correlation (Winter & Eyal, 1981). It is important to note that Gonzenbach and McGavin concluded by indicating that non-linear tests for causality may be more appropriate for agenda-setting research than the more traditional linear approaches represented by regression models. Therefore, another important methodological test is to determine how time-series data is handled.

D. Validity and agenda-setting research designs

The above review underscores that, through the history of conducting and critiquing agenda-setting research in search of a better understanding of this specific media effect, scholars have regularly identified the elements that make for stronger and weaker research designs. 'Strong' and 'weak,' in truth, have little meaning in empirical research; instead, research designs are assessed on their degree of validity and reliability. The elements identified in the previous discussion address questions of reliability but mostly address issues of instrument validity. As Frey, Botan and Kreps (2000, p. 109) underscore, communication research designs must possess both external and internal validity. The authors defined internal validity as concerning the "accuracy of the conclusions," that is, whether the instrument is designed to produce accurate findings about the phenomena being examined; while external validity addresses the generalizability of the findings to a larger context. (109-10) Questions about the external validity of an agenda-setting research design normally occur at the stage of sample selection, while questions of internal validity focus on the actual questionnaire and content analysis instruments designed to measure the media and public agenda, and ascertain the process by which a transfer of salience (and, 56 accordingly, a relationship) occurs. The concerns about external and internal validity in previous research designs are discussed below.

ft

In constructing the public opinion data within an agenda-setting research design, external validity is (usually) determined through the random selection of a sufficiently large group of respondents from a larger target population. Random selection is important in order that the public agenda obtained has a high statistical probability of reflecting the population mean of the public's agenda, and is based on the principle that, given a sufficient sample size, replacing certain respondents with other randomly-selected respondents would still yield a similar approximation of the public agenda.

Turning to the media side of the research design, however, random selection is not a viable approach in ensuring the sample's external validity. For example, assessing the national media agenda in Canada would not involve taking a random selection from a population of over 100 daily newspapers and thousands of radio and television sites, let alone internet news sources. The principle of substituting one sample for another is not appropriate at the level of selecting media outlets from which to analyze. Instead, as discussed earlier, agenda-setting researchers normally rely on subjective arguments of face validity in showing how the media sample is determined logically or by citing certain anecdotal evidence that would support the argument that the sample is representative of the larger population of media. In short, researchers establish validity by presenting the 57

argument that the sample media selected for the experiment (nm) is a valid representation of the media that would be most influential to the target population (Nm).

Using simple representativeness as a principle to establish sample validity has always been a weakness in establishing a media sample's external validity, but this weakness has, and will, become even more apparent in the future as the expansion of the population of media—the sources of news and information accessible to the public— accelerates due to changes in technology. This inherent weakness could be addressed by employing a more robust and empirically-defensible criteria for determining external sample validity: proximity. Proximity, as used in this context, refers to the relative geographic, temporal and technological distance between the sum of the elements comprising the media sample and the entire universe of media content available to respondents within an experiment. Applying proximity as a principle to evaluate the external validity of the media sample as an independent variable has three important advantages to researchers. First, it introduces a basis by which the face validity of a sample can be empirically and potentially quantifiably evaluated. Different samples can be rated and compared for their level of greater or lesser proximity to the population mean. Second, proximity encapsulates the increasing multi-dimensionality of news media content across space, time and technological platforms. Third, the empirical element provides a level of replicability for future researchers to follow.

The first element of proximity relates to technological platforms or 'channel proximity'. News is delivered across many technological platforms, and respondents now have access to news from internet, television, radio and print sources, non-traditional

(billboards) as well as non-mediated sources such as interpersonal communication.

Channel proximity is first established by estimating the number of individual news platforms available to respondents and determining whether the chosen sample closely or rather distantly represents that universe. Channel proximity reflects both the number and range of channels.

Related to channel proximity is geographic or spatial proximity: the degree to which the outlets and content sampled geographically reflect the consumption patterns of respondents. This element recognizes that the subjects of news content as well as the location of the channels are geographically scalable from local through regional, provincial/state, national and international levels. Issues such as unemployment, health care delivery, or international terrorism may be framed very differently by local versus national media (if covered at all), and would skew the rank-ordering of issues among a public. A sample possessing higher geographical proximity would reflect and incorporate this geographic range.

The third element is temporal proximity—does the news media content sampled occur within the time frame that would be available to respondents, incorporating both the qualities of immediacy (news available immediately prior to questioning respondents), as well as duration (news available over a sufficiently long period of time prior to questioning respondents). High temporal proximity would be news content that followed from the moment immediately following the questioning of respondents and would extend as far 59 back as dictated according to determinable time-lag media effects. Low temporal proximity would occur when there was no coordination between the media and public opinion sample and/or when an insufficient time-lag was allowed between responses and the media sample period.

I

If media is frequently underconceptualized in the design of agenda-setting research, then the process by which the transfer of salience occurs may be even less understood, recalling the famous Sidney Harris cartoon in American Scientist (see Figure 1-

2 below).

Figure 1-2: How some researchers explain the transfer of salience process

"PMK NOU frtOutp fet MOOE Otfuor rttce »?4 step TWO,"

While previous efforts in establishing media samples have revealed a pattern of weakness in ensuring external validity, the operationalizing of the transfer of salience belies a pattern of weakness in ensuring internal validity. Internal validity has several dimensions, but in the context of agenda-setting research design, it mostly pertains to two characteristics: criterion validity and construct validity. Construct validity was first outlined by Cronbach and Meehl (1955) and, as Carmines and Zeller (1979) explained, "is concerned with the extent to which a particular measure relates to other measures consistent with theoretically derived hypotheses concerning the concepts (or constructs) being measured." (23) Construct validity in the research design of media agenda-setting studies normally occurs at the stage when the researcher operationalizes the key theoretical proposition of how the transfer of salience occurs, designing two measurement instruments—one to ascertain the media agenda, the other to ascertain the public agenda— in order to observe a relationship between the two, and the factors that weaken or strengthen the relationship. Criterion validity addresses what elements a researcher would include within the design instrument(s) in order to ensure that they accurately test for the same elements within the phenomenon being observed. In the case of agenda-setting research designs, criterion validity refers to the elements that comprise the design of variables included in both the media content analysis design and public agenda questionnaires intended to represent the media and public agenda of issues.

Insuring stronger internal validity by more effectively operationalizing how the transfer of salience occurs involves dividing the process between the media stage (elements that involve the media communicating news and issues to audiences) and the public opinion stage (elements by which the public reject, receive and/or filter information from the media, as well as other externalities that can influence the effect). At the media stage, salience is, in theory, transferred in several, externally-visible ways discussed above: through repetition of stories; through cues that include photos, prominence, placement, and size/duration; and through less-visible means through second-level attributes inherent in the content, or through an affective dimension in which a degree of tonality may or may not be implied from the text. Over the course of time, agenda-setting research has explored the influence of all of these cues, but in many cases researchers have either omitted them, or explored only one cue—in only a few instances, as discussed in the next chapter, have researchers raised the level by which the transfer of salience is operationalized at the media stage by combining various cues. For the purposes of measurement, a design that includes most or all of these elements would be viewed as a model in which transfer of salience is highly operationalized and, therefore, would have a greater degree of both criterion and construct validity and hence more useful in explaining how an agenda-setting effect occurs.

Related to the tracking of external cues, second-level attributes and affective content is the question of the news item's relative impact. The use of cues and other elements inherent in the content can and has been used by researchers to provide a more sophisticated means of differentiating media content by weighting units of analysis in the media sample, rather than using simple frequencies that treat all content equally. Similarly, weighting data based on expected audience exposure rather than simply external cues would also provide a more sophisticated means of differentiating media content, operationalizing media salience and, thereby, improving in particular the construct validity within the agenda-setting model. This proposition rests on the assumption that news from outlets with larger audiences within the population sampled will have a greater likelihood 62 of influencing issue awareness than news from outlets with small audiences within the sample population.

The second phase of operationalizing the transfer of salience occurs when members of the public receive and incorporate information from the media. At this phase, how the public may or may not filter and assess media content is important to document, and would involve including tests for contingent conditions cited above While not exhaustive, the most oft-cited and documented among the contingent conditions include: a) issue obtrusiveness; b) real-world indicators; and c) demographics. Another contingent condition that may be particularly important is that of media consumption—often understood as the volume of news media consumed regularly, but also can include type of media consumed.

Discussion

In the interests of bringing a degree of coherence in improving the validity of

agenda-setting research concerning a media universe that is becoming increasingly fragmented and technologically-determined, it is worth summarizing the above-noted conditions for establishing internal and external validity in an outline in the tradition of a

Weberian ideal type, as illustrated in Figure 1-3 below.

Figure 1-3: Agenda-setting process defined by stages in developing the research design

1. Raise the degree of external validity of the research design by:

1.1. Explicitly identifying the boundaries of the respondent population to be sampled, and employing a means of randomly selecting respondents from the defined population. 63

1.2. Using an empirically-derived means of determining a priori the entire population of mediated sources of news and information for the selected respondent population.

1.3. Assessing the combined spatial, channel, temporal proximity of the chosen sample of media sources relative to the entire population of sources in order to ensure maximum potential exposure to mediated sources of news and information.

2. Raise the degree of internal validity of the research design by:

2.1. Identifying potential externally visible and internally inherent secondary cues from media sources

that would operationalize the media's role in the transfer of salience process.

2.2. Incorporate cues into the proposed instrument to measure the media agenda.

2.3. Incorporate audience exposure data within the media agenda measurement instrument to differentiate media based on consumption patterns.

2.4. Identify elements that would determine the public agenda of issues and operationalize them within the research design.

2.5. Control for contingent conditions that would affect the transfer of salience and mitigate media influence on the public agenda.

While the above discussion identifies and defines many of the elements common to media agenda-setting research, it does not provide a means of assessing the overall state of agenda-setting methodology as it has developed since McCombs and Shaw's first studied the phenomenon in 1968. To this end, the model presented in Figure 1-3 above provides more than simply a useful heuristic device to understand how the agenda-setting research design process should unfold. The model also provides a basis to assess the overall methodological state of past and current agenda-setting research from the perspective of how it achieves validity, particularly in the context of the treatment of media. To answer these questions, and to provide further understanding about the qualities that increase validity within the research design and the state of agenda-setting methodology in general, a more structured assessment of agenda-setting research in recent years is necessary, and is the subject of meta-analysis outlined in the next chapter. 65

Chapter 2

Meta-analysis of agenda-setting research designs

A. Outlining the meta-analysis of research designs used in agenda-setting

With agenda-setting passing its fortieth anniversary since McCombs and Shaw undertook their study in Chapel Hill, North Carolina, in the summer of 1968, the depth of material now available presents a challenge in making generalizations about the state of the field's methodological development. In order to properly assess the literature and reach actionable conclusions about the qualities of the agenda-setting research design, the following chapter adopts a meta-analytic approach. Meta-analysis, according to Johnson et. al. (2008), has become a common and generally accepted practice in communications research because it can provide the clearest conclusions from a research literature. (312)

The following eschews the narrative approach to meta-analysis and, instead, adopts six of the seven steps outlined by Johnson et. al. in conducting a meta-analysis. The seven steps include: 1) determining the theoretical domain of the literature under consideration and defining the question; 2) setting boundaries for the sample of studies; 3) locating relevant studies; 4) coding studies for their distinctive characteristics; 5) estimating the size of each study's effect on a standardized metric; 6) analyzing the database; and 7) interpreting and 66 presenting the results. As the object of the thesis and the underlying meta-analysis is not to understand and measure the agenda-setting effects determined by the studies, but rather the research design they employ, the meta-analysis does not include the fifth step of estimating the size of each study's effect on a standardized metric.

In order to determine where agenda-setting research would be most frequently discussed and published, the first stage of the meta-analysis determined the theoretical domain of the literature and set valid boundaries for the meta-analytic sample. This first stage began with an initial keyword search for "agenda setting" in eight leading academic journals selected due to their particular focus on quantitative research concerning communications and public opinion. The initial survey included Public Opinion Quarterly,

Communication Research, Journalism and Mass Communication Quarterly, Harvard

International Journal of Press/Politics, European Journal of Communication, Journal of

Communication, Canadian Journal of Communication and the International Journal of

Public Opinion Research. The sample period chosen was an approximately 25-year span between January 1982 and September 2007, which allowed the meta-analysis to encompass the most recent literature, incorporate a long time frame but still exploit the degree of online indexed archival material available using a keyword search. Using Ebscohost,

Canadian Journal of Communications archives, Blackwell, Oxford and Sage online archive search engines, a search for the term "agenda setting" was conducted within the title, abstract and keyword index frames. The initial survey indicated that five journals tended to publish the most articles (110 of the 128 items detected, or 86%) on agenda setting: Public

Opinion Quarterly, Journal of Communication, Communication Research, Journalism and Mass Communication Quarterly, and Communication Research. As these journals tended to publish the most about agenda-setting, they were considered the most representative about the methodological state of the field and were selected for the meta-analysis. The term "agenda setting" has been applied to different models and approaches, but for the purposes of serving the thesis, a filter was applied that corresponded to the original formula behind the agenda-setting process first used by McCombs and Shaw in 1968: namely, that only items that focused on the media-public dimension of agenda-setting (and not subsequent dimensions, such as agenda-building, or the relationship between media and public policy) were included. Moreover, only research items that involved a content analysis of media coverage as well as non-laboratory surveys of public opinion were included in order to ensure the meta-analysis only compared similar methodological approaches. Examples of items not included were discussions about the theory and/or the history of agenda-setting, and items discussing methodological elements of the topic (but which did not conduct an actual experiment). Using this filter, 50 of a potential 110 items were applicable for the meta-analysis. The results of this survey by journal are outlined in

Table 2-1.

Table 2-1 Results of agenda-setting meta-analysis

Name Engine Search Date Initial Total used parameters range survey applicable

Public Opinion Ebscohost "Agenda- Jan-82 9 4 Quarterly setting" in to Sep- abstracts 07

Communication Sage Agenda- Jan-82 15 8 Research setting in title, to Sep- abstracts and 07 keywords

Harvard Sage Agenda- Jan-82 5 n/a 68

International setting in title, to Sep- Journal of abstracts and 07 Press/Politics keywords

European Journal Sage Agenda- Jan-82 8 n/a of Communication setting in title, to Sep- abstracts and 07 keywords

Journal of Oxford Agenda- Jan-82 19 5 Communication setting in title, to Sep- abstracts and 07 keywords

Canadian Journal CJC Agenda- Jan-82 5 n/a of Communication archives setting in title, to Sep- abstracts and 07 keywords

International Oxford Agenda- Jan-89 23 11 Journal of Public setting in title, to Sep- Opinion Research abstracts and 07 keywords

Journalism and Ebscohost Agenda- Jan-83 44 22 Mass setting in title, to Sep- Communication abstracts and 07 Quarterly10 keywords

If the frequency of publication in these journals surveyed is any indication, academic research into agenda-setting remains strong. Of the 110 items surveyed, more than half (58) were published in the latter half of the sample period. In fact, dividing the sample period into quintiles revealed that the greatest frequency of publication on agenda- setting occurred in the last quintile between January 2003 and December 2007 (30 items), which included a special edition of Journal of Communication on the field of agenda setting, priming and framing research, and eight items in Journalism and Mass

Communication Quarterly.

Although the entirety of the published article was considered, the meta-analysis focused on the section of each research article devoted to describing the methodology used

10 Journalism and Mass Communication Quarterly was published as Journalism Quarterly until 1995. by the author(s). An obvious weakness in this approach is that researchers may have included elements in their research designs that were not reported in the articles submitted for publication. Nonetheless, assuming that the elements authors did report were deemed sufficiently important in testing their hypotheses and to their final results, and that the intent of documenting such methods in research articles is to allow a level of replicability for future researchers to follow their work, using the articles' explanation of methodology as a basis of assessing research designs was deemed appropriate. In addition to bibliographic variables as well as variables identifying statistical tests used, the coding schematic applied in the meta-analysis was divided into six sets of questions pertaining to: a) how was the element of time treated in terms of both the use of cross-sectional versus longitudinal models; b) what was the composition of the sample of media outlets chosen by researchers; c) what was the unit of analysis and what differentiation was made between media types; d) did researchers account for contingent conditions, and what types of conditions were examined and how were they treated; e) how was issue salience determined among respondents; f) what allowance was made to further probe for secondary as well as primary agenda-setting effects; and, g) whether the researchers employed appropriate temporal lags in assessing the effects of media content on respondents within a set time period. The full coding schematic is included in Appendix A. Coding for the meta-analysis was conducted by the researcher.

The elements of the survey were intended to probe three specific dimensions specifically concerning how the media sample and the media's issue agenda was generated, how the transfer of salience was operationalized within the research design, and how the public's agenda of issues was determined. The first dimension concerned contingent conditions, by which researchers controlled for variations in media effects by examining the impact of issue obtrusiveness, demographic variables, media usage, real-world conditions and interpersonal communications. The second dimension concerned how researchers defined the transfer of salience between the media and respondents, using visual cues as well as other elements of the text, such as affective characteristics and second-level attributes of the issue. Finally, introducing a new dimension rarely discussed by agenda-setting researchers, the review examines the quality of proximity discussed earlier, defined spatially and temporally, between audiences and the available media content. Prior to examining each of these three dimensions, the meta-analysis begins with several basic research elements: the composition of the media samples employed, the selection of a unit of analysis, sample size, and the use of time.

B. The consideration of time: Cross-sectional versus longitudinally designed studies.

Almost half (46%) of the studies in the meta-analysis sample were cross- sectional in nature. The number of time periods sampled ranged from one time period to

121 distinct periods. Among studies with more than one time period, the mean was 31 sample periods and a median of 25. Almost 64% included five time periods or fewer for review, which undermined any attempt to establish evidence of causation by applying a cross-lagged correlation analysis or other statistical techniques in handling longitudinal data. Among the most sophisticated were the studies conducted by Brosius and

Kepplinger (1990,1992, 1995) that involved 53 separate, weekly time periods with 1000 71 respondents per wave, with a content analysis sample of over 16,000 news items. With this level of sample size, frequency of sampling period and volume of news items, the researchers were able to detect the presence of an agenda-setting effect as well as hypothesize about causation by examining the data dynamically; when the researchers examined the same data using a static, cross-sectional approach, no agenda-setting effect was observed. (1990)

C. Media sample validity and proximity

As discussed, media sample validity is best evaluated based on a principle of that media's proximity to the sample of respondents, determined geographically, temporally, and technologically. The spatial, temporal and channel dimensions of proximity were derived from other measurements within the meta-analysis, including the number of media outlets chosen, the range of channels, composition of local and national, sampling methods, use of a proxy based on prominence, and degree of time lag incorporated.

The number of media outlets chosen by agenda-setting researchers was generally few. While the original McCombs and Shaw study in 1968 examined nine outlets that ranged in terms of type (print and broadcast) and locale (local and national), subsequent studies have, on average, been smaller and more confined in terms of type and locale. In the meta-analysis, the number of media outlets surveyed ranged from 1 to 26, and while the average was 5.1 outlets, 44% of the authors examined three outlets or less, and 72% examined four outlets or less (the mode for the sample was four outlets). 72

In terms of channel selection, researchers' choice of media outlets skewed noticeably towards broadcast more than print. Broadcast comprised 76% of the news items, and national television newscasts—comprising the biggest media segment—were examined in 74% of all research articles surveyed. The extensive use of national television likely reflected a perceived greater influence of national television news, but may also have reflected the ease of access to the content of U.S. national network newscasts provided by

Vanderbilt University's television news archive. Several articles from German researchers such as Brosius and Kepplinger cited above also used data from a similar corporate television news archive. One-third of all studies included at least one of the national evening newscasts from the three large U.S. television networks (ABC, CBS and NBC).

Local television news—often more difficult to obtain for research purposes—was used less frequently in only 24% of the sample. Print media were used less often than broadcast, appearing in 52% of the sample. Researchers again tended to resort to national print media

(44%) rather than local print media (22%) in conducting their studies.

Other media channels that serve as sources of news content were generally ignored: less than 10% of researchers considered news magazines, while radio, internet news sources and advertisements were virtually absent. The absence of the internet as a news source was not surprising given the historical nature of the meta-analytic sample, but it nonetheless raises important considerations about future research design that are discussed later.

In terms of procedures for media sampling, approximately 32% of articles (16 of the 50 surveyed) employed a sampling method that restricted the coverage within the outlet 73 surveyed, while the remainder generally included all applicable news items available within the outlet at the prescribed time. The most frequent sampling method was categorical sampling—more specifically, using the restriction of either the newspaper's front page or the lead item on television newscasts—which comprised 10 of the 16 studies that employed a sampling method. Six studies engaged in another strategy of categorical sampling of the media by sampling only on specific days, usually selecting every nth day.

To better gauge the level of proximity of the media samples chosen, a simple four-point ranking system was employed to rank the proximity of the media sample as selected by the researcher(s) to the respondents being surveyed:

• frResearcher(s) selected one type of media, and surveyed fewer than four outlets within that medium, and would not encapsulate most of the media available to potential respondents. For example, researcher(s) selected one newspaper (e.g., The New York Times) for a national survey, but omitted other newspapers and sources from other media.

• IResearcher(s) selected more than two outlets that would conceivably encapsulate most of the news coverage available to potential respondents within that medium, but restricted the sample to only one type of media. For example, selecting all major television newscasts (ABC, NBC, CBS), but omitting print coverage.

• $ Researcher(s) selected more than one type of media within the sample, but did not attempt to encapsulate most of the news media within the selected channels that would be available to potential respondents. This would include samples that contained some national print and broadcast outlets, but in both cases, would be relatively few in number.

• jResearcher(s) selected more than one type of media within the sample in an effort to encapsulate most of the news content that would be available to potential respondents, normally that would include a mix of both national and local media outlets. 74

Figure 2-1 Proximity rankings of media samples for agenda-setting research designs, by national versus local media samples

20

18 -

16

14 •

12

| 10 -

8

6 -

4 2 • 0 Low Somewhat low Somewhat high High

• National • Local Both

The meta-analytic survey, suggested that proximity was negatively skewed towards a low-to-somewhat low proximity to potential respondents, as illustrated in Figure

2-2, but at least eleven of the 50 studies surveyed ranked high,11 and a further eleven studies ranked somewhat high in proximity.

Timing of media sample linked to public opinion polling One of the most consistent elements used by researchers was how they timed the execution of the media evaluation portion of the research design relative to the public opinion field research

11 Asp (1983), Williams, Shapiro and Cutbirth (1983), Eaton (1989), Salwen and Matera (1992), Roberts (1992), Wanta & Wu (1994a & b), and Meijer & Kleinnijenhuis (2006). Three research reports within the sample from McCombs, Llamas and Lopez-Escobar published between 1997 and 2000 all involved the same study of coverage in Pamplona, Spain, and scored very high in terms of proximity for their use of national and local print and television, including political advertising. element. At some level, almost 80% of the articles did link the timing of the content analysis with the timing of the field component of the public opinion research, and of the nine items that did not link the content analysis with the poll, all were the result of the use of third-party public opinion data, in which the information about the timing of the opinion poll may not have been available to the researcher(s).

While the decision to coordinate the media content analysis with the implementation of the poll was relatively consistent, there was considerable range in time lag of the media component. Of items in which the time span of the media content analysis was defined, the most frequently used time frame was two weeks, and 49% of the distribution used 30 days or less in their analysis. The selection of an optimal time lag has been found to depend greatly on the type of media selected. According to Wanta and Hu

(1994), a 30-day window would suffice for the more immediate impact of broadcast news coverage. Nonetheless, a number of researchers probing the question of optimal time lag effects (Stone, 1975; Mullins, 1977; Zucker, 1978; Winter & Eyal, 1981; Salwen, 1988) have reported optimal lengths at greater than one month, suggesting that the use of 30 days or less by almost half of the agenda-setting research published in these journals in the last

25 years might preclude that the authors selected a time-lag that was less-than-optimal from which to assess results. Twelve of the 50 studies included a test for optimal time lag in assessing their results. 76

D. Operationalizing the transfer of salience

As noted earlier, the most common method of determining media salience—that is, whether one issue is more 'salient' to the media in question than another issue—is through simply counting the number of items. The most common unit of analysis observed in the meta-analysis was the news article or story, which comprised 80% of the sample, with the remainder employing a paragraph or statement, visual image or shot, or other more refined units than the article.12

Differentiation among units of analysis based on media type or other variables was rare. Within the sample, 56% employed only one type of media, so differentiation was only possible within a media group. Within the 22 studies that did examine more than one type of media, over half (13) did not differentiate between print and broadcast, while a

further seven studies did differentiate media in their analysis and findings, but weighted print and broadcast equally. Only two studies (Eaton, 1989; Williams, Shapiro & Cutbirth,

1983) offered a differentiation between media that also included differential weighting.

In the meta-analytic sample, only 20% of the research items identified external

media cues—a percentage that was largely unchanged between the latter half of the sample

period and the first half. Among the research studies that did track cues, there was little

consistency: the size of the article, the position of the item within the newspaper or

newscast, and the use of other cues such as photographs or graphical images, were all used.

12 The actual size of the media samples among the studies evaluated varied considerably, with a range of 35 to 25,170, with a mean of 5713, and a median of 2451. Fourteen of the 50 studies reviewed did not indicate the size of the media sample. Tracking placement and prominence was largely redundant in at least nine items as the author(s) employed a categorical sampling method that involved selecting only items that were placed prominently (front page or within the top range of items on an electronic newscast).

The detection of cues and the differentiation among units of analysis raises the question of using external cues or other factors to weight the media data. The majority of researchers did not weight their data, and simply used a frequency of mention in measuring news coverage. However, at least 29% of the research articles employed a formula that served to both differentiate and weight different news items. In five cases (Atwater,

Salwen and Anderson, 1985; Salwen, 1988; Eaton, 1989; Ader, 1995; Meijer &

Kleinnijenhuis, 2006), the cues were used as part of a formula by which to weight each relevant news item. As a basis for weighting media coverage, most researchers chose a formula that combined various cues denoting external salience, such as size/duration of item. Only three researchers (7% of sample) employed a method that was based on an estimated audience reach of the news items. Eaton (1989) and Kim, Scheufele and

Shanahan (2002) used self-response questions concerning media consumption to provide relative weights to the media coverage surveyed. Chan (1999) provided a relative weight based on the circulation of the three different Hong Kong newspapers examined for coverage of environmental issues. However, the process of actually differentiating media reports based on the varying sizes of their audiences was relatively rare, and a search beyond the confines of the current meta-analysis indicated few other studies in I communications research that have weighted data by estimated audience reach.

More recent research into agenda-setting has raised other complex elements inherent within the text as possible signifiers of salience, including affective dimensions and potential "second-level" agenda-setting attributes. The meta-analysis revealed that 13 of the 50 studies (26%) included second-level attributes in their methodology, and that 14 studies (28%) included a test for affective elements. In fact, during the latter half of the meta-analytic sampling period, 44% of articles included second-level attribute agenda- setting elements. Moreover, eight studies combined both affective elements and second- level attributes, as in several cases the affective and second-level attribute dimension were closely related, particularly in the assessment of political candidates.

Salience, of course, has two dimensions as it concerns agenda-setting: the media dimension discussed above, and the public opinion dimension. Determining salience among respondents has frequently been determined through the use of a single central question, most often the "most important problem" (MIP) question: "What do you think is the most important problem facing this country today?" McCombs and Shaw adopted a slight variation of this question in the initial Chapel Hill study, and because of the strong results they obtained, Evatt and Ghanem (2001) argue that this established the MIP as a cornerstone for future agenda-setting research in determining public salience. The

13 An exception is Gosta, C., Dahlberg, A., & Rosengren, K. (1981). Mass media content, political opinions, and social change: Sweden, 1967-1974. In K. Rosengren (Ed.), Advances in Content Analysis. Ed. Karl Erik Rosengren (pp. 227-240). Sage Annual Reviews of Communication Research Beverly Hills, CA: Sage. The authors weighted coverage by circulation in comparing real-world cues, public opinion and newspaper coverage in Sweden. 79 widespread usage of the MIP question was observed in the meta-analytic study, with 28 of the 50 studies using a single, MlP-style, open-ended central question directed at the individual level. Other studies (Wanta & Wu, 1992; Miller & Wanta, 1996; Sohn 1994;

Salwen & Matera 1992; Meijer & Kleinnijenhuis, 2006) tended to eschew the open-ended format and apply a prescribed list of issues in which respondents were asked to rank on a scale, normally a Likert scale.

E. Controlling for contingent conditions

The test for contingent conditions as control variables was among the first areas examined by researchers and remains a notable methodological element. The meta- analysis examined the use of four oft-cited contingent conditions: i) issue obtrusiveness; ii) real-world conditions; iii) audience demographics; and iv) media consumption patterns.14 fc

Only 15% of studies in the meta-analysis considered the question of issue obtrusiveness in differentiating their results. Of the 39 research articles that did not test for issue obtrusiveness, eleven examined only a single issue which, while not making the

14 This is not an exhaustive list of contingent conditions tested by researchers over the course of the last forty years, but rather some of the more common and universally applied conditions explored by researchers. Within the meta- analysis, other contingent conditions were also examined, such as country affiliation in a cross-country survey (Peter, 2003), party identification in determining voting patterns (Scheafer & Weimann, 2005), and company affiliation in examining corporate reputation (Meijer & Kleinnijenhuis, 2006). However, these above-noted contingent conditions were generally unique to that specific study's hypothesis. A more general contingent condition not assessed is interpersonal communication. Wanta and Wu (1992) suggest that interpersonal communication can both enhance issue salience among topics that receive significant media exposure, but that it can also increase salience for issues that receive relatively little media attention, with frequency (rather than intensity) of discussion among the best determinants of the effects of interpersonal communication. Nonetheless, while interpersonal communications may affect media agenda-setting, there is little consensus as to the direction. The meta-analysis indicated few examples of testing for interpersonal communication of issues despite hypothesizing in several cases a re-examination of "two-step flow" (Brosius & Weimann, 1996). question of obtrusiveness altogether moot in reaching conclusions (four research articles that focused on a single issue did include a discussion of issue obtrusiveness), it would preclude a differentiation of agenda-setting results by the type of issue under examination. ft

While not always germane to the study of agenda-setting effects, real-world conditions were only tested against media coverage and public opinion responses in 20% of the research articles sampled. Normally, testing for real world indicators was comprised of large, publicly-available indicator such as a consumer confidence indices (Demers, Craff,

Choi, Pessin, 1989; Hester & Gibson, 2003), FBI crime statistics (Lowry, Nio & Leitner,

2003), EPA environmental records (Eaton, 1989; Ader, 1995) and gross domestic product

(Scheafer & Weimann, 2005;, Scheafer, 2007).

Only thirteen (28%) of the studies included demographics as control variables, and at least three did not use or discuss the data from these variables in their results, while several other studies controlled for demographic variables, but did not discuss their results.

The meta-analysis also revealed mixed results concerning the use of demographics.

Roberts (1992) found that controlling for gender and partisanship helped to improve the predictability of a two-step model of agenda-setting concerning voter behavior. Other studies (Sohn, 1984; Miller, 1996 [controlling for race]; Peter & de Vreese, 2003

[controlling for gender, age, education or work status]), however, found no significant difference. Overall, the most noted effect was education (Hill, 1985; Roesller and Schenk,

2000). One of the most detailed studies controlling for demographic groups—Shaw and 81

Martin's follow-up paper on the original Chapel Hill study published in 1992— found that with the partial exception of education, dividing respondents solely by age, income and gender had little effect, but that controlling demographics by media consumption (see below) indicated a significant building of consensus over the rank-order of issues among demographics groups. Similar findings were examined and reported by Lopez-Escobar,

Llamas and McCombs (1998) looking at both first and second-level agenda-setting effects, who concluded that agenda-setting, viewed from this perspective, "offers a useful perspective.. .for examining the influence of news on the public mind." (347)

PI

Media consumption is of fundamental importance to the cognitive psychological underpinnings of the agenda-setting process. As explained by its leading researchers such as McCombs (2004, pp. 53-60) and Weaver (1991,1997) the evidence of agenda-setting among certain groups of a population is a reflection of the need for orientation among different members of the public. Given this importance, one of the most interesting findings of the meta-analysis was the general lack of questions concerning media consumption and/or exposure. Typically this involved asking the respondent how many days per week they would read a newspaper, and how many days they watched news on television, and subsequently dividing respondents into high/low respondents.15 Sixteen studies (35%) included some form of self-response media usage question. There was no

15 A variant was Eaton (1989, p. 944), which applied a relative weighting to content based on media grouping (TV: .462; newspapers: .308; weekly news magazines: .231) based on a previous survey of "college-educated people," but did not include an actual media usage question in his study. 82 indication that the use of such questions increased over the course of the twenty-five year sample period, with no change between the first and second half of the sample periods.

Arguably a factor behind the lack of media usage questions was the frequent use by the authors of public opinion data either obtained from third-party suppliers, such as Gallup, or from historic archives (or both), which restricted the analysis to whatever data was available. Fifty-eight per cent the articles surveyed used public opinion data that originated from a third-party source other than from the author(s). However, it should be noted that even when researchers were in more control of the design of the instrument testing public opinion, only 56% included and/or incorporated a media usage question within their framework, compared to 25% who relied on media usage questions available from data provided by a third-party source.

What makes the general lack of media usage questions even more noticeable was the consistent, significant impact that the questions had on the findings in the studies that did include such questions. Asp (1983, p. 345) found that newspaper readers with a high exposure to political content tended to be in greater agreement with the agenda of their own party press than newspaper readers who reported low exposure. Roesller and Schenk

(2000) observed that media effects were more pronounced among respondents that reported higher levels of media exposure. McCombs, Lopez-Escobar and Llamas (2000, p. 87) found stronger attribute agenda-setting effects when controlling for exposure to news outlets that respondents stated they watched the most. Schoenbach and Semetko (1992, p.

846) observed that simple exposure to political news coverage was more strongly related to

issue salience than actual interest in politics expressed by respondents. An analysis 83 conducted by Wanta and Hu (1994) examining three audience attributes—perceived credibility of the news media, reliance on news media for information, and actual exposure to news media—supported a path analysis approach that showed that media exposure plays the fundamental role in determining the intensity of the agenda-setting effect. Hill (1985) observed that prior exposure to news content (particularly in newspapers) was the strongest predictor of agenda-setting effects among various viewer characteristics surveyed. The particular influence of the data from the media usage questions in determining more nuanced aspects of the agenda-setting process would lead to the conclusion that media usage is a critical element in such studies.

F. Evaluating media sampling, transfer of media salience and contingent factors within a single index

The final stage of the meta-analytic review was to produce a single index that evaluated each research article based on a combination of the three dimensions described

above. There were three outcomes to indexing the research articles. First, the overall objective of the meta-analytic study was to provide a useful quantitative measure by which other agenda-setting scholars may assess their own research designs. Second, a single

index would best indicate which studies ranked highest in the index and what particular methodological elements they incorporated that tended to be lacking in other studies.

Finally, an index provides a more valid basis from which to indicate broader patterns and trends concerning the use of media and methodological rigor. The index is composed of a four-part scoring system applied to the three dimensions: testing for media proximity, operationalizing the transfer of salience process, and controlling for contingent conditions.

1*

1.1. Maximizing the proximity of the media sample relative to respondents

1.2. Avoiding proxy/random sampling

1.3. Applying appropriate time-lags between the media and public opinion samples

2»

2.1. At minimum, employing a central MlP-style question to determine public issue salience

2.2. Tracking and incorporating visual cues into the analysis

2.3. Examining secondary traits of content and responses

2.4. Weighting media data based on factors other than frequency

3»

3.1. Testing the influence of demographic differences

3.2. Differentiating between obtrusive and non-obtrusiveness issues

3.3. Evaluating the influence of externalities (real-world conditions)

3.4. Determining media consumption patterns

In ten of the eleven elements, the coding was based simply on the methodological element used by the researcher and published in the article, which was coded as present=l/0=not present. Missing data was treated as not present=0. The component concerning applicability of a sufficient time lag (1.3), used a range of 0 (no 85 time lag used, content not coordinated with timing of public opinion instrument), 0.5 (time lag used, but shorter than the 30 days recommended), and 1.00 (time lag used 30 days or longer). The component measuring proximity of media (1.1) used the same range of

.25/.50/.75/1.00 noted above. The scores were combined and then divided by 11 to generate an index score of 0 thru 1. The results were divided into two groups: T=1 (N=27 cross-sectional studies involving a single sampling period), and T>1 (N=23 longitudinal studies incorporating more than one sample period).

Figure 2-2 Distribution of agenda-setting research articles by methodological index

Score

Overall, the meta-analysis produced a mean score of 0.38 among the research articles, and a median score of 0.34, with a range between 0.15 and 0.67. There was little difference in the score or variance in distribution between the cross-sectional studies

(mean=0.39, std. dev. 0.15) and the longitudinal studies (mean=0.37, std. dev.=0.13).

The review also showed no particular change in scores over time that might suggest a development in methodology based on this scale over time. Scores were only marginally higher during the latter half of the sample period (0.40) than in the first half

(0.36). This was partially due to the fact that the test for secondary agenda-setting effects by examining traits within the content was a development that occurred in the latter half of the sample period. Of all twelve factors tested, only this factor of secondary traits showed a significant correlation with the time variable (r=.37, p<.05).

The studies that ranked highest in terms of the scoring among the more valued longitudinal studies included Schoenbach & Semetko, 1992; Eaton, 1989; Watt, Mazza &

Snyder, 1993; Roberts, 1992; Scheafer, 2007; and Ader, 1995. Of note is that the top three studies were more cognizant of using media cues as elements within their study.

Schoenbach and Semetko included print and broadcast media, and differentiated between the two, highlighting differences in audience reach as well as media consumption patterns and variation in media cues, such as use of film and pictures in television newscasts, concluding that the tone of coverage of environmental issues in certain outlets (the tabloid

Bild) affected public salience of environmental issues. Eaton included both print and broadcast media and weighted data based on certain cues. Watt, Mazza & Snyder also weighted data based on audience reach.

It should be noted that, while not included in the meta-analysis, the original 1972

McCombs and Shaw article would have obtained a score of 0.58 using this scale, which would rank well above the mean among cross-sectional studies. The design of the original study was strong along one of the three dimensions—the validity of the media sample— which combined a range of channels and blended local and national news sources, while providing an appropriate time lag for the content analysis relative to the implementation of the poll. However, the original design contained few elements that would shed light on how the transfer of salience was operationalized by adding tests for media salience other than issue's frequency of mention, and offered very basic tests regarding contingent

conditions involving media consumption and demographics (party affiliation).

Nonetheless, the research design of the original study should have provided researchers

following McCombs and Shaw with a model to improve on the original research design by

continuing to use a media sample with high validity, but focusing on improving elements of the other two dimensions.

Given that the McCombs and Shaw study scored above the mean among studies published ten years later or more would raise the deeper question of what factors might have contributed to this apparent lack of methodological development. One factor may be

that many researchers exploring the agenda-setting effect subsequent to McCombs and

Shaw did not have the degree of control over the research design as McCombs and Shaw

exercised in their seminal study and, instead, employed research designs that were

proscribed by the third-party data available to them. As noted earlier, the skew in the

results towards U.S. national television news broadcasts, while arguably a logical choice

due to their considerable audience reach and influence, was also likely influenced by their

availability to U.S. researchers through the archive maintained by Vanderbilt University. Even more evident in the evolution of the research design has been the influence of third- party public opinion data. Twenty-nine of the fifty studies evaluated used public opinion data obtained from a third-party rather than generating it within their experiment. The use of third-party public opinion data was much more prevalent among the longitudinal studies

(21 of 27) that would offer richer data regarding the agenda-setting effect than among the cross-sectional experiments (8 of 24).

The use of third-party data has afforded agenda-setting researchers certain useful advantages in understanding the effect, principally the ability to examine historical events as well as evaluating longer time frames (Winter & Eyal, 1981; McCombs & Zhu, 1995).

Nonetheless, it must be conceded that the use of third-party data relative to results generated by researchers also imposes important limits on overall research design and, consequently, on what can and cannot be understood about the agenda-setting relationship between the public and the media. First, evaluation of the media's top issues will likely always be conducted a posteriori to determining the public's top issues, rather than in conjunction with the evaluation of public opinion. This fact can influence the structure of key components of the media sample, including selection of media outlets, the time lags incorporated by the researcher in evaluating media content, and the selection and definition of issues. Second, the lack of crucial information within the data set obtained by third- party researchers can introduce important qualifications to a researcher's findings, such as the presence of audience attributes. For example, Soroka (2002, p. 13) notes that the nature of most longitudinal data available is often not equipped to deal with audience 89 attributes. In this case, the evolution of the research model in agenda-setting has led to certain limitations likely not intended by McCombs and Shaw when they first began.

Another element of concern about the overall methodological development of the agenda-setting research design illuminated by the meta-analysis is the disparity between the selection and number of media outlets chosen relative to the reality of the number of

choices for information consumers now have. While not a definitive test of the condition, the average number of outlets used by researchers during the twenty-five year sample

period actually decreased each successive five-year interval, rising marginally during the

last five years, as illustrated in Figure 2-3 below.

Figure 2-3 Average number of media outlets surveyed per agenda-setting study 7.00 6.33 -a >u- 6.00 >

3 0 4.00 nj TD 01 3.00 £ o c 2.00 0O) 1.00 <

0.00 (M oi r>» oo at oi o o o> o o «H «a>H at (oN 90

The almost exponential expansion in the number of news sources due to technological and market changes raises very important theoretical issues about the agenda- setting model overall, and the capacity of the mass media to project, and audiences to become aware of, a coherent, observable agenda of issues. Selecting a relevant, valid media sample forty years ago was a far simpler task than it is today. At the advent of agenda-setting research forty years ago, the choices available to national news consumers would have been relatively limited: the local daily newspaper and community weekly, a few local radio stations available on the AM and FM frequencies, a few, predominant national television channels, and a variety of magazines and other sources. Citizens now have an abundance of sources from which to source out local, regional and national news, with both an almost exponential increase in television news sources, several on a 24-hour basis, internet-sourced news from mainstream and social media, digital satellite radio, and national satellite printing of national newspapers, to name the most prominent.

Agenda-setting researchers have only begun to examine this question. Shaw and

Hamm in 1997 warned about "agenda fragmentation:" that the advent of new technologies will lead audiences to shift to new individualized sources of information, and that there is a spiral effect as like-minded, single-issue individuals use new consumer-driven technologies seek out those with similar views and agendas, and turn less to mass-mediated forms of news and information. However, Takeshita (2005) suggests that while the risk of agenda fragmentation certainly exists and is an important element to add to new research agendas, there are countervailing factors (consumers' preference for general news, likelihood that certain internet hubs will come to dominate) working against this trend. For example, while choice in news services is now more abundant, the time available by citizens to devote to news consumption is limited and, as a result, certain media sources, by virtue of resources, reputation, brand recognition and other factors, still retain large audiences and exert considerable influence. Moreover, many traditional news outlets are adapting to new technologies by incorporating streaming video, websites and RSS feeds that simply shift largely the same content on to other platforms, which slow or halt the erosion of their traditional audience base. For example, while evidence suggests that daily newspaper readership is in a more precipitous decline in the United States, in Canada, the decline has been less pronounced and, when online audiences of newspaper websites are included, has been shown to be marginal. Despite such countervailing tendencies, the rising number of media sources puts direct pressure not only on the theoretical premise of agenda-setting, but also on the methodologies behind it, requiring an ever-increasing range of outlets in order to meet basic standards of sample validity, and a more 21st century research model that reflects this reality.

G. Defining and constructing the media-centric agenda-setting research model

The results of the agenda-setting meta-analysis suggested that the transfer of salience process is generally underconceptualized within the research design, and that the scope of media chosen by researchers to sample has been limited—a flaw made more acute by the emergence of new information technologies. At the same time, the exercise also pointed to the elements that would be required to construct a more valid research design intended to achieve a higher level of proximity between respondents and the chosen media 92 sample, that more fully operationalizes how salience is transferred, and that meets the rising number of media sources available to the public. Given its focus on the media side of the equation in assessing agenda-setting effects, this approach is labeled as the media- centric model.

The media-centric model for agenda-setting research is defined by three notable characteristics. First, the design of the media sample is guided by how the public generally

(and the respondents within the survey portion study specifically) obtain, use and consume news media. The media sample is selected with the objective of achieving a high level of proximity between respondents and all potential media they would be exposed to, which can be empirically evaluated and justified along temporal, spatial and technological lines.

Second, there is a clear emphasis within the research design in explicating the news media's role in the transfer of salience—not by viewing the media and media content as monolithic and undifferentiated, but rather as highly differentiated. Media is differentiated based on cues and other elements in which content from different channels is presented to the public, and in how the public may or may not consume or filter different content.

Third, the media-centric model envisages a process by which the media and public opinion components of the design are developed in tandem by the researcher, rather than having the media component proscribed or otherwise limited by the a priori structure of public opinion data obtained from a third-party source. In short, the media-centric model places greater emphasis than many previously-developed research designs that, while appropriate in detecting the presence and, in many cases, the nature of the agenda-setting effect, may have been ill-suited to determine the more specific role that media and different media content play in increasing or diminishing the agenda-setting effect detected among the public sampled.

Given this definition of the media-centric model, there are several elements that would be required in an agenda-setting research design to achieve it. Starting with the development of the media sample, the first step would be to take an assessment of the universe of news media available to a chosen target population from which the sample of respondents would be drawn in order to a achieve a high degree of temporal, spatial and technological proximity. At least four major news channels (television, radio, newspapers and Internet) should be selected for sampling, with the selection of outlets based on known or estimated audience consumption of the main outlets within each channel. The assessment should consider the temporal flow of news from each outlet. Media content should be obtained up to the point in time in which respondents are probed regarding the public agenda of issues, and there should be a sufficient lag period from the opinion survey period(s) to allow for any cumulative effects of media content to be identified, arguably up to at least 90 days. Moreover, different lag periods should be examined to determine the relative differences in lag periods by issue.

The second step would involve how salience is operationalized within the research design. The content analysis portion should allow for the documentation of various explicit media cues (prominence and placement of the news story, use of photos), as well as the affective dimension (if any) present in a story and any potential secondary agenda-setting effects. The public opinion polling portion of the study should allow for a detailed understanding of which news media channels are preferred, and how frequently the channels are consulted for news content. The design should weight individual media content based on a combination of media consumption questions as well as the external or affective/secondary cues in order to fully differentiate media channels and content based on prominence, frequency, consumption and other elements. In order to test for the filtering effects of contingent conditions, the survey should include and review the impact of demographic variables, particularly education and media consumption, and include where applicable the influence of real-world indicators as well as the nature of the issue as obtrusive or unobtrusive to the individual.

A third important element is the definition of issue salience within both the media content and public opinion sampling phases of the research. Unlike studies in which the public opinion data, obtained from a third-party, dictates the definition and framing of issues identified in the media content portion of the study, the definitions and structure by which issues are delineated within the evaluation of media content should be constructed in tandem with, and correspond to, the definitions and structure of issues identified within the sampling of public opinion. The public opinion portion of the study would also benefit from a more fully developed method of identifying issues salience than relying solely on the MIP question, possibly by adding a secondary MIP question, differentiating for respondents between local and national issues, and adding close-ended questions about the rank-order importance of certain top issues identified within the media sample.

The final element is the treatment of time. Critical to the media-centric model would be an evaluation of the relationship between media and public agendas across time, rather than a simple cross-sectional analysis. For how long should the model be implemented? While longitudinal approaches are preferred over static or cross-sectional designs, there is little consensus to suggest a minimum range in which a longitudinal study investigating an agenda-setting effect can be undertaken. Ideally, longer time frames offer two advantages in identifying and explaining agenda-setting effects: i) the opportunity to generate more data points often necessary to run certain statistical tests, such as a Granger test, in order to see whether the media time series is useful in forecasting the public opinion time series; and ii) the opportunity to increase periodicity among the media and public opinion tests. Given this understanding, sample periods conducted within a period of eight

to ten weeks in total (eight to ten sample periods lasting one week) might be considered the minimum at which a longitudinal study might be conducted.

Discussion

The meta-analysis, like the previous literature review, highlighted a number of

areas of methodological weakness in the evolution of agenda-setting research designs,

particularly in how they conceptualize the role of the media.

First, what comprises the media relative to what respondents would see or read

was both limited and skewed. Exactly 72% of researchers chose four or fewer outlets.

Media samples were often biased towards national network television broadcasts—

ostensibly due to the perceived greater impact but arguably due to the easier access

researchers have had in accessing this data. This has resulted in research that has largely

ignored the influence of other media types such as radio and, more critically, the internet.

One-third of studies employed restrictive sampling criteria in choosing content. 96

Ultimately, only 22% of studies showed a high degree of spatial and technological proximity to the respondents chosen in the sample.

Second, how the media agenda is transferred to the public agenda (as was often the focus of most studies) was underconceptualized by treating media content as monolithic and undifferentiated. Research designs often did not include tests to identify elements that differentiate news items, distinguish between print and electronic delivery, determine audience consumption patterns for various media, assess effects by issue type and other factors that would allow researchers to explain what factors may have contributed to, or increased or diminished, the observed agenda-setting effect. Differentiation between media channels was often moot, with only one channel (or even one or two outlets) used in many studies to represent "the media". Only 20% of studies identified external cues inherent in the content. Secondary agenda-setting effects and/or the affective dimension of certain content were probed in approximately one-quarter of the studies. Concerning how contingent conditions alter the agenda-setting effect, few studies included demographic filters or considered the influence of a third, exogenous variable represented by real-world indicators. Only 15% incorporated issue obtrusiveness in their analysis. Most tellingly, only 35% of studies included some form of media consumption question within the study, and only 10% offered a weight based on consumption or some other formula that would further differentiate the impact of certain content.

Finally, despite the critical importance of time as an element to determine the presence of a media effect, two-thirds of the studies were either cross-sectional in design or used too few sampling periods to determine a longitudinal effect. However, there was a generally higher linkage of the media sample to the public opinion sample, and many studies did incorporate longer time-lags in evaluating a cumulative impact of media content over time.

While there has been considerable expansion of our understanding of the agenda-setting effect, there appears to have been much less scrutiny paid to the underlying research design behind the model. In fact, in evaluating research designs over the most recent twenty-five years, only five studies out of fifty evaluated could arguably be considered to have a stronger research design than that first employed by McCombs and

Shaw forty years ago.16 While it is beyond the scope of this study to determine why there has been little advancement in the underlying research design, one characteristic observed in the meta-analysis was the frequent use of third-party data, particularly for the public opinion component that, over time, has arguably influenced the direction of research in this area—imposing restrictions on the type of outlets that could be chosen researchers, the timing of analyses, and the framing of the media questionnaire. Another more important limitation is that past research designs, focusing mostly on dominant mainstream media, are increasingly open to challenge due to the expansion of news and information sources to the public.

The exercise of the meta-analysis following the literature review provided the basis to outline a possible advancement in the research designs used in evaluating media agenda-setting effects: the media-centric model. The media-centric model, by emphasizing an expanded sample that better approximates the content available to respondents, and its broader range of factors used to explicate the transfer of salience process, addresses a potential deficiency in the past that undermines the overall validity of agenda-setting research, and more importantly tackles the problem of how to deal with the increasing fragmentation of media in the future.

While the following discussion has focused mostly on methodological issues, there are important theoretical questions raised by the discussion to which the media- centric model provides an important contribution. The fundamental hypothesis of media agenda-setting is that the public's exposure to content generated by mass media, through a transfer of salience, produces a common ranking of more and less important issues as a society: that is the media's purported "effect". Such an observation, however, contributes relatively little to an understanding of how the effect occurs. For agenda-setting to offer communications scholars meaningful, critical insights into the relationship between media and audiences, then "media" must be unpacked and refined as a concept within the model itself. If instead media is poorly modeled—limited in geographic and technological scope, monolithic in transmission, undifferentiated in content and ignored in terms of its audience—then little can be said about the implications of the effect, if and when it is observed. The media-centric model, viewed from this perspective, offers the opportunity for researchers to view agenda-setting more as a process, and thus better observe various how different types of media and content more or less influence the observed effect. In

16 Among cross-sectional, they would be Peter 2003; Lopez-Escobar, Llamas & McCombs, 1998; and Atwater, Salwen & Anderson, 1985; and among the longitudinal studies, would be Schoenbach & Semetko, 1992; and Eaton, 1989. short, the importance of the media-centric model is not simply to answer deficiencies observed in past models, but to more effectively observe the media's agenda-setting process within a given research design. 100

Chapter 3 Design and implementation of a media-centric model

While the media-centric model has the potential to better explicate the agenda- setting effect as it relates to mass media influence, it is unproven. As the meta-analysis illustrated, no media agenda-setting study conducted in the last 25 years has implemented a research design that could be considered to reflect all the properties of a media-centric approach. In order to test its efficacy, the following chapter outlines the elements of a media-centric model implemented during the first five months of 2008 in the Waterloo region of southern Ontario. The process by which the media and population sample was

selected, and how actual research instruments were designed, are described in detail so that

future researchers wishing to use or test this model have a full understanding of its

construction.

A. Site selection

The selection of the specific region of Waterloo, rather than a broader national or

even provincial study, was intentional. Due to the requirement of maximizing the

proximity between respondents and available media content, it is an almost necessary

condition in implementing a media-centric research design that its scope be restricted to a

manageable level, as there is an inverse relationship between respondent proximity and the 101 scope of the project. Selecting a national sample, or even a provincial sample, would involve an almost exponential increase in the volume of local news media that would need to be tracked for each respondent. A sampling strategy of selecting targeted population subsamples from across Canada might allow for a national perspective. Nonetheless, at the outset, the necessity of ensuring there is a high level of proximity between respondents and the media content makes the media-centric model very resource-intensive as its scale increases.

There are a number of qualities about the Waterloo region that make it particularly attractive to test the media-centric model, and should be noted carefully by future scholars in selecting a site community. First, the region and the cities of Kitchener,

Cambridge and Waterloo and surrounding townships provide a sufficiently large population to draw a representative sample over time. According the 2006 Canadian census, the Waterloo region has a population of 438,515 and has experienced 9% growth

i -j since the 2001 census, ranking it as the sixth fastest growing community in Canada. The region boasts a mixed economy that includes high-tech companies, led by Research In

Motion, as well as substantial manufacturing and agriculture industries, along with post- secondary institutions such as Wilfred Laurier University and the University of Waterloo— one of Canada's leading research-led universities. Second, the Waterloo region has its own major media outlets that offer local news: its own daily newspaper (the Waterloo Region

17 Census bulletin. http://www.region.waterloo.on.ca/web/Region.nsf/- 97dfc347666efede85256e590071a3d4/0776E1882A72B3DC85256BlB006F8ADB/$file/Bulletin I.pdHopenelement. 102

Record), radio stations, a local television station (CKCO-TV) and local mainstream internet sites for print, radio, and television. Citizens also have access to national and regional news media outlets. Other jurisdictions in Ontario, such as Kingston, lack a complete inventory of news outlets covering local news. Third, for media researchers, the media available to respondents can be more easily defined and isolated than would be the case in choosing a similar-sized community with greater proximity to news from a major metropolitan area. Choosing a market such as Hamilton with its proximity to Toronto and

Toronto-area news could affect the media sample and responses from the sample, making it more difficult to isolate effects from local versus non-local news sources. Residents of the

Waterloo region have access and are known consumers of Toronto-based news outlets, but actual consumption of Toronto-based news is much more limited and, in many cases, is incorporated within the research design, as discussed below. Finally, the homogeneity of the populace has its advantages in testing the model. The region is predominantly English- speaking (92%), with French comprising a further 7%, while German (5%) and Portuguese

(3%) are also languages spoken by a notable minority. As of the 2001 census, visible minorities comprised just over 10% of the local population, predominantly South Asian

(4% of total population).

B. Media sample selection

In implementing a media-centric model, the first step is to identify the news media available to the populace in order to build a media sample with high proximity between respondents and available news. The following outlines the major news outlets available to respondents in the Waterloo region from which the media sample is drawn. The sample is broken down into four components by type: print, television, radio and internet.

Within each media type, the sample of news outlets is chosen based on three criteria presented in each section: a) available news outlets within the media type are identified and described; b) market data about the known audience for the outlets are reviewed; and c) the self-reported media consumption patterns of respondents are presented. With this information, the proximity of media to respondents can be empirically assessed and defended.

First, however, is a note about the sampling period. The sample period for the media content analysis portion of the survey began January 1, 2008, and ended June 7,

2008. The collection of data for the online internet sources began March 1, 2008. The public opinion survey began April 2, 2008. By taking this approach, the model provided data on three months of news media content prior to the implementation of the public opinion survey for mainstream media, and at least one month for internet content. Overall, the experiment was conducted over a 23-week period.

Is a 23-week sample period, with the opportunity to test for effects over a three- month lag, sufficient? As discussed in the first chapter concerning the treatment of time within research designs, there has been considerable range in length of time selected and tested by researchers examining agenda-setting effects. The meta-analysis of media agenda-setting studies indicated an average time lag of 100 days in the studies, although the median was 42 days. In terms of total time devoted to the analysis, the median was 108 days, and the most common was a 28-day period. Given this history, the selection of a 23- 104 week period to test the experiment with a potential three-month lag would fall within the norm of previous studies, but the total sample size of respondents coupled with the ten- week time frame for the public opinion component of the study does allow for a brief examination of the data longitudinally.

f

The Waterloo Region Record is the primary daily newspaper serving the region, and is one of several dailies operating in southern Ontario owned by the TorStar group, others being the nearby Hamilton Spectator and the Guelph Mercury, as well as the

Toronto Star, the country's largest newspaper. According to the NADBank Fall 2007 survey, 61% of adults in the region reported having read either the print or online versions of the Waterloo Region Record over the last seven days within the extended market area that incorporates the Waterloo Regional Municipality. The NADBank survey puts the readership of the print version at 234,200 individuals weekly, while the online version therecord.com attracts a more limited audience of 39,500 readers per week.

Free community weeklies also operate in the market. The Waterloo Chronicle and the Cambridge Times, operated by the Metroland group (a wholly-owned subsidiary of

TorStar), are issued three times weekly and have a claimed unpaid circulation of over

41,000. While the circulation of the community weeklies is hypothetically comparable to

I 8 the Record (paid circulation of 64,485), the fact that they provide fewer issues, less news

18 According to the Audit Bureau of Circulation statement for the publisher for the period between October 1, 2006, and March 31, 2007, published in the September 2007 edition of Canadian Advertiser Rates and Data. content and resources and have a circulation based on unverifiable, unpaid distribution, would suggest that their impact may be much less than the Waterloo Region Record in terms of generating an agenda-setting influence, even at the level of local news and issues.

While print media coverage is largely dominated by the offerings of the TorStar group of newspaper assets, residents also have access to a number of newspapers from outside the municipality. These include the nationally-produced Globe and Mail and the

National Post, as well as the regional Toronto Star and the Toronto Sun and daily newspapers serving nearby areas, such as the Hamilton Spectator, London Free Press,

Guelph Mercury and Stratford Beacon-Herald. According to the NADBank 2007 survey,

15.8% of local respondents said they read the Toronto Star, 14% said they read the Globe and Mail and 7% said they read the National Post, while the highest level of non-local newspaper readership was the Toronto Sun at 20%.

This study's survey of respondents indicated the following when asked which newspaper they had read today or yesterday, outlined in Table 3-1. The survey probed until the respondent had listed all newspapers.

Table 3-1 Newspaper readership by respondents per week

Newspaper 3 or 1 or 2 Total % Total more times a times a week week KW Record 836 159 995 48.4% Cambridge Times 190 83 273 13.3% Globe arid Mail 196 27 223 10.9% Toronto Star 123 25 148 7.2% Toronto Sun 101 27 128 6.2% National Post 78 13 91 4.4% Waterloo Chronicle 24 6 30 1.5% New York Times 10 1 11 0.5% 106

Guelph Mercury 8 2 10 0.5% Hamilton Spectator 6 3 9 0.4% Elmira Independent 7 1 8 0.4% Ayr News 4 2 6 0.3% Other 25 4 27 1.3%

Similar to the market research data, the study's survey indicated that the

Waterloo Region Record was the dominant newspaper, with notable readership for the community weekly Cambridge Times as well as the four Toronto-based dailies. Other newspapers in the region, as well as foreign titles such as the New York Times, were evident but minor at less than 2% of the sample.

For the purposes of the content analysis portion of the survey, the final sample selected was all coverage in the Waterloo Region Record contained within the front, local and op-ed sections of the newspaper, and the front pages of the Globe and Mail, Toronto

Star and National Post. Sunday editions of the Toronto Star were excluded. The

Cambridge Times and the Toronto Sun were also excluded—the former due to restrictions in obtaining a reliable supply of the community weekly, as well as the lower actual news hole provided by the newspaper relative to other newspapers surveyed and the above-cited concerns about actual audience reach, while the Toronto Sun was rejected due to its tabloid format and, again, certain supply restrictions. The decision to limit the sample to the front- pages of the non-local newspapers was again due to limited resources, that the market research and polling data suggested that they would have less of an impact, and that the front-pages would likely have the greatest agenda-setting influence. Excluded from the sample were column briefs (brief news items, usually a paragraph only, contained in digest sections dedicated to national or local news), index references (references to stories on 107

inside pages outlined in indices) and letters to the editor. Nonetheless, the chosen newspaper sample still meets the test of high proximity by providing the main sources of both local and national news content within the channel selected, as determined by market research as well as respondents' own claims. The sample was generated by obtaining full

print copies of the newspapers. This yielded a sample of 6132 print items: 4270 from the

Waterloo Region Record, 580 from the National Post, 111 from the Globe and Mail and

511 from the Toronto Star.

W

Respondents in the Waterloo region have access to a wide variety of televised

sources of news, but the dominant source is the local CTV affiliate CKCO-TV. Historical

market research data19 indicated that over the Monday-thru-Friday noonhour time slot, the

news show CKCO News at Noon commanded a 45% share of the local television market at

that time, compared to CHCH in Hamilton (2%), CITY-TV (2%) and CBC's midday show

(1%), while the remaining audience tuned into non-news and/or non-local programming.

For the supperhour time slot at 6pm, CKCO's market share was 55%—well above other

stations, including the regional Global-TV program (4%), while the late-night news at

11:30pm dominated with a 64% share.

National newscasts available to respondents include the CTV National News at

11:00pm, CBC's The National at 10:00pm, and Global's First National at 5:30pm, as well

19 Market research data for television based on BBM Spring 1999 survey. BBM Bureau of Measurement. (1999). Television - Kitchener-Waterloo/London: Spring 1999. Don Mills, Ontario. While dated, it is assumed that local news broadcasts still command a dominant share of the television market within their time slot. 108 as 24-hour versions available on CTV Newsnet and CBC-Newsworld. CTV's national newscast is the dominant news program with an estimated nightly audience of over 70,000, while CBC's The National has a lower estimated audience of 27,000, and Global's First

National at 21,000.

The size of the audiences indicated by market research is similar to that identified by respondents surveyed. Once again, respondents were asked which television news programs they watched today or yesterday, and were probed continually until they no longer offered any further responses. The results are outlined in Table 3-2.

Table 3-2 Television news consumption by respondents, per week

News program 3 or more 1 or 2 times Total % total times a a week week CKCO News at Six 364 45 409 19.9% CTV National News 314 53 367 17.9% CBC: The National 247 47 294 14.3% CTV Canada AM 144 18 162 7.9% CKCO's Latenight news 129 22 151 7.4% CNN 115 22 137 6.7% CKCO News at Noon 110 8 118 5.7% Global News at 6 96 7 103 5.0% CITY-TV 53 15 68 3.3% Global's First National 63 1 64 3.1% CFTO 31 5 36 1.8% Global latenight 11:00pm 24 1 25 1.2% CHCH 20 2 22 1.1% Global's Morning News 16 2 18 0.9% Global's News at Noon 15 3 18 0.9%

For the media content analysis, the survey chose all of the programs cited by respondents, with the following exceptions: a) CTV Canada AM (problems occurred in receiving reliable summaries of news items from the designated media monitoring 109 supplier); b) CNN (would not contain local or Canadian news); c) CITY-TV, CFTO,

CHCH (would not provide local news for residents of the Waterloo region); and d)

Global's morning and noonhour newscasts, due to low audience exposure. The full newscast was used, with the news item constituting the unit of analysis. Once again, the television sample meets the criteria for high proximity to the respondents as verified through market research data as well as the statements by responses. Of note, however, was the number of respondents that did list CNN among their preferred television news sources. This would potentially influence agenda-setting results as certain issues, such the economy, war and conflict, or the environment, would likely be raised by non-Canadian television outlets that might influence the selection of top issues by respondents, even when asked to limit their responses to top Canadian issues. The total television broadcast sample consisted of 11,706 news items, from CBC-TV (1732), CTV (1467), Global (2220) and the three newscasts of CKCO (6287).

The broadcast sample was comprised of summary monitoring notes made available from Cision Canada, a national media monitoring company. Due to the use of the more brief summary notes rather than actual tapes or transcripts, certain restrictions on the coding of the news item were imposed, discussed below.

ft

As noted earlier, radio is an often overlooked within agenda-setting research, which is surprising given that it remains a very important source of news, particularly for many Canadians. Among the four sources of news surveyed among respondents, radio 110 scored the highest, with 62% stating that they turn to it three or more times a week for news, compared to television (59%), newspapers (53%) and the internet (29%).

A large number of radio stations are available in the Waterloo region. In the survey, the greatest number of respondents selected CHYM-FM, a "lite rock" station that is the region's biggest station by audience, with 21% of respondents saying they listen to the station for news, followed by CFCA (7%). CBC Radio One was identified by 14% of respondents. Both CFRB and CFTR—Toronto-based stations dedicated to news, traffic and weather—were identified by almost 3%. For the media content analysis portion of the study, CBC Radio One—the country's national public broadcaster—was selected, along with CKGL-AM (570News), Waterloo's local all-news station. The choice of CKGL was due to the fact that, while it has a lower audience reach, it provides mostly news content.

The NADBank 2007 survey found that 5% of respondents in the region stated that they listen to CBC Radio, while 6% selected CKGL-AM 570. The survey chose the main news programs from the stations: CBC's The Current at 8:30, The World at Six at 6:00pm and

As It Happens at 6:30pm, while for CKGL, three news slots were selected: news at the top of hour at 8:00am, 12:00pm, and 4:00pm. Again, the selection of top local and national news-format radio stations allows for a high proximity between news coverage selected and what respondents would be exposed to.

Similar to the television sample, the radio sample was obtained from Cision

Canada and consisted of individual summary notes of each newscast. The total radio sample was 4763 items, 1976 from CKGL-AM 570, and 2787 from CBC Radio One's broadcasts. Ill

ft

More than any other technological development in the delivery of news in recent years, the impact of the internet and the seemingly infinite number of ever-changing, ever- ranging sources of news available from the technology puts in question many of the past research findings and assumptions around agenda setting. Future research models into media agenda-setting will either have to include internet as a source, or in some ways qualify their results due to its growing prevalence.

Unlike print, radio and television, accurate, audited information of audience size for internet sites is limited and, when available, quite controversial in terms of appropriate methodologies. Traditional panel-based research designs are frequently criticized as lacking penetration to the various means by which people can now access the internet: mobile phones, laptops accessing the internet remotely at public sites using WiFi, not to mention work-based access, greatly complicates panel-based assessments. Research models based on click-through rates are also faulted for not being able to clearly identify whether the click is actually initiated by a single user, or by technology (web crawlers, for example), as well as its tendency to double-count what would be normally considered a single visit. Such audience metrics are even more difficult to obtain when restricted to a particular region such as Waterloo. Nonetheless, evidence does suggest certain news sites are regularly used. The 2007 NADBank survey found that 78% of respondents indicated they had read a newspaper last week, with 75% responding they had read a print version, while 17% responded they had read an online version. Less than 3% indicated that they 112 only read an online version, suggesting that online versions are supplementing, rather than replacing, traditional forms, at least for the present.

For the Waterloo region, market research (again, the NADBank 2007 survey) indicated that 39,500 residents viewed the online version of the Waterloo Region Record in the previous week, while readership of the globeandmail.com website was 19,500, while figures for the Toronto Star and National Post website versions were lower and below a reliable sample level.

The survey of respondents indicated that both the globeandmail.com and therecord.com sites were popular but, as expected, the sources of news from the internet varied widely: while respondents were asked if they had viewed any news site from major

Canadian media, most ignored the "Canadian restriction" and offered various sites. Again, respondents were probed until they offered no further sites. The results of the top ten sites

are provided in Table 3-3.

Table 3-3 Internet site consumption by respondents, per week

Rank Site 3 or more 1 or 2 Total % Total times a times a week week 1 CBC.ca 84 1 85 4.1% 2 Globeandmail.com 70 8 78 3.8% 3 Yahoo 43 5 48 2.3% 4 MSN Sympatico 34 11 45 2.2% 5 ctvnews.ca 39 2 41 2.0% 6 therecord.com 33 5 38 1.9% 7 thestar.ca 33 2 35 1.7% 8 google news 21 1 22 1.1% 9 nationalpost.com 19 1 20 1.0% 10 thetorontosun.ca 11 1 12 0.6% 113

CBC.ca and the globeandmail.com were the top two sites chosen by respondents, followed by news aggregators Yahoo and MSN Sympatico, then CTV News' website and the Waterloo Region Record site. Due to resource constraints, only two sites were selected for the content analysis: the globeandmail.com site to represent the influence of national news, and therecord.com to reflect local news, thus achieving a higher threshold of proximity. However, the influence of the aggregators and the CBC.CA site should be considered a limit to the media sample. The sample was obtained through a daily download each morning of the front page of the website, with the above-the-banner stories chosen for the analysis. The total sample size obtained from the internet news sites was

641, which comprised of 240 from the globeandmail.com and 401 from therecord.com.

C. Content analysis design and data handling

A complete list of the questions and definitions used in the media content analysis study is provided in Appendix B.

The content analysis questionnaire was divided into four parts: a) primary bibliographic variables (outlet, date, page number/time of broadcast, and type of item [i.e., wire service, staff reporter, editorial, etc.]); b) valence questions (placement/prominence of news item, use of picture/graphic); c) geographic scope of the article (local, provincial, regional, national, U.S., other international, including country); d) main issue/topic; and e) secondary agenda-setting questions on two topics: i) crime (local topic); with questions concerning type of crime and the prominence of crime in the article; and ii) the conflict in 114

Afghanistan (national topic), with questions concerning the prominence of the conflict as well as reference to casualties. The unit of analysis was the individual article.

The media questionnaire was designed by the author and the coding conducted by three individuals with previous experience in coding news media for content. Intercoder reliability tests were conducted prior to the implementation of the media study in order to redefine questions, and then were conducted for the initial sample and at the end of the sampling period, focusing on two variables: topic and scope. The results of the intercoder reliability tests indicated that the data was reliable. The test and results are outlined in

Appendix C.

D. Weighting media results

Critical to the study was the implementation of different weights for the media content. As noted in the meta-analysis, most agenda-setting researchers fail to differentiate news content by either valence or relative audience size and consumption through data weighting, ascribing each unit of analysis a weight of one regardless of the media from which it originates, the degree of prominence ascribed to the news item, or the relative difference in reputation between news outlets. With this approach, a brief buried mention in the Wichita Eagle can have the same weight as a front-page feature report published in the New York Times.

The study assigned four different weights:

• no weight;

• iiweight based on market research data concerning the audience reach of each outlet combined with the valence of the article; 115

• |t weight based solely on the self-response questions concerning media consumption provided in the public opinion survey, and

• pi hybrid combining the valence of the article and the weight based on self-response media consumption.

Valence weightings were applied to print and internet content. For radio and television, the audience for the specific program and/or time slot was used for all news items during that period. For print, the valence responses were combined with market research data segmenting respondents based on the average amount of time spent reading the newspaper. The logic was that while all newspaper readers, regardless of the amount of time spent reading a newspaper, would see a news item with the highest possible valence

(that is, front-page, with photo), while only readers that on average spend longer periods with a newspaper would observe items with low valence. There were several factors used to determine a story's greater or lesser relative valence.

a) Prominence: stories were ranked on each page whether they were prominent

(above the fold, major item on page), or not prominent (below the fold, less

prominent than other stories on the same page). Included in prominence was

whether the story appeared on the front page. Prominence was differentiated

using a formula based on how long people typically spend reading a newspaper.

Based on the NADBank survey, for the four newspapers surveyed, a certain

percentage of readers only spend 20 minutes or less each day reading the

newspaper. Assuming that given this low amount of time spent reading, readers

would most likely observe only the most prominent stories, this segment of

readers was added to the weighting of news items that were prominent, but not to stories that were less prominent. Items that were on the front page but not the

lead story were attributed to all readers except the <20 minutes group. Items that

were prominent but were not on the front page were attributed all readers that

devoted at least 40 minutes a day to a newspaper. Non-prominent items were

weighted by the number of readers who spend at least 60 minutes a day.

b) Placement: the section of the newspaper was taken into account, with audience

demographic data obtained from NADBank used to weight coverage in the more

widely-read front section of a newspaper, relative to the op-ed section or local

section. For example, for the Waterloo Region Record, 75% of respondents read

local news, 60% read provincial/national news, and 36% read the op-ed pages.

Therefore, only 36% of total estimated audience exposure was applied for stories

appearing on the op-ed pages, while local stories garnered 75% of audience

exposure, and national/provincial news 60%.

The formula used to break down valence is provided in Appendix E, which provides both a description of the method and weights applied, as well as programming language using SPSS Weights based on the opinion survey's self-response section on media consumption were based on total number of respondents who stated that they viewed the news outlet and/or program today or yesterday, as outlined in the appendix. While

20 The hybrid self-response weight, along with the market research weight, uses a formula that ascribes a specific value derived from amount of time spent reading a newspaper as well a time spent reading certain sections or 117 market research data is rarely available to most researchers, the self-response data and the hybrid self-response/valence weightings are intended to be more replicable by researchers, as the information is largely based on data that would be obtained from the execution of a media-centric model.

E. Sampling public opinion

For the public opinion component of the study, a random-digit dialed telephone poll of 2054 residents living in the Waterloo Regional Municipality was conducted over a ten-week period between April 2 and June 6, 2008. The poll was put in the field twice a week, Tuesday through Saturday, gathering approximately 100 responses per day, with each weekday sampled at least four times. The poll was restricted to English-speaking adults aged 25 years or over. The restriction of age to 25 and older, rather than the more typical 18 years and older, was done in order to obtain a sample of respondents who generally are known to have a higher consumption of mainstream news; typically, the 18-

24 demographic are among the least frequent consumers of news. The response rate was

9.3% based on total numbers dialed, and averaged approximately 16 minutes to complete.21

The questionnaire was designed by the author, and was encoded for computer-assisted telephone interviewing (CATI) software and placed in the field by an independent, private market research firm in Toronto.

types of news to each level of valence established by the external cues. However, in principle, researchers could skip this step and simply create their own scale that they would ascribe to each level of valence.

21 Response rate is based on a total of 54,231 numbers attempted, of which 25,307 were deemed invalid (faxes, business/non-residential), and 11,657 were viewed as unsolved (busy, no answer, answering machine). A total of 118

The survey was comprised of four parts. The first part tested issue awareness at the national and local level, and was designed to begin with broad, open-ended questions about issue awareness followed by prompted questions concerning certain major issues.

The survey began with the basic Gallup-style MIP question asking respondents the following open-ended question: "In your view, what is the most important national issue facing Canadians today?" Respondents were also asked a follow-up: "If you could choose another important national issue facing Canadians today, what would it be?" Respondents were then asked the same two questions, but asked to identify local rather than national issues of importance. The prompted portion of the survey asked respondents to rank, on a ten-point scale, their level of importance of a series of twelve issues. The issues were selected based on overall level of media coverage during the first three months prior to the implementation of the survey.

The second part of the survey tested awareness of major stories as they emerged in the media. The first set was open-ended, asking respondents to name top national or provincial stories: "Thinking about news coverage you may have seen or heard in the past three months what, in your view, are the three most important national stories?" The second step asked respondents whether they had heard a lot, somewhat or not heard at all a list of ten different national and provincial stories. This list of stories was drawn from the media analysis study and updated each week 24-hours prior to the implementation of the survey. Some questions were asked only once, while some were asked over a period of as

14,590 were non-responsive, mostly due to respondent refusal. Responding units were 2677, of which 622 were disqualified due to age (under 25). 119 many as six weeks. In order to ensure a variety of stories, stories were selected based on

four elements:

• stories deemed to have high and low media exposure;

• stories that received coverage from several types of media, as well as stories that were primarily concentrated in a certain media;

• stories concerning a range of issues;

• national and provincial stories; and • stories with long versus short time frames relative to the point in time of the interviews (that is, stories that occurred recently relative to the interview, or occurred more distantly in the past).

In total, 69 stories were selected over the ten-week survey period: 31 local

stories, and 38 national and provincial stories; although in several cases, certain major

stories were reported by both the local and national media, such as Earth Hour, or the

negotiations leading to the release of Waterloo region resident Jessica Martin from a

Mexican prison.

F. Media consumption and demographics

The fourth battery of questions concerned media consumption by the

respondents. Respondents were asked how often they obtained news from newspapers,

radio, television, the internet and magazines, with responses divided into three groups

(seldom, 1-2 times per week, 3-or-more times per week). For each media type, if they

responded that they at least occasionally consulted it for news, they were asked which

sources they consulted, and were probed continually until they could no longer provide a

response. This structure allowed for a deeper and more thorough understanding of the total news consumption patterns of respondents. Respondents were divided into high and low media consumers based on an aggregate of the responses to the series of media consumption questions. In addition, frequency of discussion of news with friends and family were also asked of respondents. High and low news media consumers within the

sample were identified based on a scoring derived from the media consumption questions

into two sets: high/low, and high/medium/low news consumers.

The fifth and final battery concerned basic demographics used for control purposes, and included education, age, gender and, possibly relevant to attention to local versus non-local news, whether respondents were born in the Waterloo region and how

long they have lived there.

Discussion

Even before the results of the study were examined, there are several aspects

about the implementation of the media-centric model and the patterns of media

consumption and exposure found in the initial survey that deserves mention, as they speak

directly to the efficacy of the model.

A first observation was that the media sample of the study was very large for an

agenda-setting research project. In total, the media sample for the project was comprised of

23,242 items—a very high volume that, based on the meta-analysis, has been surpassed by only one other published study to date: (N=25,170, by Wanta & Wu, 1993). The average media sample evaluated based on the meta-analysis was N=5713. Even adopting a restriction of most prominent items (top five broadcast as well as front-page), would have yielded a sample of over 6000 items. The high sample size is a product of trying to achieve a level of cross-media proximity of outlets relative to respondents and reflects the reality that audiences, even in relatively smaller and restricted markets such as Waterloo, have access to and consume media from an ever increasing array of sources. The fact that sample sizes may need to become larger has implications for future research.

A second observation to be noted is that this researcher was fortunate to have access to a variety of media monitoring sources that made it easier to obtaining certain media, such as radio broadcasts and local television newscasts, which are normally difficult and expensive to secure. This is not a flaw of the model; indeed, the inclusion of these sources, particularly local media, was critical to many of the findings, as outlined in the following chapter. Nonetheless, the reality of the study of media effects is that it takes resources to obtain content from traditional formats; and the less ubiquitous the outlet, the less likely it can be obtained, due to either expense and/or availability.

A third observation is that, of the four media surveyed, the internet proved to be the most problematic for testing media effects, for several reasons. First, it is an almost impossible exercise to identify a priori the potential sources of news from internet sites available to a sample of respondents compared to traditional media sources such as newspapers, television and radio stations. Self-response questions concerning internet news consumption go part way in providing information to determine a proximate sample, but as was shown in the survey, there was no dominant internet source for news as there was observed for television and newspapers, and many sources identified were from aggregators such as Sympatico, or foreign sources, such as CNN. Internet news consumers are not bounded by geographical, channel or other restrictions, and chose a range of sources even though certain mainstream media sites (CBC.CA, globeandmail.com) rose to the top.

Third, as discussed, the measurement of the audiences for internet channels is in its infancy and, to date, there is considerable debate among commercial research organizations such as

Nielsen as to how to effectively measure internet audiences. This lack of audience data makes it more difficult to weight content in the same manner as traditional media. Finally, the results of internet news consumption were relatively low: the globeandmail.com site— one of the biggest in the survey and among the largest of the internet newspaper sites in the country—attracted only 4% of respondents. NADBank data suggests that while growing, the internet audience for dedicated news sites such as globeandmail.com remains a fraction of the audience of the actual print edition of the newspaper, and that there is mounting evidence (2007 Canadian Internet Project) that such sites are being used as a supplement, rather than substitute, for traditional news sources. In summary, it was more difficult to detect and measure the agenda-setting effects of internet news content relative to mainstream media content. 123

Chapter 4 Analyzing the results of the media-centric model

A. Establishing testing criteria: media-centric model versus a non- media-centric model

The basic approach taken in testing the efficacy of the media-centric model was to compare the results it provided against the results of a non-media-centric model under similar circumstances and conditions. This approach is concurrent with common practices in assessing new research models, namely: does the proposed new empirical model provide a better explanation of the underlying phenomenon than the old model. In this case, the former model, the non-media-centric or "representative" model described in the first two chapters, differs in several key qualities from the media-centric model tested.

First, the "representative" model would apply a media sample of weaker validity by virtue of having a low proximity between media sampled and respondents sampled. Second, the

"representative" model would offer a weak operationalization of the process by which salience is transferred between media and the public, normally by not differentiating content by externally visible cues or relative audience exposure, or by factoring in certain filtering, contingent conditions. Given that the media-centric model is a more complex, detailed version of the representative model, it was possible within the Waterloo experiment to assess these two models by constructing a simplified "representative" model within the data provided in implementing the full media-centric model. 124

There were two groups of tests conducted against the model based on the

dimension of media sample validity and operationalization of the transfer of salience.

Concerning media sample validity, the data was analyzed based on a subset of media

outlets that would have low proximity. Typically in past agenda-setting experiments (many

of which have stemmed from the United States), the selection of media has been that which has been considered the most influential, and often using a proxy, such as the front-pages

of the New York Times or the broadcasts of the major U.S. national networks. A similar

sample of lower proximity was used in the following tests: a sample of the front pages of

the Globe and Mail and the National Post, the broadcast items from the national evening

telecasts of CBC-TV's The National and CTV-TV's CTV National News, and a third that

combines the four news outlets into a "national" media sample. This was compared with a

sample of higher proximity which included all the outlets sampled in the Waterloo

experiment.

The second group of tests concerned the operationalization of the transfer of

salience. In this case, a model that was not designed to operationalize how transfer of

salience occurs would ignore factors about media content and respondent usage. All media

data would be unfiltered, unweighted and, as such, undifferentiated. This model would be

in contrast to the media-centric model, which by including such elements as externally

visible cues and media consumption to differentiate media content, suggested factors that

strengthen or weaken the transfer between media and respondents. In this case, several

formulae are offered which combine both external cues offered by the media and media

consumption/audience exposure factors to weight and differentiate media content. 125

Two sets of tests were conducted. The first set of tests ignored time as a variable and examined the data cross-sectionally. Although cross-sectional treatment of agenda- setting data has been criticized because a truer test of a media effect would be one observable over time, many past and current approaches to understanding agenda-setting still use cross-sectional analysis. The second set of tests treats data longitudinally, examining changes across the model over ten, one-week periods, using various time-lag periods from the point of measurement. The framework of the tests is provided in Figure

4-1.

Figure 4-1 Matrix of tests of the media-centric model

Cross-Sectional Longitudinal

Media sample Low proximity High proximity Low proximity High proximity validity • Nat'l. print • All channels • Nat'l. print • All channels • Nat'l. broadcast • Spatial (local • Nat'l. broadcast * Spatial (local • All national and national) • All national and national)

Salience not Salience explicated Salience not Salience explicated transfer of explicated • Weight by explicated • Weight by salience • Unweighted market • Unweighted market • Undifferentiated research/cues • Undifferentiated research/cues • Weight by • Weight by consumption consumption • Weight by • Weight by hybrid hybrid consumption & consumption & cues cues

There are some caveats to this approach in testing the media-centric model.

First, the presence or non-presence of an agenda-setting relationship is declared if a basic non-parametric, rank-order correlation between media and public opinion data sets is observed. Admittedly, this is the most elementary test for a relationship. However, the 126 establishment of a basic rank-order relationship between two agendas remains the most fundamental test for agenda-setting, and not passing this hurdle would make it more difficult to justify more complicated statistical tests. Second, the proximity test creates a national media sample group for its low-proximity test, but draws a respondent group from a local rather than national population. However, with the exception of people living in the Greater Toronto Area that would have a higher likelihood of being exposed to the

Globe and Mail or the National Post due to the fact that both dailies contribute to the local as well as national newspaper market in the region, the residents of Waterloo would be as representative as any urban English-speaking population across Canada.

Before undertaking the two sets of tests, it is valuable to provide an overview of both the public opinion and media content analysis results from implementing the model in the Waterloo region.

B. Overview of media and public opinion data

In interpreting the results of both the media and public opinion components of the study, it is useful to provide some context as to the major events occurring in the region during the time of the study. Economically, the region was negatively affected during the sample period by a series of closures of large manufacturing plants hampered by the marked rise in the value of the Canadian dollar in 2007 and into 2008 that hurt local exports. TD Economics reported that the province of Ontario generally saw one of its weakest quarters during the three months ending March 31, 2008, with a 1.4% drop in real GDP growth and a 10% decline in business investment. 99 Exports fell 16% in the first quarter of 2008, and were forecasted to be down 3% for the second quarter. The Waterloo region was within the epicentre of the decline in southern Ontario's manufacturing sector, with high-profile plant closures announced by Kitchener Frame, Ledcor, Martinrea,

Campbell's and ClosetMaid, in addition to reduced output in automobile manufacturing in

Windsor, Oakville, and Oshawa.

As a result, it was not surprising to observe that most respondents chose the economy and employment as the most important national and local issue from the first battery of open-ended questions. Among national issues, respondents were divided in the selection of the second most-important issue between the environment and health care.

Consumer issues, primarily rising gas prices, ranked as the fourth most-prominent issue, followed by war and conflict, specifically Canada's involvement in Afghanistan. Other national issues tracked tended to garner a lower response rate. Locally, public infrastructure, led by road conditions, ranked a strong second after the economy, followed by the environment which, in this case, also included respondents concerns about water quality. Traffic and public transportation also ranked high, ahead of health care (which, due to family doctor shortages and wait lists in the region, was identified as both a local as well as a national issue). Local taxes and budget issues, and housing and property issues were also identified by at least 10% of respondents as among the top three issues. There was a statistically significant but weak positive association between the rank-order of top

~ TD Economics, Time for a Vision of Ontario's Economy, September 29, 2008. 128 national versus local issues (x=.12, p<.0l). The results of the two sets of open-ended questions are provided in Tables 4-1 and 4-2 below. Note that because respondents were asked to name a first and second choice (both are included in the tables), percentages do not add up to 100.

Table 4-1 Top national issues by respondents (N=2054) Topic % respondents Economy 57% Environment 31% Public health care 29% Consumer issues 20% War/conflict 17% Budget/finance/taxes 6% Citizenship/human rights 5% Public education 4% Politics 4%

Table 4-2 Top local issues by respondents (N=2054) Topic % respondents Economy 43% Public infrastructure 30% Environment 26% Transportation/transit 19% Public health care 16% Budget/finance/taxes 12% Housing/property 10% Consumer issues 7% Crime / policing 6% Politics 4% Welfare/social services 2% 129

Similarly, looking at the overall composition of news coverage reaching residents in the Waterloo region at the same time (Tables 4-3a and 4-3b), there were notable and not unexpected differences in the subjects of news content that were provincial and/or national origin, and news of local events and issues in the Waterloo region.

Table 4-3a Top provincial and national issues by media coverage, by month (N=12031)

Jan-08 Feb-08 Mar-08 Apr-08 May-08 Jun-08 Total Crime 15.2% 13.2% 11.9% 10.6% 13.1% 9.3% 12.6%

Politics/scandal 9.3% 12.1% 15.2% 11.4% 12.0% 11.6% 12.0%

Economy 11.8% 7.8% 8.5% 13.0% 8.8% 15.9% 10.3%

Fires/floods/accidents 11.1% 8.1% 8.9% 9.3% 12.8% 7.2% 9.9%

War/conflict 11.4% 11.2% 6.9% 5.1% 4.1% 6.7% 7.6%

Terrorism and security 4.3% 3.7% 5.0% 7.5% 6.3% 6.1% 5.5%

Citizenship/human rights 2.4% 2.9% 4.8% 7.9% 6.1% 7.0% 5.0%

Human interest/achievement 3.2% 4.9% 4.1% 4.7% 5.5% 4.7% 4.5%

Science/medicine 3.3% 4.4% 3.7% 5.3% 5.1% 3.0% 4.3%

Health care 3.7% 2.9% 4.1% 2.4% 4.8% 6.7% 3.7%

Environment 3.4% 1.9% 4.3% 4.1% 3.7% 3.2% 3.5%

Consumer issues 2.4% 3.1% 3.0% 4.1% 4.2% 2.6% 3.4%

Taxes/budgets 1.5% 6.7% 4.8% 1.6% 0.8% 1.4% 2.9%

Transportation/transit 3.4% 2.9% 3.3% 1.5% 2.0% 2.3% 2.6%

Education 4.1% 2.2% 2.8% 1.9% 2.0% 1.2% 2.5%

Weather 2.0% 2.4% 1.6% 0.8% 0.1% 0.2% 1.3%

Entertainment/Arts/Sports 0.8% 2.0% 0.8% 1.3% 0.6% 1.6% 1.1%

Other 6.6% 7.4% 6.1% 7.8% 7.8% 9.5% 7.3%

Total 100% 100% 100% 100% 100% 100% 100%

Table 4-3b Top local issues by media coverage, by month (N=4876)

Jan-08 Feb-08 Mar-08 Apr-08 May-08 Jun-08 Total

Crime 29% 27% 20% 21% 20% 19% 22%

Economy 17% 21% 20% 10% 9% 16% 15%

Fires/floods/accidents 5% 6% 7% 18% 16% 12% 11%

Education 7% 7% 8% 6% 5% 10% 7%

Environment 3% 5% 6% 5% 3% 6% 4% 130

Transportation/transit 5% 7% 5% 2% 3% 2% 4%

Health care 3% 4% 3% 4% 5% 2% 4%

Human interest/achievement 0% 2% 1% 7% 7% 4% 4%

Taxes/budgets 6% 4% 5% 2% 0% 2% 3%

Terrorism and security 4% 2% 3% 2% 4% 2% 3%

Citizenship/human rights 2% 1% 1% 2% 3% 2% 2%

Ente rta i n me nt/Arts/Spo rts 1% 1% 3% 2% 2% 4% 2%

Science/medicine 1% 1% 1% 1% 2% 1% 1%

Politics/scandal 1% 1% 1% 2% 2% 0% 1%

Weather 1% 1% 1% 1% 0% 1% 1%

Consumer issues 0% 0% 0% 1% 1% 1% 1%

War/conflict 1% 1% 1% 0% 0% 0% 0%

Other 16% 12% 15% 16% 17% 17% 16%

TOTAL 100% 100% 100% 100% 100% 100% 100%

In visually comparing the public agenda of top issues compared to media

coverage, there are several characteristics that would influence the results of an agenda-

setting research study. First, despite the exclusion of items from specific sections of a

newspaper or newscast concerning weather, sports, arts and entertainment, many topics

covered as news did not correspond to an identifiable public issue. Topics that did not

correspond to an identifiable public issue included predominantly stories of catastrophic

events, as well as personal/human achievement, and weather, sports and entertainment

23 stories included in the news sections sampled. This group comprised 22% of the sample

at a national level, and 20% at a local level. Furthermore, an additional group of topics,

categorized as "other", included subjects that received relatively little media attention, but

23 Weather, sports and entertainment stories from dedicated sections of newspapers and broadcast news were excluded from the sample. However, in some cases, these topics emerged as major general news stories (weather affecting transportation, for example) and were then included in the sample. 131 in some cases might be considered as an identifiable public issue, including agriculture and food, parks and recreation, religious/moral issues, lifestyle, welfare and social services, and historical topics. In total, this combination of topics with either a low level of media volume or which did not pertain to an identifiable public issue comprised 35% of the media coverage of local topics, and 29% of national topics.

The second observable difference between the media and public opinion data sets was the marked difference concerning the issue of crime. Within the media sample, there was a prevalence of crime stories at both the national and, more particularly, the local level. Crime stories comprised 17% of the sample of news items at a national level and

22% at a local level to emerge as the most prevalent topic in news reportage. However, despite the amount of crime content in news reporting, respondents did not rank crime as a top issue, with only 5% citing it locally, and 3% citing it as a national issue. When asked to rank certain issues on a scale of one to ten, respondents ranked crime among the lowest surveyed, second only to national security. In fact, crime may not be a major problem in the Waterloo region.24 Statistics Canada did report an increase in violent crime in 2006, but the region still ranked among the top five cities in Canada in terms of low crime rates per capita. As a result, respondents may not view crime as a major problem, either in their local area or nationally, despite the regular diet of crime stories they may encounter. This observation does run counter to several agenda-setting studies focusing on the issue of crime, which have indicated that salience is affected by exposure to crime stories and/or

24 Outhit, Jeff. 19 July 2007, "The state of crime in Waterloo region" Waterloo Region Record, pg Bl. heavy users of media, particularly television (Smith, 1987; Gross & Aday, 2003; Lowry,

Nio & Leitner, 2003), but can also be affected by personal experience with crime

(Einsiedel, Salomone & Schneider, 1984). Another possible explanation is that respondents did not associate crime as a public issue in the same context as the environment, health care or energy prices, but as noted, crime has been recognized in other agenda-setting studies using similar questioning. In the end, the significant, regular coverage of crime stories, particularly in the local media, is an exception to the agenda- setting influence of the media, at least within this experiment.

A third observable difference was the reverse situation with health care and the environment, which ranked as a top issue among respondents but received relatively low media attention over the five-month period. Less than 4% of total news coverage of provincial and national issues concerned health care, and less than 3% addressed an environmental issue, yet both issues were selected by roughly 30% of respondents as a top issue—a result consistent with other general public opinion surveys among Canadians in recent years.

Despite these differences between news content and public identification of top issues, some similar patterns were evident. First, excluding crime stories and catastrophic events, the economy and jobs ranked at the top among issues covered by the news media at both a local and national level. Second, other key issues such as the environment, health care and, locally, transit and transportation, while comprising a low share of total media volume, also ranked near the top in terms of topics covered by the media. Again, excluding crime and catastrophic events, transit and the environment ranked fourth and third, 133 respectively, in the public opinion survey, and fourth and fifth, respectively, in the media survey, followed by public health care. War and conflict ranked among the top five issues in both the public opinion survey of top national issues and in the media survey of national stories.

C. Cross-sectional tests of the media-centric model »

The most basic test for the presence of an agenda-setting effect is to conduct a cross-sectional rank-order correlation of the media's coverage of issues against the top issues selected by respondents. Despite its limitations, particularly in ignoring the potential dynamic influence of media over time, cross-sectional tests for agenda-setting are still widely used and can be considered at least as a starting point to test for the presence of an agenda-setting effect from the model.

The first test was to determine whether an agenda-setting relationship between respondents and the media was present within a representative sample of only national news outlets. As discussed earlier, a potential weakness in the research designs of certain agenda-setting studies has been the assumption that a media sample consisting of elite national outlets that are most readily available (CBC, the Globe and Mail in Canada, or the major TV networks in the U.S. coupled with the New York Times) can serve as a sufficient proxy for all news media. In this survey, however, only 10% of respondents reported reading the Globe and Mail on a regular basis, compared to almost half that read the local newspaper. Similarly, a much higher percentage of respondents reported watching local 134 television news than national news broadcasts. Given the significant consumption of local news, for researchers to assume that a sample of only national media outlets can be valid, it must assume that the national media either influence and/or reflect the same issues as that of the local media, An example of this assumption is how a number of agenda-setting studies have cited the now-dated 1972 text The Boys on the Bus by Crouse in projecting the influence of the New York Times on the national media national news (Winter & Eyal,

1981; Massing, 1984; Mazur 1987).

The content analysis comparing local coverage in Waterloo and national media coverage does not support this assumption. Instead, a noticeable difference in the issue

selection between national and local media was observed. The difference was most visible

in comparing local versus national broadcast media. Table 4-4 provides a topic breakdown

of the top stories reported on the broadcasts of major local and national outlets over the five month sample period, while Table 4-5 provides a rank-order correlation matrix of the results. National media broadcasts from different media (radio versus television) and from

different networks (CBC versus CTV) were highly correlated, as were local television and radio broadcasts. However, local and national broadcasts were either not correlated along major topics, or the correlation was noticeably weaker compared to that within the national

outlets. Generally, local news tended to offer proportionately more attention to stories

about crime, business and the economy, and health care. National newscasts devoted more

attention to politics, war and conflict, human interest/personal achievement, human rights and the environment. Global's supperhour newscast (not including the national newscast), 135 with its focus on Ontario regional news, did not correlate with either local or national news outlets.

Table 4-4 Distribution of top topics by broadcast news channel: January 1—June 7,2008.

CBC- Clll- Radio CBC- Global Topic CKGL One TV CTV-TV CKCO Ontario Total

1 Crime 12.6% 7.4% 11.3% 9.1% 29.4% 34.1% 18.2%

2 Fires/floods/accidents 15.6% 8.8% 16.5% 20.4% 16.7% 20.3% 15.8%

3 Economy 21.9% 6.6% 7.4% 8.2% 15.0% 9.8% 12.6%

4 Politics/scandal 8.4% 22.8% 19.7% 20.1% 2.7% 1.2% 11.3%

5 War/conflict 4.6% 14.5% 10.8% 8.2% 0.8% 0.3% 6.0%

6 Terrorism and security 2.4% 8.4% 6.3% 4.9% 2.2% 1.6% 4.1%

7 Citizenship/human rights 2.7% 6.8% 3.4% 4.2% 3.1% 1.2% 3.7%

8 Transportation/transit 4.6% 1.0% 2.2% 2.0% 4.0% 7.8% 3.5%

9 Human interest/achievement 1.7% 5.8% 2.3% 4.7% 2.7% 2.6% 3.3%

10 Health care 2.9% 3.1% 2.2% 1.1% 3.2% 2.1% 2.7%

11 Taxes/budgets 4.9% 1.3% 2.0% 1.9% 2.7% 1.0% 2.5%

12 Education 2.6% 1.2% 0.5% 0.3% 3.8% 4.7% 2.4%

13 Environment 1.6% 2.2% 2.9% 0.6% 2.8% 1.9% 2.1%

14 Weather 2.6% 0.4% 2.3% 3.5% 0.9% 4.1% 1.9%

15 Consumer issues 1.7% 1.2% 3.1% 2.7% 1.1% 2.4% 1.8%

16 Science/medicine 1.1% 2.2% 3.2% 2.8% 1.0% 0.5% 1.6%

17 Entertainment/Arts/Sports 1.0% 0.6% 0.4% 0.1% 0.3% 0.2% 0.4%

18 Other 7.0% 5.7% 3.6% 5.2% 7.5% 4.1% 6.0%

Note: Based on top five items on each news broadcast. N=5433.

Table 4-5 Rank-order correlation of top topics by broadcast news channels (N=15).

CBC CBC CKCO- CKGL- CTV Clll Radio TV TV AM (Global- TV CBC-Radio 1 .555" 0.172 0.287 .433* -0.077

CBC-TV .555" 1 0.295 0.371 .632** 0.143

CKCO-TV 0.172 0.295 1 .657** .440* 0.314

CKGL-AM 0.287 0.371 .657" 1 .459* 0.238

CTV-TV .433* .632" .440* .459* 1 0.325

Clll (Global- - 0.143 0.314 0.238 0.325 1 136

TV) 0.077

Kendall's tau-b. *p<.05. **p<.01.

When the study was limited to only front-page stories selected from national print media outlets (Globe and Mail and the National Post, as well as the Toronto Star) along with the selection of the most important national issues chosen by respondents, no relationship was observed (x = -.02, p=.93, n=10). The agenda-setting effect appeared to be stronger for national television, but was not statistically significant (x=.42, p=.09, n=10).

This preliminary finding would suggest that a basic agenda-setting experiment conducted among a representative (if geographically confined) group of Canadians over a five-month period compared with the typical selection of national media outlets would not indicate a significant agenda-setting influence.

A media-centric model would select and filter outlets and news content that meet certain tests of spatial, channel and temporal proximity to its respondents. Using all outlets surveyed, there was no significant rank-order correlation between the top ten national issues selected by respondents among all outlets surveyed (r=0.42, p=.09, N=10), and even less correlation among the top local issues named by respondents relative to the full volume of media content (r=0.24, p=.33, N=10). This top-line result also suggests that no relationship exists between respondents' perceptions of top issues and the selection and reporting of those issues by the media.

25 While eleven top issues were tracked, crime was excluded from the remaining results, as few respondents indicated crime as a top local or national issue despite the high prevalence of crime-related stories in the news. 137

This initial lack of evidence of an agenda-setting effect is not surprising when examined against the notable difference in composition between stories of local, national and international scope or origin. The economy emerged as both a major national and local

story, with health care and the environment also emerging as major national and local topics. War/conflict and terrorism/security were primarily national and international

subjects, while infrastructure (roads, bridges, civic buildings) along with crime were

disproportionately local in nature.

Table 4-6 Breakdown of major topic by scope

Major topics Local National International Total

Economy/business 19% 18% 9% 16%

Environment 7% 6% 9% 7%

Crime 25% 16% 8% 16%

Education 7% 3% 1% 3%

Health 7% 8% 5% 7%

Infrastructure 13% 4% 2% 5%

Terrorism/security 0% 2% 6% 3%

War/conflict 0% 8% 8% 6%

Other 22% 35% 53% 37%

100% 100% 100% 100%

N=23242

Returning to the original rank-order correlation test, but this time filtering for the

qualities of news proximity to respondents (location of channel, and the scope (i.e., origin

or location of news story), statistically-significant results of an agenda-setting effect were

evident that supported testing for media proximity. Table 4-7 breaks down the media

sample into four categories: a)dthe full sample of media coverage with no restrictions on

scope or outlet; b)iiiiedia coverage based on the prescribed audience for each type of media outlet (i.e., national outlets for national issues, versus local outlets for local issues), with four sub-categories listed based on media type; c)|nedia coverage based on the origin or scope of the subject of the news item (i.e., national stories versus local stories); and finally, d)ipnedia coverage based on a combination of the proscribed audience and scope of the subject of the news item (national, local). The results indicated that when media proximity to respondents was ignored ("all media" - the full media sample of both national and local media outlets and issues of local, national and international in scope), no statistically- significant agenda-setting effect was observed. The first filter based on type of media improved the correlation and, while not statistically significant, the relationship between all national media coverage and top national issues was notable (x=.47, p=.06, N=10).

Further refinements to the sample to bring it in closer proximity to respondents yielded stronger rank-order correlations. When the media sample was differentiated by location and scope of the issue covered, the potential agenda-setting relationship was most evident.

The observed effect was stronger concerning the relationship between local issues for respondents and local media coverage of local events, particularly when compared with the full sample of all media. Regarding national issues, there was only a statistically significant effect for national media coverage of national issues (x=.49, p<.05, n=10), and the difference between the high proximity and low proximity samples was marginal. 139

Table 4-7 Rank-order correlation: top national and local issues by media and origin of content

Top issues

Media National Local

All .42 .24

Media .47 .29

Print -.02 .24

TV .42 .16

Internet .41 .24

Radio .29 .24

Scope .33 .56*

Media + scope .49* .56* Kendall's tau-b. "p<.05, "p<.01. N=10.

Overall, the cross-sectional analysis provided limited support for using a media sample with high proximity in testing for a media agenda-setting effect. First, the national print media sample showed very little correspondence with respondents' selection of top issues. While the national newspapers did provide strong coverage of the economy, ranking as the top issue, environmental issues and health care ranked among the lowest given front-page treatment over the course of the sample period, as well as consumer issues, while proportionately more attention was devoted to war/conflict, terrorism/security, politics and budget/tax issues. However, the rank-order of top news stories provided by national television outlets and the national internet site selected (globeandmail.com), tended to correspond more closely with the top issues selected by respondents. Given the media consumption patterns witnessed among respondents, the correlation with national television compared to national print media was not surprising, while the results from the globeandmail.com sample can possibly be attributed to the fact that its headline format and 140 editing in many respects has a closer correspondence to the format of broadcast news than to traditional newspapers.

The cross-sectional analysis showed no correspondence between local media coverage and top local issues chosen by respondents until a filter was applied to the content based on high proximity (scope and media), which had a notable effect on the results, indicating the presence of a significant relationship between the two agendas. The analysis would suggest the presence of a relationship between top local issues identified by a population and media coverage of those issues, but that such a relationship is only evident when properly filtered for content and channel that is proximate to respondents. A similar effect was observed within the national sample, but the difference between a less-proximate and more-proximate sample was marginal.

Testing for demographic control variables and the level of interpersonal discussion of issues did not affect this finding. Previous agenda-setting studies indicated that only one demographic control variable tends to affect results - education. The analysis of the data confirmed the impact of education, although it did not greatly change the overall level of correlation found in the cross-sectional analysis. Dividing respondents into three groups based on low- (high school or less), mid- (some university/college) and high- education (university or post-graduate) indicated little difference between the three groups, as outlined in Table 4-8 below. Correlations based on education segmentation were stronger than the sample of all respondents when focusing on national media coverage of national issues, but were weaker and non-significant when focusing on local media coverage of local issues. A similar pattern was observed when controlling for the self- 141 response variable concerning how often respondents claimed to have discussed news and current events with others. The correlation with the national media agenda of issues was somewhat stronger among those that discussed news more frequently, and weaker among respondents that stated they almost never discussed news and issues. However, again, the discussion variable did not explain the correlation observed in local coverage. The application of demographic data as a controlling factor proved to be less useful in explaining the variance in issue agendas relative to the tests for media proximity.

Table 4-8 Rank-order correlation controlled for level of education and interpersonal discussion—local and national issues

National Local media/issues media/issues

All respondents 0.49* 0.55*

Education

- high 0.60* 0.36

- mid 0.64* 0.29

- low 0.51* 0.27

Discussion

- often 0.57* 0.29

- occasionally 0.51* 0.29

- never 0.47* 0.23

2 P

The media-centric model proposes to better explicate the transfer of salience process between the media and the public by pursuing a strategy of greatly differentiating the units of the media content analysis study based on a combination of the unit's externally-visible cues as well as the relative difference in each unit's expected audience exposure. Using this strategy, two different weights were calculated for each unit of media 142 content analysis as outlined in Chapter 3 and presented in Appendix E: a) a weight based on a combination of external cues (valence) and market research data, and b) a weight based on self-response data from media consumption questions of the public opinion survey. The working hypothesis was that the media's potential agenda-setting influence would be more evident in examining data that is more differentiated, based on valence and audience exposure weightings than the same data set that is undifferentiated. A rank-order correlation was conducted between the public's agenda of the top ten national issues against four different segmentations of the media sample: a) all media coverage of

Canadian topics, both local and national; b) all coverage from national media outlets of

Canadian topics, local and national; c) all coverage of national topics only; and d) all local media coverage of Canadian topics, both local and national. The results are presented in

Table 4-9 below.

Table 4-9 Rank-order by national issues—weighted versus non weighted media data

All National National Local media media topics media

no weight 0.29 0.47 0.33 0.33

market data/valence 0.38 0.51* 0.29 0.42

self-response 033 047 029 0.33

Kendall's tau-b. *p<05, "p<.01. N=10.

There was no categorical difference between the strength of the rank-order

correlation observed using unweighted data compared to the weighted data. Audience and

26A third weight was calculated based on using the same valence formula applied to the market research data, but used to differentiate coverage based on media self-response weights. In a series of tests, there was no difference in the rank-order correlation between the self-response data set and the data set further weighted by valence scores. 143 valence weights had no significant effect on the results examining top national issues.

However, in three of the four tests, the application of audience market data that was further differentiated based on valence cues did result in a higher correlation, but in only one case

(national media) did the application of the weight result in a statistically significant correlation. The results of the test, while not supporting the application of audience weights in order to better understand the transfer of salience process, may not entail rejecting the strategy outright, and the use of audience weights applied to the media data sets is further examined in the longitudinal tests below.

The rank-order correlations were also tested against two subsamples of the population divided between high and low news consumer groups based on each respondent's self-reported frequency of consumption per week of print, television, radio and internet news. There was a very high correlation in the rank-order agenda between the groups for both top national issues (x=.94,p<.01, N=10), as well as local issues (x=.91,

/7C.01, N=10). A further test to determine if there was changes between groups using a battery of close-ended questions ranking twelve issues on a 10-point scale revealed similar results, with high versus low consumers of internet use showing the greatest variation in top issue selection, while television saw the least variation, as summarized in Table 4-10 below. 144

Table 4-10 Rank-order correlation of top issues by high/low news consumers, by media type

AN TV Radio Newspaper Internet

Issues ranked high .697" .870" .718" .809" .667"

Issues ranked low .788" .848" .862" .788" .788"

Kendall's tau-b. 'p<.05, "p<.01. N=12.

D. Longitudinal tests of the media-centric model

As previously noted, the frequent use of static, cross-sectional approaches in

agenda-setting research has been recognized as a weakness in the research designs of many

agenda-setting research models (Eyal, Winter, & DeGeorge, 1981; Behr & Iyengar, 1985;

Brosius & Kepplinger, 1990; Gonzenbach & McGavin, 1997). Given the dynamic, rather

than static, nature of the media effect and its potential influence on audiences, agenda-

setting research models should be able to track and measure the relationship between media

and public influences over time.

In the following tests, media was treated as an independent lag variable to

public opinion. While the approach suggests a causal and unidirectional relationship in

which media influences public opinion, the usual caveats apply in correlation: that even

applied over time, correlation does not entail causation. The use of correlation matrices is

intended to determine if at least a basic condition for causation is met.

For the public opinion element, the ten-week period provided ten data points

comprising 205 respondents each. Due to the low sample size within each period, the ten 145 issues examined in the cross-sectional data was reduced to five issues which respondents had, within the open-ended questions, selected as a top local or national issue, or in which the respondent ranked it at the top in terms of importance (ten out of a ten) within the battery of close-ended questions. The issues, and their sample sizes, were: economy

(N=1467), health care (N=1255), the environment (N=1112), consumer issues (i.e., food and gas prices, N=962), and war and conflict (N=630). Media content was evaluated based on all national media coverage of the issue, as well as sub-groups comprised of print, television coverage, national/Ontario media coverage, and local media coverage. Each media segment was further evaluated based on the four audience exposure weights (no weight, weight based on market research data, weight based on self-reporting, weight based on self-reporting adjusted for valence cues).

Table 4-11 Respondent's selection of top issues by survey week

1 2 3 4 5 6 7 8 9 10 Total SD

Economy 80% 72% 74% 74% 78% 76% 66% 75% 66% 74% 73% 9.03

Consumer 39% 30% 50% 48% 51% 54% 53% 52% 49% 59% 48% 16.30

Health 63% 65% 61% 64% 65% 58% 63% 67% 60% 63% 63% 5.08

Environment 55% 57% 59% 62% 58% 51% 59% 52% 50% 56% 56% 7.91

War/conflict 41% 44% 33% 32% 32% 28% 34% 24% 25% 26% 32% 13.29

Generally, research designs into agenda-setting have either approached time in one of two manners. The first is to focus on a single issue, and correlate the changing importance of that issue among a representative public over time relative to the media's attention to it, typified by Eyal and Winter's 1981 study of change in importance of civil rights as an issue among Americans between 1954 and 1977 relative to the coverage 146 devoted to the issue in the New York Times. Similarly, researchers have examined a slate of issues, but treated each issue separately (Zucker, 1978). The second approach is to examine rank-order changes in an entire agenda over time (Eaton, 1989). Although the study did test for correlations across the entire agenda, the sampling used for this experiment makes testing the model on a per-issue basis preferable. While there was some variation in the number of respondents likely to select one of the five issues as the most important, there was little change in the rank-order over the ten weeks, as illustrated in

Table 4-11 outlining the percentage of respondents each week choosing the issue.

Respondents consistently chose the economy as a top issue, followed by health care, with some fluctuation in the rank order of the last three issues.

Given the few issues selected, a better test of the model over time is to examine each issue separately, analyzing the relationship between respondents' selection of the issue as a top issue each week relative to different levels of media coverage devoted to the issue on a time-lag basis. For the study, five time lags were examined, incorporating all media coverage of each issue between the point of time in which the questionnaire was administered to the end of the lag period. The lag periods ranged from an immediate period of seven days, to a longer period of 56 days extending to the beginning of the content analysis. The results of the time lag correlation, by issue and using the same audience data weights applied above, are summarized in Table 4-12 below. 147

Table 4-12 Correlation by top issue over time by media weights—all media

No Market Self- Adjusted weight Response Self- response

7- Economy 0.67* 0.66 0.67* 0.68* day lag Consumer prices 0.06 0.12 -0.24 0.04 Health care -0.19 -0.15 -0.08 -0.15

environment 0.10 0.06 0.04 -0.04

War/conflict 0.68* 0.64 0.49 0.66*

14- Economy 0.54 0.57 0.54 0.56 day lag Consumer prices 0.56 0.54 0.14 0.52 Health care 0.34 0.48 0.44 0.51

environment 0.20 0.17 0.10 0.10

War/conflict 0.67* 0.64 0.56 0.65

28- Economy 0.33 0.40 0.38 0.35 day lag Consumer prices 0.85* 0.85* 0.62 0.83* Health care 0.10 0.15 0.13 0.17

environment 0.31 0.46 0.50 0.42

War/conflict 0.80* 0.78* 0.75* 0.79*

42- Economy -0.37 -0.37 -0.30 -0.39 day lag Consumer prices 0.71* 0.70* 0.69* 0.69* Health care 0.00 -0.02 -0.01 0.01

environment -0.09 0.04 -0.06 0.01

War/conflict 0.47 0.40 0.53 0.41

56- Economy 0.18 0.53 0.36 0.42 day lag Consumer prices 0.79* 0.76* 0.81* 0.79* Health care -0.08 -0.04 -0.12 -0.07

environment -0.18 -0.12 -0.16 -0.10

War/conflict 0.84* 0.85* 0.84* 0.84*

An initial review of whether concerns about the economy correlated with media coverage produced correlations that ranged from r=0.67 over the seven-day lag (the only 148 significant result) to r=-0.37. The short sample period likely belies the fact that for residents of the Waterloo region, the economy had been a top issue for a number of months. Early in the New Year the area was hit by several major plant closings, with the ensuing loss of hundreds of jobs blamed on declining export markets for Canadian- manufacturing goods during a period in which the value of the Canadian dollar was high.

Local media covered these stories in earnest, including demands by union and municipal leaders that provincial and federal government authorities come to the aid of the region's manufacturing sector. However, the economy was not as big an issue with national media outlets during this period, and while news stories about the problems facing Southwestern

Ontario's manufacturing sector were reported by the national media, they garnered relatively less attention than by the local media. These events took place between two and twelve weeks prior to the sampling of Waterloo residents. 149

Figure 4-2 Media coverage of economic stories by local and national media, by week, January 1—June 6, 2008.

Ledco strike;

u c XI -Q V— >- Ol roc (O 0) ro ub o T-H ro t-I (N CM

Local National

The effect of local media coverage of these events prior to the sampling of public opinion is illustrated in Figure 4-2 that plots national and local media coverage relative to public selection of the economy as a top issue during the ten-week period, particularly when compared over a longer time lag (56 days) that incorporates the major local economic news events that occurred in late January and early February.

Table 4-13 Correlation of economy as a top issue by time lag and media type

Time All National Local TV Print lag

56 0.06 -0.44 0.84** 0.23 -0.22

35 0.43 0.00 0.68* 0.49 0.31

14 0.65* 0.38 0.64* 0.65* 0.57 150

As illustrated in Table 4-13, across three different time lags, local media coverage showed a significant correlation over the ten-week period with respondents' concerns, while national media coverage had a dampening effect on the results. On this issue, controlling for channel proximity had a logical and significant effect in explaining the agenda-setting effect. This is apparent in charting local versus national media coverage of the economy over time relative to importance cited by respondents, illustrated in Figure

4-3.

Figure 4-3 Standardized scoring of economy as a top issue by local and national media over a ten-week period

2,00

1.50 1.00 »« -«,

0.50 — Opinion 0.00 Q — Local media iSl 9 10 • >> National media -0.50 -1.00

-1.50

-2,00

•2.50

Chart plots standardized scoring (number of standard deviations, or SD) for three sets of data: "opinion" is the number of respondents that ranked the economy as a top issue each week; "local media" is the total number of stories about the economy in local radio, print, television and internet media outlets in the Waterloo region, and "national media" is the total number of stories about the economy in national radio, print, television and internet media.

The test for the influence of news media consumption on the issue of the economy as outlined in Table 4-14 showed no significant results, with one important 151 exception. When restricted to a long lag time and to local news media coverage, there was a significant correlation between high news media consumers and the selection of the economy as a top issue, but the relationship was not evident among low media consumers.

Not only is this correlation the most likely one to occur, given the economic events that have affected the Waterloo region and the significant media coverage they have received by local outlets, it might arguably be maintained that the only significant correlation that should occur in the experiment would be among high news media consumers, along local media coverage, over a longer time lag that would incorporate the major events that took place during the winter months.

Table 4-14 Correlation of the economy as a top issue by lag periods, media type and level of media usage by respondents Media All National Local TV Print usage 7 High 0.27 0.14 0.27 0.19 0.29 day Low 0.48 0.38 0.41 0.45 0.35

14 High 0.37 0.35 0.40 0.37 0.35 day Low 0.18 0.25 0.14 0.21 -0.04

28 High 0.53 0.26 0.43 0.49 0.47 day Low - -0.42 0.04 - -0.44 0.39 0.29 42 High - -0.31 -0.15 - 0.19 day 0.28 0.37 Low - -0.16 0.02 - -0.40 0.11 0.06 56 High 0.28 -0.20 0.69* 0.47 0.03 day Low - -0.12 -0.08 - -0.23 023 0.24

Pearsons's r. *p<.05, **p<.01. N=10.

The final test concerned the impact of audience weights. The test for the effectiveness of audience weights was whether: a) a correlation was evident when one of 152 the three audience weights was applied to the media data, and b) whether that correlation was at least 10% higher than that observed using unweighted data. Again, the weighted data was examined based on two dimensions: location of media (national versus local) and type of media (television versus print). The test did not support the hypothesis categorically. Audience-weighted data did prove significantly more effective in indicating a correlation looking at local media over a seven-day lag period, and for television over a

56-day lag period, but the other tests failed, and the results overall for using weighted data were inconclusive.

H

The study indicated no presence of a relationship between media coverage and public opinion on health care. A review of local and national media coverage of health care issues between January 1 and June 7, 2008, saw no notable spikes compared to the economy, and no major difference in volume between national and local media, as illustrated in Figure 4-4. Media coverage was higher towards the end of the sample period in early May and early June, but much of the rise was produced by events which respondents may not associate with health care issues (a woman who succumbed to an unknown illness aboard a VIA Rail train in northern Ontario) or a series of stories from a single outlet (Global News' week-long special broadcasts from Ontario hospitals). 153

Figure 4-4 Tracking local and national media coverage of health care issues by week: January 1—June 6, 2008

Global News Mystery illness 35 on Via train, C special on difficile hsopitals 30 \ 25

| 20 4-o« t 1-I0 c 10

5

0 o rj O O Q. a. Q —-i —rsp u_ u_ < < 6 ro O no r\l t-H rsi ob Week • Local National

Applying news media consumption to the model did not change the fact that health care coverage did not correlate with public opinion, and the application of audience exposure weights also revealed no correlation. In fact, among the five issues tracked, health care showed the least variance across the ten-week sample, as indicated in Table 4-

12 above. It is likely that respondents, like many Canadians, still continue to cite health care among their most important issues despite the relative lack of recent media attention,

and the results may reflect a longer lag effect of the substantial media coverage that health care issues received earlier in the decade, despite less evidence of health care as a top-of- mind media topic in recent months. 154

Table 4-15 Correlation matrix of health care as a top issue by lag periods, media type and level of media usage by respondents

Media usage All National Local TV Print

7 day High -0.18 -0.24 -0.02 -0.20 0.39

Low 0.01 0.09 -0.19 0.01 -0.46

14 day High 0.32 0.19 0.47 0.25 0.55

Low -0.01 0.06 -0.17 -0.02 -0.61

28 day High -0.39 -0.33 -0.41 -0.25 -0.13

Low 0.47 0.46 0.26 0.25 0.21

42 day High 0.02 0.02 0.04 0.00 0.27

Low 0.47 0.58 0.24 0.30 0.43

56 day High -0.08 -0.01 -0.17 -0.08 0.44

Low 0.11 0.18 -0.01 0.11 0.08

Pearsons's r. *p<.05, **p<.01. N=10.

Bi

A similar phenomenon may have been at work on the issue of the environment.

An initial review of all media coverage against the environment cited as a top issue saw no correlation, with scores ranging from r=0.31 to r=-0.18. 155

Figure 4-5 Tracking local and national media coverage of environmental issues by week: January 1—June 6, 2008

CBC series on Earth Hour Earth Day

Week Local — — National

While there was no general trend in environmental coverage, the sample period was marked by two major events: a) the inaugural Earth Hour celebration that took place on March 29, 2008, and b) Earth Day on April 22, 2008. Both garnered very significant local as well as national media attention.

While the analysis featuring all respondents selecting the environment as a top issue, either national or local, showed no correlation, it is worth noting that over a seven- day lag period (and only when selected by respondents among top local issues), a significant correlation was observed (r=0.75, p<.05, N=10) that mostly reflected local media and television coverage. This correlation stems from two brief but very prominent 156 instances of media coverage of local events involving: a) Earth Hour on March 29, 2008, and b) Earth Day on April 22, 2008, as illustrated in Figure 4-6 below.

Figure 4-6 Local media coverage and public opinion—environment identified as a top local issue, by survey week

•Opinion

• Media S 0.50

Chart plots standardized scoring (number of standard deviations, or SD) for two sets of data: "opinion" is the number of respondents that ranked the environment as a top local issue each week; "media" is the total number of stories about the environment in local radio, print, television and internet media outlets in the Waterloo region.

A high number of respondents named the environment as a major issue locally soon after these events, but the level of concern was not sustained. Within all issues

(environment cited as top local or national issue by respondents), the environment appeared to be correlated with local media coverage within a shorter 14- to 28-day lag period, and only among high news media consumers, with low media consumers showing no effect. 157

Table 4-16 Correlation matrix of the environment as a top issue by lag periods, media type and level of media usage by respondents

Media usage All National Local TV Print

7 day High 0.33 -0.05 0.52 0.30 0.52

Low -0.07 -0.38 0.12 0.03 -0.19

14 day High 0.57 0.14 0.76* 0.46 0.63

Low -0.09 -0.36 -0.02 -0.14 -0.28

28 day High 0.51 0.08 0.67* 0.44 0.68*

Low 0.09 -0.33 0.31 -0.02 0.28

42 day High -0.09 -0.08 -0.06 0.00 -0.14

Low -0.09 0.01 -0.17 -0.02 -0.27

56 day High -0.24 -0.06 -0.23 -0.20 0.03

Low -0.09 -0.05 -0.12 -0.08 -0.12

Pearsons's r. *p<.05, "p<.01. N=10.

•V

The most unobtrusive issue surveyed, war and conflict, proved to be highly sensitive to media coverage. The initial test to determine whether the issue was positively correlated without any media-centric variables indicated a high correlation across all five time-lag periods. Differentiating by media consumption, different channels and different audience weights produced no change—the results remained highly correlated. As illustrated in Figure 4-7 below, media coverage trended precipitously downwards throughout the sampling period, which corresponded with fewer respondents naming the issue almost every week. During the first half of 2008, Canada's involvement in

Afghanistan had been the subject of very strong media coverage in January due to the tabling of the Manley Commission report concerning Canada's past and future role in

Afghanistan, and the government's subsequent motion that would end Canada's mission in 158

2011. Media coverage diminished from that peak period during subsequent months and remained relatively low throughout the ten-week sampling period.

Figure 4-7 Tracking total audience exposure of media coverage of war/conflict issues, by week: January 1—June 6, 200827

wiamev Parliamentary Commission report motion on

27 Chart measures media coverage based on "impressions". Impressions is a measurement of the aggregate of the potential audience reach of each news item. Audience reach figures based on NADBank and Nielsen Media Research figures. Print media coverage is weighted based on valence cues. For a full formula, see Appendix E. 159

Figure 4-8: Public opinion and media coverage concerning war/conflict issue: 56 day time lag

2.00

• Opinion

Q • Media

-1.00

-1.50

Chart plots standardized scoring (number of standard deviations, or SD) for two sets of data: "opinion" is the number of respondents that ranked war/conflict as a top issue each week; "media" is the total number of stories about the war/conflict in both local and national/provincial radio, print, television and internet media outlets.

The public selection of the issue of war/conflict was significantly correlated with media coverage across all time-lags with the exception of the 42-day period, in which the first week of media sampled was relatively low only due to the fact that coverage of the

issue in mid-February was relatively low. The issue was also strongly correlated across all media types, as all media types devoted less coverage to the issue through the first five months of the year. 160

The analysis also indicated a high correlation on consumer prices - a generally obtrusive issue, (i.e., the type of issue that individuals would typically experience outside of media coverage). Initial analysis using all coverage indicated no correlation over the shorter time lags, but significant positive correlations over longer time-lags of 28 days

(r=0.85), 42 days (r=0.71) and 56 days (r=0.79).

Table 4-17 Correlation matrix of consumer issues as a top issue by lag periods, media type and level of media usage by respondents

Media All National Local TV Print usage 7 day High 0.05 0.03 0.09 0.24 -0.43

Low 0.07 0.15 -0.08 0.14 0.05

14 day High 0.55 0.59 0.15 0.82" -0.38

Low 0.51 0.48 0.23 0.66* -0.21

28 day High 0.87" 0.81" 0.79** 0.90** 0.26

Low 0.69* 0.70* 0.57 0.70* 0.20

42 day High 0.77** 0.75* 0.78** 0.76* 0.70*

Low 0.52 0.56 0.38 0.53 0.44

56 day High 0.80** 0.77** 0.79** 0.75* 0.88"

Low 0.69* 0.68* 0.64* 0.66* 0.68*

Pearson's r. *p<.05, **p<.01. N=10.

Including media consumption variables to the analysis suggested that high media news users were more likely to be correlated with changes recorded in the priority of consumer issues compared to low news media users, and in several cases, only the responses of high news users correlated with the results. Analysis also indicated that television was likely a much more influential source of coverage on consumer issues than other media types. This conclusion does stand up logically: among the 36 different topics tracked in the media survey, consumer issues ranked at the top in terms of percentage share 161 of coverage derived from television (68%) relative to other media types, as reports on gas and food prices and product recalls and warnings were disproportionately covered by the television news media.

While not evident in the shorter time frames, the correlation between media coverage of consumer issues such as food and gas prices as well as product safety issues produced a significant correlation with rising concerns about these issues among respondents over the sampling period. However, unlike the case of war and conflict, in which media would logically be the almost exclusive source of information about the issue, real world indicators (namely, rising food and energy prices) could exert a significant influence. The sample period between April and June 2008 was a period of sharply rising gas prices, from a low in early February of $0.99/litre in the region to a peak for 2008 of

98 $1.36 in mid July—a record for gasoline prices in the province. Indeed, examining media, opinion and a real-world indicators on this question (gas prices in Toronto), illustrated in Figure 4-9 below, points to a strong correlation between all three (the

strongest is between media and real-world at r=0.92, followed by media and public

(r=0.81) and public and real world (r=0.78). Given the influence of real-world factors, the

influence of the media on public opinion is highly mitigated.

28 MJ Ervin Associates, Regular Unleaded Gas Prices 2008. http://www.mjervin.com/subscriptions/week_rul_posted_2008.htm. 162

Figure 4-9 Standardized scoring of public opinion ranking of consumer issues, media coverage, and gasoline prices in Toronto, by week

2.00

O

-2.50

Chart plots standardized scoring (number of standard deviations, or SD) for three sets of data: "opinion" is the number of respondents that ranked consumer prices as a top issue each week; "media" is the total number of stories about consumer prices in both local and national/provincial radio, print, television and internet media outlets; "gas prices" is the average national per-litre gasoline price per week over the sample period. Gas prices obtained from MJ Ervin's survey of weekly gas prices: http://miervin.com/gasoline prices.htm.

Discussion

The results over time among the five issues and the cross-sectional analysis highlight several strengths and weaknesses in the design and implementation of the media- centric model.

The emphasis of the model on a sample with high proximity to respondents— incorporating a broader range of media outlets defined both technologically and spatially— proved to be an important element in identifying and explaining underlying media agenda- 163 setting effects that would likely not have been observed using media samples of low proximity to the respondents. Cross-sectionally, the dispersion of top issues was positively correlated with both a set of all outlets surveyed and, under further examination, it was shown that local media was an important factor. Analysis of consumption patterns of news and information among respondents as well as available market research data for the target area supported the emphasis on local rather than national. Moreover, the longitudinal analysis of top issues tended to identify local coverage of key local events surrounding the economy and the environment that correlated with local media coverage over time, but did not associate with national media coverage.

The importance of media sample proximity and, in particular, the audience consumption and subsequent influence of local media, raises important questions about research designs purporting to understand a "national" media agenda. As already noted in the initial discussion leading to the promotion of a media-centric design, achieving high media sample proximity within a national agenda-setting research project would be costly and resource-intensive if (as it should) involve examining the influence of local media from each of the population centres that respondents are drawn from. Yet the implementation of the media-centric model clearly points to the influence that local media can exert.

Cluster sampling the respondent population and relevant media around several major centres across the country may provide a means of lowering research costs while still providing a high media sample proximity. Nonetheless, the results further highlighted the questions about sample validity around national agenda-setting research projects. 164

While the differences in audience exposure to and consumption of certain media outlets appeared to be a factor in explaining why local media showed a higher correlation with the public agenda than national media, weighting content by audience exposure and consumption was not useful. The strategy was to offer a partial explanation of how salience is transferred between the media and the public by substantially differentiating each news item based on external cues and/or media consumption and exposure. However, the results from this strategy were never consistently, significantly different than that produced by simply not differentiating data. There may be several reasons why this strategy to differentiate media content failed, which future researchers may want to explore in further detail. First, the experiment could suffer from Type 2 error: that external cues and media exposure/consumption could, in fact, be an important factor in explaining the transfer of salience, but the calculations used to weight the audience data and the assumption behind them were at fault. A different set of assumptions about the weight of external cues and/or audience exposure and consumption factors may have resulted in more consistent, significant results. A second and related explanation may lie in the fact that media content, while weighted, was still treated largely as an aggregate in comparison to the entire sample of respondents, both in the cross-sectional and longitudinal analysis. This treatment of the data somewhat belies the principle of high proximity between respondent and media, and stronger correlations may have become evident if agendas were compared between a subsample of media outlets that corresponded with the selection of top outlets selected by a subsample of respondents. However, the sample of 2055 respondents with rather diverse media consumption patterns did not allow for any clear division into valid subsets. Respondents that either consumed almost entirely one type of news, either defined by scope (national versus local) or by technology (internet versus print) comprised very small subsets within sample. Instead, most respondents chose a variety of media types, and scope was highly skewed towards local news consumption. A third explanation may be that other, less-empirically measurable factors concerning the nature of the content and the nature in which respondents consume and interpret news and information better explain the transfer of salience process than the more traditional classic, measurable factors such as prominence, frequency, placement and even market audience data and self-reported media consumption patterns. While the exercise of differentiating media based on cues and consumption patterns did not yield significant results, its failure still offers avenues for further research into the transfer of salience process, and offers a testable response to authors such as Kouisis (2000, 2004) who have argued for other factors to be included in research designs, such as valence, in order to better explicate the transfer of salience process.

The results of the longitudinal study also provided useful information on the treatment of time. With the exception of local opinion surrounding the environment, the model tended to indicate greater relationships across longer time periods that corresponded with similar research into time-lag effects surrounding agenda-setting. Within the 7- and

14-day lag periods, less than 10% of the pairings indicated a correlation between media coverage and opinion. Within the 28-day lag period, 36% of the tests indicated a significant correlation in at least one of the four sets of media data weights, and while this dropped to 28% over the 42-day lag, it was 48% over the 56-day lag. The tests, and the 166 data set, did not allow for the selection of a possible optimum lag period as has been suggested by previous authors (Chafee, 1972; Stone & McCombs, 1981; Eyal, Winter &

DeGeorge, 1981), although the results were consistent with the observation by Salwen

(1988) that a period of at least 5-7 weeks is required before an adoption of a media agenda occurs, and that it likely peaks at a later period of 9-10 weeks. However, the analysis involving both a single issue over time as well as the rank-order of top issues over time did not yield a result that would indicate an optimum time lag and, in fact, would suggest that time lag is highly dependent on the type of issue and the influence of specific, high-profile events that can skew results, as indicated by the example of the environmental issue.

The implementation of the media-centric model did point to the importance of ascertaining certain contingent conditions, particularly the obtrusiveness of the issue and the influence of potential non-media, real-world factors. Obtrusiveness was an important factor. Changes in the rank-order of obtrusive issues such as consumer prices were more likely explained by real-world factors, particularly the rising price of fuel and food during the sample period. Changes in a distinctly unobtrusive issue—war and conflict—were among the most sensitive to media coverage in tracing the agenda-setting effect. Even more than the typology of an issue as 'obtrusive' or 'unobtrusive' was the influence of underlying stories and events on particular issues. The distribution over time of the economy as an issue was shown to be correlated with a series of high-profile plant closures and other labour and economic issues in the region that received local coverage, but which was ignored by national media. Similarly, changes in the importance of the environment appeared to be sensitive to brief but highly visible environmental initiatives conducted by 167 stakeholder groups designed to raise the importance of the environment as an issue among the public, and which drew brief but sharply prominent media coverage that was reflected in the longitudinal analysis. These results of the media-centric model suggest that agenda- setting as a media effect may be less susceptible to overarching rules than is often conveyed in the research, and that is more sensitive to the nature of the issue and the events within a reasonable time-lag that drive that issue in the media. 168

Conclusion

As it approaches the fortieth anniversary of the publication of McCombs and

Shaw's Chapel Hill study, agenda-setting research is reaching a critical juncture. Since

1968, many findings have been produced that have broadened our understanding of the media's agenda-setting influence. Over the same forty-year period, however, the basic research design that underpins these findings has seen little overall development, particularly in how it views a key part of the equation: the role of the media. Technological advances, however, are changing the way in which we view and consume news, producing a fundamentally different news media landscape for researchers studying media effects in

2008 compared to the landscape on which McCombs and Shaw tested their hypotheses forty years ago. Investigating how we study agenda-setting in light of the fundamental changes occurring to its main independent variable was the motivation behind this dissertation, and is intended as its primary contribution to the field of communications research.

In this investigation, the dissertation set out with three objectives. The first objective was to systematically examine the legacy of research designs employed in agenda-setting and critically evaluate the assumptions underlying how media was modeled within these designs. The second objective was to offer a new, more "media-centred" 169 approach to agenda-setting research design that placed more emphasis on how media was sampled and conceptualized within the critical transfer of salience phase. The final objective was to implement and test a media-centred approach within a market and determine its efficacy, and what unique conclusions could be drawn as a result.

The analysis of the legacy of agenda-setting research designs was an important first step. Over the years, several writers have addressed specific areas of agenda-setting's research design, particularly its use of time and longitudinal data, as well as the identification of contingent conditions to be included that would filter or differentiate perceived effects. However, a review of the literature indicated that despite calls by several researchers for a more thorough review of the methodological underpinnings of agenda- setting, there has been no attempt to systematically catalogue and review the research designs, and the 40-year-mark provided a very good opportunity to conduct the review and meta-analysis.

The review of agenda-setting research designs revealed how these designs frequently underconceptualized the role of media in two ways. The first concerned questions about whether the methods used to generate the media sample met standards for establishing external validity. Unlike public opinion polling that employs random sampling within a population—a known, straightforward and empirically testable procedure to address questions of external validity—there is no similarly reliable means to validate the media sample. Instead, researchers have often cited very dated material or relied on face validity arguments to present samples that, collectively, revealed several biases. For example, the number of outlets sampled was often very low, with 72% of studies surveyed using four or fewer outlets. The samples skewed heavily towards national media, with national print outlets used in 44% of samples compared to 22% that used local print sources, and 74% using national broadcast media compared to 24% that used local broadcast. There was little use of other alternative forms of news media, with an almost complete absence of radio as a source of news, and little examination (to date) of the growing influence of the internet.

The second area in which the analysis indicated that media was underconceptualized concerned questions of internal validity, specifically in how the transfer of salience process was modeled within research designs. Despite the suggestion by researchers that externally visible cues play a role in how salience is transferred between media and the public, only 20% of studies surveyed over the last 25 years included such cues, although at least 29% of studies did provide some form of differentiating content from various sources, either through the use of cues or other formulas based on differing audience consumption and exposure patterns. Perhaps the most constructive development in research modeling of agenda-setting effects involving the transfer-of-salience phase has been the more recent investigation into second-level effects as well as the affective elements of text, in addition to the advent of how such salience is filtered through the presence of contingent conditions. Scoring the various research designs for how they conceptualized media at the sampling and questionnaire stage indicated no overall development in the research design over the course of the last 25 years and, in fact, the original 1968 McCombs and Shaw study scored well above the mean in terms of meeting certain design pre-conditions. 171

This discussion formed the basis for meeting the second objective: building the characteristics of a media-centred approach to agenda-setting research design that, in the tradition of a Weberian ideal type, would address the inconsistencies around internal and external validity raised as a product of the methodological critique and meta-analysis.

The media-centred model as proposed had two main characteristics. First, it

addressed questions of external validity by proposing tests to establish the proximity between the target population (represented by a randomly-selected sample of respondents)

and the media (represented by a subgroup of media taken from a larger population of all

media to which respondents could be potentially exposed). The qualities for assessing

proximity were identified as geographic, technological and temporal. In this manner,

media sampling could be generated using a means that was more replicable and empirically justifiable than previous approaches. Second, it called for a more nuanced role for media in

how researchers operationalize the transfer of salience by differentiating media content

based on a combination of externally visible cues such as prominence and placement, as

well as audience exposure to and consumption of that content from various media, using

either market research data or consumption questions contained in the public opinion

survey. In short, the media-centric model would reestablish and reinvigorate the role of

media as the independent variable in media agenda-setting research designs and,

accordingly, make the design more relevant to the 21st century realities of how people view

and consume news.

Despite its promise, there were several sobering lessons learned in implementing the media-centric model in the Waterloo region, and deserve to be highlighted for future 172 researchers. First, in an effort to achieve a media sample that had a high degree of proximity to respondents—including a mix of local and national media, across different media platforms, and conducted over a longer time frame to account for lag effects—the media content analysis portion of the study yielded one of the biggest media sample sizes on record. While the sample could have been reduced by sampling for only high- prominence items, it still yielded a large sample, which will likely have repercussions for successfully replicating this approach. Second, despite the high degree of proximity achieved, there were still notable gaps. The media consumption part of the public opinion survey revealed several significant sources that respondents said they turn to for news and information that were not included in the survey, with four in particular: a) the local community press, b) international news sources, especially U.S. cable network news, c) local commercial radio stations, and d) the many sources available from the internet.

The findings from the survey concerning the internet deserve special mention due to their future significance to agenda-setting research design and study. First, the number of respondents that claimed to get their news from internet sources was low compared to mainstream sources. As illustrated table 5-1, there was an almost inverse relationship between internet usage as a news source compared to the other three, traditional sources of

29 news. Second, while the population of television, radio and newspaper media potentially available to a group of respondents can be generally identified, there are an almost infinite

29If anything, the results of media consumption raise more questions about why researchers have not included already-existing, widely prevalent of and oft-used sources of news and information—namely, radio—in examining the media's agenda-setting effects, certainly more so than whether they should include new media technologies such as blogs, news aggregators or other sources of news. number of internet news sites available, making it difficult to establish the boundaries for a proximity test. Third, while respondents may gravitate to known media brands, the opinion survey suggested much less concentration of viewers around a few outlets compared to newspaper, radio and television. For example, the globeandmail.com site—among the most popular selected—was cited by only 4% of respondents. Fourth, in general, the analysis indicated no statistically-significant difference in the rank-order of issues between high and low users of specific types of media, including the internet. Fifth, other research into the usage of the internet as a news source relative to traditional media (NADBank

2008, Canadian Internet Project 2008) suggests that rather than replace traditional forms of news media, internet sources are complementing it. Finally, the research design, in an effort to produce a sample of higher consumers of news, chose to restrict its sample to an age group of adults 25 years of age and older, which excluded a group that likely would be greater users of the internet. As a result of these points, the agenda-setting effect of the internet observed in the study was less significant, and other factors, particularly the nature of specific events and issues and how they were covered, or the coverage offered by local media generally, were observed to be more influential than media type.

Table 5-1 Percentage share of media usage per week by respondents

Per week usage Internet Newspaper Radio Television

Seldom or never 60% 28% 26% 24% 1-2 times per week 11% 19% 12% 17% 3 or more times 29% 53% 62% 59% TOTAL 100% 100% 100% 100% N=2051 174

The results of the implementation of the model tended to confirm the importance of proximity in establishing the media sample. While there was an interesting correlation in the news agendas of disparate national media organizations, from CTV, to CBC-TV, to

CBC-Radio, and a similar correlation between equally disparate local sources, there was no correlation between local and national news agendas. Yet there was a statistically significant relationship between local news agendas and selection of top local issues.

Moreover, local news consumption relative to national news was noted to be high, particularly concerning supperhour broadcasts, local radio stations and the local newspaper.

The study of the public's rank-ordering of several top issues, such as the economy, may have been driven by strong, sustained local media coverage during a period in early 2008 when the national media had not yet turned significant attention to economic issues, at least in terms of the pronounced downturn in the southern Ontario manufacturing sector that was almost daily fare in the Waterloo media.

The model proved to be less useful when it came to operationalizing the transfer of salience. The various strategies undertaken to weight data based on valence cues and audience exposure and consumption factors yielded results that suggested a slightly stronger agenda-setting effect, but the results were neither consistent nor statistically significantly different from using unweighted data. This result may be a Type 2 error: a reflection of an error in the model itself or a problem in the calculations and underlying assumptions used in weighting the media content data—either of which might suggest that differentiating data based on external cues and audience exposure factors may still hold merit. However, another quite plausible explanation is that the tests suggest that the most 175 externally obvious cues about media salience may not be the most important in explicating the transfer of salience, and that other factors should be examined. The media-centric model still likely does not go far enough in addressing Kiousis and others who have criticized agenda-setting research designs as viewing salience in too rudimentary a fashion.

Certainly, others have argued (Ghanem, 1996; Ghanem, 1997; Coleman & Banning, 2006) that less intrinsically-observable factors should be pursued to illuminate how salience is transferred. These methods, which arguably could include frame-analysis as suggested by

McCombs himself (McCombs & Ghanem, 2001), may provide a better explanation of how salience is transferred than is offered strictly within the theoretical framework of agenda- setting. These methods could and should be adopted into a media-centric model that still preserves the importance of media sample validity.

The dissertation has focused on the methodological - specifically, the role of media

in agenda-setting research designs - and in that limited context, makes a meaningful contribution to our understanding of the strengths and weaknesses in past designs, and offers a new approach to modeling media within the context of agenda-setting research

intended to address some of its past, present and undoubtedly future shortcomings concerning internal and external validity.

However, the discussion about agenda-setting's methodology serves as a backdrop to highlight more fundamental theoretical issues about what the future has in store for the broader study of primary media effects. Along with agenda-setting, primary effects include such models as priming, issue-attention cycle, knowledge gap, knowledge activation models, framing and cultivation analyses. Studies into primary media effects share in 176 common two main characteristics. First, they focus primarily (but not exclusively) on the relationship between media and a defined public. Second, they usually model that relationship using quantifiable and statistically verifiable results generated by field and/or laboratory experimentation using representative forms of media and public. Agenda- setting, due to its longevity and prodigious publication record, can easily be considered as one of the leading, as well as the most straightforward examples of research in this area.

While there is evidence to suggest that academic interest into agenda-setting may now be diminishing (Bryant & Miron, 2004), the fact still remains that it was, and is, an important area of research into media effects that still draws interest.

While theoretical considerations tend to drive methodology, there is a reciprocal influence that must be recognized. Methodological considerations taken in past research set boundaries for what can be concluded about the theory in question and, in turn, affect the trajectory and directions of a theory's development. Primary media effects, if viewed as a theory about how media influences and/or affects public perceptions, is arguably more affected by methodological considerations. Several examples about how agenda-setting has evolved are evidence of this fact. First, the tremendous growth of agenda-setting as an area of study, while certainly interesting for what it allowed us to understand about public awareness of issues, could easily be explained by the basic characteristics of its research design: the marrying of two highly-accessible and established forms of empirical research

30Arguably other common characteristics of primary effects is that they are theories that have largely originated from American scholars, and that many were first published within the context of communications research within a relatively brief period spanning the mid 1960s and the mid-1970s. Ill

(content analysis, opinion polling) yielding statistically-significant, replicable results.

Second, the bias towards national media in many studies and the dearth of sampling around others, such as local radio, may be partly explained by the fact that the media that is studied is not necessarily the most relevant, but the media that is more accessible to researchers through resources such as the Vanderbilt University television archive, or now through online archives of content from major newspapers. Third, while agenda-setting did not

31 start out with Gallup MIP question, that question is now the major standard of defining issues, and its frequent use has to be partly attributed again to its accessibility to researchers rather than to its actual validity. In fact, 58% of agenda-setting studies surveyed in the meta-analysis used public opinion data obtained from third-party sources, usually commercially-available polls. The information available in these polls continually set limits and defines the construct of the media variable, as well as limiting what researchers can and cannot determine theoretically.

Agenda-setting, viewed as either a model or a theory, is at its most basic form simply a highly structured perspective of just one observable element among many that likely influence the relationship between media and audiences. While models such as agenda-setting inform us about assumptions about the media/audience relationship, it is in turn built on another set of assumptions that are implicit and rarely stated. In building their

31McCombs and Shaw (1972) in 1968 used a variation they borrowed from Trenaman and McQuail (1957): "What are you most concerned about these days? That is, regardless of what politicians say, what are the two or three main things which you think the government should concentrate on doing something about?" (178) This is a rather different question than the Gallup MIP question, which uses different language, framing the key term as "issue" rather than "concern" and eliminating the active political agency surrounding the role of government suggested in the Chapel Hill study. Since then, MIP has been the standard. See Min, Ghanem and Evatt, 2007. first experiment in 1968, McCombs and Shaw operated under these assumptions, which included: a) that the primary technological channels for delivering news were television, radio and print; b), that within these channels, respondents have access to relatively few, restricted, publicly known outlets; c) that for broadcast, the time available to view news was relatively constricted to certain periods; d) that the construction of news and information was conducted by a small set of professionally-trained journalists operating under a set of generally known ethics and standards; and e) that the audience of the news media was generally known, both to the media and, implicitly, members of the audience,

and was considered to be significantly large and representative of the general public.

These implicit assumptions about the nature of the relationship between news media and audiences were, and remain, fundamental to the concept of agenda-setting and to

many other areas of study of primary media effects. They are, as well, the very

assumptions that are threatened by media fragmentation and the advent of new media

channels and the changing relationships they create with their audiences. The changes in

the nature of news media and how audiences relate (or fail to relate) to them raises direct

questions about the very existence of a known and communicable media "agenda" in the

future, and how researchers respond to this theoretical challenge will be evaluated based on

how successfully they model "media" and "audience" within their research designs.

Adopting some of the principles and elements outlined in this dissertation concerning a

media-centred approach will, in all likelihood, become more crucial in determining how

valid and credible future study into media effects will be. Agenda-setting, as Kosicki

noted, may in fact be the future of media effects research. Just as agenda-setting itself 179 heralded a certain degree of paradigmatic change in the area of study of media effects from a more "limited effects" model, the ability or inability of researchers in this field to meet new challenges posed at the methodological as well as theoretical level about the role and nature of media will determine its future. 180

Appendix A

Media agenda setting meta-analysis code book

All articles that mentioned "agenda-setting" in the title or abstract within the five journals were included at this initial stage. N=110.

A. pin what academic journal was the article published? 1. International Journal of Public Opinion Research 2.Communication Research 3. Journalism Quarterly 4. Public Opinion Quarterly 5. Journal of Communication B. HPublication volume number C. B D. M E.p Author name (Last, First). Up to five authors. F-P G. BWas the public opinion component conducted through field research or laboratory experiment? 1. Field research 2. Laboratory 3.Other/not applicable: (methodological review, discussion, historiography) H. MWhat type of agenda setting model does the article principally address? 1. Media-Public 2. Media-Policy 3. Public-Policy 4. Intermedia - agenda building 8. Other

9. Not applicable 181

I. • Only articles that were coded for E(l) "Field Research" and F(l) "Media-Public" were included for the following questions. N=50.

J. MDid the research design examine a single issue or multiple issues? 1. Single issue 2.2-3 issues 3.4-9 issues 4.10 or more 8. Other 9. N/A K. If Did the research design also examine sub-issues of the main issue? A sub-issue would be any issue that would be identified by the researcher as a sub-issue of a broader category. 0.No sub-issues 1. Included sub-issues L.SpWhat is the range of the media content analysis sample, in number of days? M. fPDs the sampling of the media content analysis continuous, or is it segmented into distinct and separate periods of time? 0. Segmented 1. Continuous N. Bis the focus of the study national in scope? "National" would include media outlets with a national audience reach, or a collection of outlets that was, in aggregation, intended to represent the national media. 0.Not national (local, regional, international) 1. National O. JBDoes the research design incorporate more than one type of media, or does it limit its analysis to a single medium? 0.1 media 1.More than one media P.pList the number of outlets sampled for each of the following categories: 1. National newspapers 2.Local newspapers 3.News magazines 182

4.Radio 5.National television 6.Local television 7. Internet Q. BList the media outlets sampled in the research design. R. BAre different types of media treated differently in the analysis, or are all media aggregated? 0.No differentiation, aggregated. 1. Differentiated, but equally weighted 2. Differentiated and weighted accordingly 8. Other 9.N/A S.llWhat is the unit of analysis? 0. Entire article 1. Paragraph 2. Statement/sentence 3. Story 8. Other 9.N/A T.iBDoes the research design employ a method of sampling the media coverage? 0.No sampling - all items included 1.Categorical sampling (i.e., only front pages) 2.Random sampling 3.Systematic sampling 8. Other 9.N/A U. pif T(1 thru 8), describe the method of sampling used. V. BDoes the research design identify externally visible media cues such as prominence, use of photos, or page/section placement? 0.No cues 1.Uses cues 183

W. JRDoes the research design weight the media data based on externally visible cues or other information? 0.No weighting: uses simply frequency as measurement criteria 1. Weight: based on size/duration of item 2. Weight: based on prominence/placement of item 3.Weight: based on potential audience exposure 4.Weight: based on salience formula 8.Weight: other 9. N/A X. BDoes the research design take into account issue attributes (secondary-level agenda-setting elements) in examining media content? 0.No second-level effects 1. Includes second-level effects Y. PDescribe/list the issue attributes analyzed in the research design. Z.KDoes the research design examine the affective dimension of media coverage? 0.No affective dimension 1. Affective dimension probed AA. pDescribe how the research design defines the affective dimension? How does it define "favourable" versus "unfavourable?" BB. Slow is data on public opinion and the public agenda obtained in the research design? 0. Conducted by researcher 1. Obtained from third-party 8. Other 9.N/A CC. Hi generating the public opinion data, does the research design employ a convenience sample, a panel, or is it generated through random sampling? 0. Convenience 1.Random sampling 2. Panel 8. Not determined 9.Not applicable 184

DD. BDoes the research design employ a Gallup-type open-ended MIP (Most important question) that determines salience of issues among respondents? 0.No central question 1. Employs single, open-ended MIP question 2.Employs close-ended question (Likert scale) 3.Employs combination 8. Other 9.N/A EE.liype verbatim the MIP question used in the research design. FF. BDoes the research design or results of the study take into account issue obtrusiveness? 0.No test/consideration for issue obtrusiveness 1.Tests/considers issue obtrusiveness GG. BDoes the research design or results of the study take into account respondent demographics. 0.No demographics tested/discussed 1. Demographics tested, not part of results 2.Demographics part of results 8. Other 9.N/A HH. Prescribe the demographic variables probed in the research design. II.pDoes the study take into account exogenous variables and/or 'real world' indicators in analyzing results? 0.No test for real world/exogenous variables 1. Tests for realworld/exogenous variables JJ. pBescribe the real-world/exogenous variables used in the study. KK. BDoes the research design include media consumption questions for respondents or other methods to determine the media outlets used by respondents? 0.No media usage/consumption questions 1. Employs media usage/consumption questions LL. prescribe the media consumption questions used/discussed in the study. MM. BEnter the number of distinct sampling periods that were examined in gathering the public opinion data portion of the study. 185

NN. pf more than one distinct sample period was used, how does the research design or the analysis treat the longitudinal data? 0.No longitudinal treatment of data 1. Cross-lagged correlation 2. Granger analysis 3. Pre/post test 8. Other 9. N/A MM. IDoes the media content analysis precede and/or coordinate with the sampling of public opinion? 0.Gap between content analysis and public opinion survey 1. Content analysis conducted in coordination with public opinion survey 8. Other 9. N/A NN. IDoes the research design or results test for a time lag effect between the media and public opinion sample? 0.No test/discussion of time lag 1.Tests/discusses appropriate time lag 8. Other 9.N/A OO. IDoes the research design conduct and report intercoder reliability testing among coders for the media content analysis portion of the study? 0.No intercoder reliability tests 1. Percentage 2.Scott's Pi 3.Cronbach's Alpha 4.Krippendorff s Alpha 5.Pearson's R 8. Other 9. N/A 186

Appendix B

Media-centric model Media content analysis - Code book: Print/Internet Media

Applicability

Do not code the following items: • Index items • Advertorials • Corrections • Column briefs • Letters to the editor • Items in the sports, business, entertainment, life, classified or any other section other than Front, Local and Op-Ed section A. Bibliographic Information

Al. Entry number [F4] A2. Day [F2] (of publication date) A3. Month & Year (of publication date) mm.yy format A4. Media outlet [F4] 1050 Kitchener-Waterloo Record 107 7 Globe and Mail 107 8 Toronto Star 0050 KWRecord Online 0077 globeandmail.com A5. Page number [S4] A6. Section [Fl] Name the section in which the article appears 0 N/A 1 Front 5 Op-ed 7 Local/City A7. What type of print item is it? [F2] Editorial cartoon 2 Staff writer 3 Op/Ed 4 Columnist 5 Editorial 187

6 Letter 7 CanWest 8 Canadian Press 9 Presse Canadienne 10 Reuters 11 Dow Jones 12 AP 13 Bloomberg 14 Canada News Wire 15 Hollinger 16 Sun News 17 StarService (Toronto Star, Hamilton Spectator, Guelph Mercury 18 UPI 19 AFP 20 New York Times 21 Freelance 22 Knight-Ridder 23 Staff photographer 24 Domestic wire photographer 25 Foreign wire photographer 98 Undeclared/unknown 99 Other

A8. Photo/illustration [Fl] Does the item include a photo or illustration? 0 No 1 Yes - minor element to story - less than one-third of story size 2 Yes - major element to story (proximate size: greater than one-third of story size) A9. What is the prominence of the story? [Fl] Where in the newspaper is the story located? 1 Above-fold - primary • (the main story on the page) • Only one story per page can be given this designation • It is the top-most story on the page • If two stories share the top space, it is the one given more space or larger typeface for the headline. 2 Above-fold - secondary • (above the fold, but not the main story on the page) • More than one story can have this designation. • If it runs both above and below the fold, can be considered above fold. 3 Below fold - minor A10. Location of byline Where is the story written? Location given directly under headline. If no location given, do not enter. Type in as given by newspaper. 188

B. Topic

Bl. What is the main topic of the news item? 1 Agriculture & food Farming issues, crops (non consumer)

2 Budget/finance/taxes 3 Business/industry issues Affecting individual businesses or industries: restructuring, expansions

4 Charity/philanthropy 5 Citizenship/human rights aboriginal rights, immigration/refugee issues, judicial rights, death penalty

6 Consumer issues Prices, product safety, service

7 Crime Murder, robbery, fraud, white collar, hate, arson

8 Economy job growth, currency, trade figures, interest rates, commodity prices

9 Entertainment Hollywood, arts, literature, theatre

10 Environment Incentives, air/water/ground pollution, landfills, energy issues, alternative energy

18 Fires/flooding/natural disasters (affects group) destruction, significant loss of life/injuries from accidents, natural calamities,

12 Historical. 13 Housing/property 14 Human interest/ achievement focus on individual: overcoming adversity, challenge, achievement, obituaries

17 Labour .... Strikes, wages, working conditions

32 Lifestyle Family issues (not in lifestyle section)

33 Medical Research 19 Parks and recreation hours, rules, changes

21 Personal/individual accidents (affects individual/family), fires, drownings, MVAs 189

34 Policing Non-crime: budgets, treatment of prisoners

22 Politics Elections, reform, leadership, fed/prov relations

23 Post-secondary 24 Public education Elementary to high school, curriculum, class sizes, teaching issues, safety in schools

11 Public health care Hospitals, health care delivery, doctor shortages

16 Public infrastructure Local building development, new roads, bridges, markets

25 Religious 26 Scandal/mismanagement 27 Science & technology 35 Sports Sport issues (not in sports section)

28 Terrorism and security Acts of terrorism, preventative measures, threats, border security issues

29 Transportation/transit traffic issues, road safety issues

30 Travel/tourism 31 War/conflict Afghanistan, Iraq, military issues

36 Welfare/social services 37 Weather Non-environment: snowstorms, heat wave, etc.

99 OTHER If Other, please type in a term that best describes the topic.

B2. What is the geographic scope of the article? The geographic scope should be proximate to the protagonist, conflict or subject of the news story. If it includes more than one geographic scope, choose the widest possible scope one.

0 No scope 1 Local - Kitchener/Waterloo/Cambridge area 190

2 Other SW Ontario (Perth, London, Middlesex, Hamilton, Guelph, Woodstock, Brantford, Fergus)

3 Other area in Ontario/all Ontario 4 Other province 5 National (Canada) 6 U.S. (and U.S. and Canada issues) 7 North American 8 International/global B3. Country If Scope<=5, form will fill in Canada and proceed to summary. If not, fill in the country that the story is primarily located in. If more than one country, use byline as a guide. See attached list of country codes.

B4. Domestic angle For items that are located primarily in a country outside of Canada, but which have a domestic angle.

0 No domestic angle 1 Canadian angle 2 Local KW angle

B5. Does the item discuss either crime, the environment or Canada's involvement in Afghanistan? Can include items that only briefly mention one of the three subjects, as well as items that are primarily about these subjects. Form will skip to the section concerning secondary agenda-setting effects.

1 CRIME 2 ENVIRONMENT 3.... CANADA IN AFGHANISTAN

CI. Attributes: Secondary agenda-setting information: CRIME

How prominent is CRIME noted? 0 Buried - not subject of story 1 Prominent - subject of story What type of crime does it concern? If 99=Other, enter in the term that best describes the type of crime. 191

1 Murder/manslaughter 2 Robbery 3 Fraud/Identity theft 4 Rape/Sex crimes 5 Assault 6 Domestic violence 7 Kidnapping 8 Hate crime 9 Drugs/alcohol 1 0 Motor vehicle 1 1 Vandalism 1 2 Arson 99 Other Is there a victim identified? 0 No victim 1 Victim identified/cited 2 Victim quoted 3 Victim profiled/subject of article What is the level of violence described in the news item? Violence defined as the use of or existence of force intended to hurt or damage a subject. Should include anything in which physical or psychological damage to a person, place or thing has been inflicted.

0 No violence 1 Violent - property damage 2 Violent - personal/human damage

C2. Attributes: Secondary agenda-setting information: CONFLICT IN AFGHANISTAN

How prominent is Canada's involvement in Afghanistan noted? 0 No Canadian involvement noted 1 Buried - not subject of story 2 Prominent - subject of story What level of casualty is identified in the report? 0 No casualties noted 1 Afghan casualties, briefly noted 2 Intl casualties, briefly noted 3 Canadian casualties, briefly noted 4 Afghan casualties, prominent 5 Intl casualties, prominent 6 Canadian casualties, prominent

Does the item offer positions about Canada's involvement in Afghanistan? 0 No position

1 Expresses or highlights support for Canada's involvement 2 Balances pro/anti arguments 3 Expresses or highlights opposition for Canada's involvement 9 Notes debate - no position 193

Appendix C

Content analysis: intercoder reliability testing procedures and results

Intercoder reliability for the media content analysis portion of the agenda-setting study was tested among the three trained coders for the project at two stages: prior to the start of the study using to determine accuracy among the coders, and at a mid-way point during the analysis to confirm that accuracy was being maintained.

The statistical coefficient chosen was Krippendorff's a. As discussed by Krippendorff (1995, 2003), Krippendorff's a is superior to other commonly used coefficients to determine intercoder reliability such as Scott's n or Cohen's K, and allows for multiple coders and multiple responses.

The sample was restricted to newspaper items only from the Waterloo Region Record, Globe and Mail, National Post and Toronto Star. There were 139 items randomly selected from the sample between February 1, and May 31, 2008, that were coded by all three coders. Two variables were tested for intercoder reliability: topic/issue of item, and scope (geographic location), as these were the two variables that were integral to the analysis and were most open to subjectivity by respondents. The 36 available topics were recoded to the eight central topic/issue categories used in the analysis: the economy, health care, environment, war/conflict, terrorism/security, consumer issues, crime, education, with all other topics coded as "other". Scope consisted of five choices: local, provincial, national, U.S. and international.

There was 78% agreement on topics after the recoding for Krippendorff s a of 0.78, while scope had a 68% agreement for a Krippendorff s a of 0.71. Both agreement levels were considered acceptable. A review of areas where there was disagreement on topic tended to be present around stories that concerned both environment and economic topics, differences in war/conflict compared to security/terrorism particularly as it related to 194 international stories (which were not included as part of the agenda-setting study) as well as a story involving the medical inquest into the conduct of pathologist Dr. Charles Smith, with coders disagreeing on whether this constituted a public health or crime issue. 195

Appendix D

Media-centric model: Public opinion questionnaire

Hello, this is [INTERVIEWER - FIRST NAME] We are calling a random sample of residents only in the Waterloo region, about their views about local and national issues. The results are going to become part of a student university thesis. The survey will take approximately ten minutes depending on how you answer each question. If you have any questions about this questionnaire, you can contact Andrew Laing at (xxx) xxx-xxxx. Andrew Laing is sponsoring this research. No individual names will be included in the research report, your answers will be aggregated with all the other people who answer the survey. The data will not be used for commercial purposes.

Screener

To pick someone at random in your household, may I please speak to the person in your household who is 25 years of age or older and whose birthday comes soonest after today's date? [RE-INTRODUCE YOURSELF AND CONFIRM AGE OF RESPONDENT IF NECESSARY. IF NOT AVAILABLE, OBTAIN FIRST NAME AND ARRANGE CALL BACK DATE AND TIME. IF NO ONE OVER AGE 25, THANK AND TERMINATE], [IF RESPONDENT SOUNDS YOUNG AND HESITATES OVER AGE, CONFIRM YEAR AND MONTH OF BIRTH. IF RESPONDENT IS NOT YET 25, TERMINATE],

Questionnaire

Qla. In your view, what is the most important national issue facing Canadians today? [OPEN END. RECORD ONE RESPONSE.] 1.

Unemployment Health care

Economic Growth /Industrial Development / Expansion Gas Prices

National environment

Highways

Out Migration 196

National taxes

Revitatlization / Construction of downtown

Cost of Living

Crime in Canada

Cost of Energy to heat homes

Local taxes

Canada's involvement in Afghanistan

National security

Cost of university education

Quality of grade school education

Corruption in government

Qlb. If you could choose another important national issue facing Canadians today, what would it be? [OPEN END. RECORD ONE RESPONSE.] 1.

Q2a. In your view, what is the most important local issue facing residents of the Waterloo region today? [OPEN END. RECORD ONE RESPONSE..] 1.

Q2b. If you could choose another important local issue facing Waterloo region residents today, what would it be? [OPEN END. RECORD ONE RESPONSE..]

Unemployment Health care 197

Economic Growth Industrial Development / Expansion Housing affordability / availability

Gas Prices

National local environment

Roads condition

Transit in the area

Out Migration

Local taxes

Revitatlization / Construction of downtown

Cost of Living

Local crime

Cost of Energy to heat homes

Local taxes

Canada's involvement in Afghanistan

National security

Cost of university education

Quality of grade school education

Corruption in government

Q3. How important to you are each of the following issues, on a scale of 1 to 10, where 1 is

Not at All Important, and 10 is Very Important.

[ROTATE]

a. Crime 123456789 10

b. Canada's involvement in Afghanistan 123456789 10 198

c. The environment 2 3 4 5 6 8 9 10

d. The economy 2 3 4 5 6 8 9 10

e. Energy prices and supply 2 3 4 5 6 8 9 10

f. National security 2 3 4 5 6 8 9 10

g. Health care 2 3 4 5 6 8 9 10

h. Public education 2 3 4 5 6 8 9 10

i. Corruption in government 2 3 4 5 6 8 9 10

j. Taxes 2 3 4 5 6 8 9 10

k. Roads and transit 2 3 4 5 6 8 9 10

I. Housing affordability 2 3 4 5 6 8 9 10

Q5. Thinking about news coverage you may have seen or heard in the past three months, what in your view, are the three most important national stories? [OPEN END. RECORD UP TO THREE

RESPONSES.] IF RESPONSE IS AN ISSUE (IE. CRIME/ ECONOMY) PROBE

FOR A STORY: "WHAT SPECIFIC STORY HAVE YOU SEEN OR HEARD ON THIS ISSUE".

READ IF NECESSARY: AN ISSUE IS A TOPIC IN GENERAL. A STORY IS A SPECIFIC NEWS ITEM

ABOUT AN ISSUE.

Q6. These provincial or national stories have recently been covered in the news. For each story, would you say that you... [READ LIST] 199

know or have heard a lot about it 01

know or have heard a little about it 02

have not heard about it 03

Q7.Thinking about news coverage you may have seen or heard in the past three months, what in your view, are the three most important local stories? [OPEN END. RECORD UP TO THREE RESPONSES.]

IF RESPONSE IS AN ISSUE (IE. CRIME/ ECONOMY) PROBE FOR A STORY: "WHAT SPECIFIC STORY

HAVE YOU SEEN OR HEARD ON THIS ISSUE" AN READ IF NECESSARY: AN ISSUE IS A TOPIC IN GENERAL A

STORY IS A SPECIFIC NEWS ITEM ABOUT AN ISSUE..

Q8.These local stories have recently been covered in the news in the Waterloo region. For each story, would you say that you... [READ LIST]

know or have heard a lot about it 01

know or have heard a little about it 02

have not heard about it 03

In general, how often do you.... [READ LIST. FOR FIRST ITEM IN LIST, ASK THE FREQUENCY SCALE. REPEAT FREQUENCY SCALE AS NEEDED.] 200

[RANDOMIZE]

a. Read or look at a newspaper 01

c. Watch television news 03

d. Listen to news on the radio 04

e. Read news on the Internet which may include news blogs or 05 downloading news to an IPOD or PODcasts,

[FREQUENCY SCALE: ] Would that be....

3 or more times a week 01

1 or 2 times a week 02

Seldom or never 03

[IF READ A NEWSPAPER (code 1 or 2 in Q9a) ASK:]

QlOa. Which newspapers did you read or look at TODAY or YESTERDAY? [DO NOT READ; CODE TO LIST], QlOb. Did you read or look at any other newspaper today or yesterday? [ACCEPT ALL MENTIONS; DO NOT PROBE.].

Unaided OTHER Unaided MENTIONS

QlOa QlOb

The Globe and Mail 01 01

Kitchener-Waterloo Record 02 02

National Post 03 03 201

Toronto Star 04 04

The Hamilton Spectator "The 05 Spec" 05

The Cambridge Times 06 06

Other [Please describe] 07 07

None 09 09

[IF WATCHED TV NEWS (code 1 or 2 in Q9c) ASK:]

Qlla. Which TV news casts did you watch TODAY or YESTERDAY? DO NOT READ; CODE TO LIST], [IF STATION MENTIONED, BUT NOT PROGRAM, PROBE TO GET NAME OF PROGRAM AS SHOWN IN LIST] Qllb. Did you watch any other TV newscast today or yesterday? [ACCEPT ALL MENTIONS DO NOT PROBE.].

Unaided

Unaided Qllb

Qlla OTHER

MENTIONS

CBC : The National with Peter Mansbridge

CKCO's Latenight news (CTV channel 12) 01 01

CKCO News at Noon (CTV channel 12) 03 03

CKCO News at Six (CTV channel 12) 04 04

Global Television's Morning News 05 05

Global Television's News at Noon 06 06 202

Global Television's Global News at 6 07 07

Global Television's First National with Kevin 08 08 Newman

Global television's latenight newscast at 11:00pm

CTV Canada AM 09 09

CTV National News with Lloyd Robertson 10 10

Other [Please describe] 19 19

None 13 13

[IF LISTENED TO RADIO NEWS (code 1 or 2 in Q9d) ASK:]

Q12a. Which radio stations did you listen to TODAY or YESTERDAY for news or discussion of important issues? [DO NOT READ; CODE TO LIST], Q12b. Did you listen to any other radio stations today or yesterday for news or discussion of important issues? [ACCEPT ALL MENTIONS; DO NOT PROBE.].

Unaided Q12b Unaided OTHER Q12a MENTIONS

CBC Radio 01 01

570 News Radio 02 02

CFRB 1010 (AM) 03 03

680 NEWS (AM) 04 04

Other: Specify (include even partial 05 05 203

info such as call letters or station

name)

[DO NOT READ: REFUSED] 09 09

None 06 06

[IF READ NEWS ON A WEBSITE (code 1 or 2 in Q9e) ASK:]

Q13. Did you read any of the internet news sites from major Canadian media TODAY or YESTERDAY? [DO NOT READ, CODE ALL THAT APPLY TO LIST. PAUSE FOR RESPONDENT TO LIST THEM, AND IF RESPONDENT DOESN'T NAME ANY SITES, ASK: ] Which ones?. [ACCEPT ALL MENTIONS DO NOT PROBE.].

Unaided

ALL

MENTIONSQ1

3

Canada.com 01

CBC.ca 02

Globe and Mail.com 03

MSN Sympatico 04

Yahoo 05

Kitchener Waterloo Record "The Record".com 06

The Hamilton Spectator "The Spec" .com 07

National Post 08

CTV news 09 204

Google News 10

Other: Specify 11

[DO NOT READ: REFUSED] 99

None 13

Q14. How often do you discuss news and events with your friends, family or co-workers?

READ LIST

Often 01

Occasionally 02

Almost never 03

[DO NOT READ: REFUSED] 09

The next few questions are just so we can group your answers with those of other people.

Q15. Which category best describes your level of education that you have reached so far: [READ LIST IF NECESSARY],

Some elementary 01

Completed Elementary 02

Some High School 03 205

Completed High School 04

Some Community College/ 05 vocational/technical school

Completed Community College/ 06 vocational/technical school

Some university 07

Completed university 08

Some post-graduate school 09

Completed post-graduate school 10

[DO NOT READ: REFUSED] 99

Q16. In which of the following age categories do you belong? [READ LIST.]

35 to 44 55 to 64

25 to 34 45 to 54 65 or Better

Q17. [RECORD GENDER. ASK IF NECESSARY.]

Female 01

Male 02

Q18. Were you born.... [READ LIST. CHECK ONE ONLY.] 206

in the Waterloo region, OR 01

Somewhere else in Ontario, OR 02

Somewhere else in Canada, OR 03

Somewhere else in North America, OR 04

outside North America 05

[DO NOT READ: REFUSED] 09

Q19. How many years have you lived in the Waterloo region? [IF

RESPONDENT SAYS "ALL MY LIFE" CODE 20 OR MORE YEARS]. [CODE EITHER ACTUAL NUMBER IF THEY GIVE

IT OR A RANGE IS FINE]

Under a year 01

1 to under 2 years 02

2 to undedr 5 years 03

5 to under 10 years 04

10 to under 20 years 05

20 or more years/ All my life 06

[DO NOT READ: REFUSED] 09

Thank you very much for participating! 207 208

Appendix E

S>

The following provides a description of the calculations used to assign weights to individual media items in the content analysis portion of the study. Three weights were created:

a) A weighting based on available market research data for the outlets surveyed, using BBM Bureau of Measurement's survey of broadcast outlets and NADBank's survey of newspapers, further weighted based on the placement and prominence of the item in the media outlet;

b) A weighting based on self-response data obtained from the public opinion survey concerning media consumption of surveyed media outlets; and

c) A weighting based a hybrid of the self-response data that would include a further valence calculation incorporating the placement and prominence of the item in the news outlet.

The description of the calculation is provided in bolded text, beginning with an asterisk (*). Unbolded text in smaller font refers to SPSS syntax commands used to generate the weightings, for researchers who may want to try to duplicate the weighting formula used in this study. The market research data, using BBM Bureau of Measurement and NADBank audience reach figures, are difficult to obtain and less replicable for academic researchers. However, the self-response data and the hybrid use data obtained almost entirely from the media-centric model and, consequently, are easily replicable.

Researchers may want to omit the step in the hybrid that uses part of the market research data to ascribe weights to the valence calculation, and substitute their own sliding scale. 209

B

*Outlets and time values are recoded into their relevant programs.

if (outlet="CBC-TV"&range(timeval,time.hms(21,00,00),time.hms(23,05,00))) program=l.

if (outlet="CTV-TV"&range(timeval,time.hms(22,55,00),time.hms(23,35,00))) program=3.

if (outlet="Global-TV"&range(timeval,time.hms(18,00,00),time.hms(19,05,59))) program=6.

if (outlet="Global-TV"&range(timeval,time.hms(17,30,00),time.hms(18,59,59))) program=7.

if (outlet="CKCO-TV"&range(timeval,time.hms(ll,55,00),time.hms(13,05,00))) program=10.

if (outlet="CKCO-TV"&range(timeval,time.hms(17,55,00),time.hms(19,05,00))) program=ll.

if (outlet="CKCO-TV"&range(timeval,time.hms(23,25,00),time.hms(23,59,59))) program=12.

if (outlet="CKCO-TV"&range(timeval,time.hms(0,0,0),time.hms(0,5,59))) program=12.

if (outlet="CBC-R"&range(timeval,time.hms(8,29,00),time.hms(10,05,00))) program=14.

if (outlet="CBC-R"&range(timeval,time.hms(17,55,00),time.hms(18,29,59))) program=15.

if (outlet="CBC-R"&range(timeval,time.hms(18,30,00),time.hms(20,05,00))) program=16.

if (outlet="CKGL-AM"&range(timeval,time.hms(7,55,00),time.hms(9,05,00))) program=17.

if (outlet="CKGt-AM"&range(timeval,time.hms(ll,55,00),time.hms(13,05,00))) program=18.

if (outlet="CKGL-AM"&range(timeval,time.hms(15,55,00),time.hms(17,05,00))) program=19.

*Using share of total potential audience based on ratings points for the program, each program is assigned a weight.

f (program=l) onexpbr5=0.08.

f (program=3) onexpbr5=0.11.

f (program=6) onexpbr5=0.02.

f (program=7) onexpbr5=0.02.

f (program=10) onexpbr5=0.05.

f (program=ll) onexpbr5=0.2.

f (program=12) onexpbr5=0.07.

f (program=14) onexpbr5=.028.

f (program=15) onexpbr5=.028.

f (program=16) onexpbr5=.028.

f (program=17) onexpbr5=.01. 210

if (program=18) onexpbr5=.01.

if (program=19) onexpbr5=.01.

fi

*Percentage share of the number of KW residents surveyed that said they read yesterday's paper, according to NADBank 2007 survey, is computed as reader5a.

*Also determines whether it is a front page article as entered by coders.

* Audience reach fluctuations for weekday versus weekend (Saturday -

Weekday=7) edition taken into account.

*WR/?=Waterloo Region Record, G&M=Globe and Mail, TORSTA/?=Toronto

Star, NATPOST=National Post.

do if (weekday ~= 7).

recode outlet (WRR=0.359)(G&M=0.044)(TORSTAft=0.038)(NATPOST=0.021) into reader5a.

else if (weekday=7).

recode outlet (WRR=0.390)(G&M=0.049)(TORSTAR=0.038)(NATPOST=0.012) into reader5a.

end if.

exe.

Var lab readerSa "Readership - weighted by time spent reading newspaper".

*Further weighted based on time spent reading each newspaper, both weekday edition & Saturday edition to compute TMRDLE variables.

*Impact of news story based on the amount of time newspaper readers spend reading newspapers. 211

*if on the front page & lead story, weighted item with full readership.

if (promin=l&front=l) reader5b=reader5a.

*If item on the front page but not the lead story, discounted readership by removing those that would spend less than 20 minutes reading a newspaper.

if (front=l&promin ~= 1) reader5b=reader5a*(l-tmrdle20).

*if item not on the front page, but is above the fold, discounted full readers by those that would spend less than 30 minutes reading the newspaper.

if (front ~= 1 & any(promin,l,2)) reader5b=reader5a*(l-(tmrdle20+tmrd2029)).

*if item not on the front page, & is below the fold, discounted full readers by those that would spend less than 60 minutes reading the newspaper.

if (front ~= 1 & promin=3) reader5b=reader5a*(l-(tmrdle20+tmrd2029+tmrd3059)). 212

*Readership was further weighted based on the location of the news story by section and byline, based on likelihood of respondents to read certain sections and/or topics of coverage, again using the NADBank 2007 survey of reader preferences.

*Based on NADBank 2007 readership survey in KW region, 75% respondents read local news (Scope, 1&2), 60% read the provincial/national news (Scope, 3,4,5), &

36% read the op-ed pages.

*Section=5 is opinion-editorial.

if (section=5) reader5c=reader5b*.36.

if (section ~= 5 & any(scope,l,2)) reader5c=reader5b*.75.

if (section ~= 5 & range(scope,3,5)) reader5c=reader5b*.60.

fc

*The internet weighting was based on a percentage of the newspaper weighting according to NADBank 2007 data.

* According the survey, 10% of respondents reported to have read the internet version of the Waterloo Region Record (WRRi) in the last seven days, while 4.7% reported reading globeandmail.com (G&Mi)

if (outlet=WRRi) onexpint5=0.1.

if (outlet=G&Mi) onexpint5=0.047. 213

* Weighting of internet data was further downgraded depending on the prominence of the news story and use of photos.

if (type="internet")&promin=l&photo=2) wt5onexp=onexpint5.

if (type="internet")&promin = l&photo=l) wt5onexp=onexpint5*.8.

if (type="internet")&promin ~= 1 &photo=2) wt5onexp=onexpint5*.7.

if (type="internet")&promin ~= l&photo=l) wt5onexp=onexpint5*.6.

if (type="internet")&promin ~= l&photo=0) wt5onexp=onexpint5*.5.

exe.

if (type="print") wt5onexp=reader5c.

if (type="broadcast") wt5onexp=onexpbr5.

Var lab wt5onexp "Market data weight".

P *From poll responses, each program was weighted based on the number of respondents who stated they either watched the news program or read the print or internet news outlet.

*Responses divided by the number of total respondents queried.

if (program=l) wt6onexp=294/2055.

if (program=3) wt6onexp=367/2055.

if (program=6) wt6onexp=103/2055.

if (program=7) wt6onexp=103/2055.

if (program=10) wt6onexp=118/2055.

if (program=ll) wt6onexp=407/2055.

if (program=12) wt6onexp=151/2055.

if (program=14) wt6onexp=283/2055.

if (program=15) wt6onexp=283/2055.

if (program=16) wt6onexp=283/2055.

if (program=17) wt6onexp=266/2055.

if (program=18) wt6onexp=266/2055.

if (program=19) wt6onexp=266/2055.

if (outlet=G&Mi) wt6onexp=78/2055. 214

if (outlet=WRRi) wt6onexp=38/2055.

if (outlet=WRR) wt6onexp=994/2055.

if (outlet=G&M) wt6onexp=223/2055.

if (o utlet=TORSTAR) wt6onexp=148/2055.

if (outlet=NATPOST) wt6onexp=91/2055.

exe.

Var lab wt6onexp "Self-Response Weight".

B

* Impact of news story based on the amount of time newspaper readers spend reading newspapers.

*if on the front page & lead story, weighted item with full readership based on

the self-response data outlined above.

IF (ANY(OUTLET,WRR,TORSTAR,G&M,NATPOST)) READER7A=wt6onexp.

if (promin=l&front=l) reader7b=reader7a.

Var lab reader2 "Readership - weighted by time spent reading newspaper".

*If item on the front page but not the lead story, discounted readership by removing those that would spend less than 20 minutes reading a newspaper.

if (front=l&promin ~= 1) reader7b=reader7a*(l-tmrdle20). 215

*if item not on the front page, but is above the fold, discounted full readers by those that would spend less than 30 minutes reading the newspaper.

if (front ~= 1 & any(promin,l,2)) reader7b=reader7a*(l-(tmrdle20+tmrd2029)).

*if item not on the front page, & is below the fold, discounted full readers by those that would spend less than 60 minutes reading the newspaper.

if (front ~= 1 & promin=3) reader7b=reader7a*(l-(tmrdle20+tmrd2029+tmrd3059)).

*Readership was further weighted based on the location of the news story by section and byline, based on likelihood of respondents to read certain sections and/or topics of coverage, again using the NADBank 2007 survey of reader preferences.

*Based on NADBank 2007 readership survey in KW region, 75% respondents read local news (Scope, 1&2), 60% read the provincial/national news (Scope, 3,4,5), &

36% read the op-ed pages.

*Section=5 is opinion-editorial.

if (section=5) reader7c=reader7b*.36.

if (section ~= 5 & any(scope,l,2)) reader7c=reader7b*.75.

if (section ~= 5 & range(scope,3,5)) reader7c=reader7b*.60.

compute onexppr7=reader7c. 216

if (type="print) wt7onexp=onexppr7.

*Broadcast data the same as in the self-response section.

if (type="broadcast) wt7onexp=wt6onexp.

*Internet data weighted based on same sliding scale as in print, but uses self- response data as the base.

if (type="internet"» onexpint7=wt6onexp.

if (type="internet")&promin=l&photo=2) wt7onexp=onexpint7.

if (type="internet")&promin = l&photo=l) wt7onexp=onexpint7*.8.

if (type="internet")&promin ~= 1 &photo=2) wt7onexp=onexpint7*.7.

if (type="internet")&promin ~= l&photo=l) wt7onexp=onexpint7*.6.

if (type="internet")&promin ~= l&photo=0) wt7onexp=onexpint7*.5.

var lab wt7onexp "Hybrid Self-response-Valence weight". 217

Bibliography

Aday, S. (2006). The framesetting effects of news: An experimental test of advocacy versus objectivist frames. Journalism & Mass Communication Quarterly, 83, 767-784

Ader, C. (1995). A longitudinal study of agenda-setting for the issue of environmental pollution. Journalism & Mass Communication Quarterly, 72, 300-311.

Althaus, S. L., & Tewksbury, D. (2002). Agenda setting and the 'new' news: patterns of issue importance among reader of the paper and online versions of the New York Times. Communication Research, 29,180-207.

Althaus, S. L., Edy, J. A., & Phalen, P. F. (2001). Using substitutes for full-text news stories in content analysis: which text is best? American Journal of Political Science. 45.707-23.

Asp, K. (1983). The struggle for the agenda: Party agenda, media agenda, and voter agenda in the 1979 Swedish General Election. Communication Research, 10, 333-355

Atwater, T., Salwen, S., & Anderson, R. (1985). Interpersonal discussion as a potential barrier to agenda-setting. Newspaper Research Journal, 6, 53-61.

Atwater, T., Salwen, S., & Anderson, R. (1985). Media agenda-setting with environmental issues. Journalism & Mass Communication Quarterly, 62. 393-397.

Bassili, J. N. (1995). Response latency and the accessibility of voting intentions: What contributes to accessibility and how it affects vote choice. Personal and Social Psychology Bulletin. 21. 686-695.

Becker, L., & McCombs M. E. (1978). The role of the press in determining voter reactions to presidential primaries. Human Communication Research, 4, 310-317.

Becker, L., McCombs, M. E., & McLeod, J. (1975). The development of political cognitions. In S. Chafee (Ed.), Political Communication: Issues and Strategies for Research (pp.21-64). Beverly Hills, CA: Sage.

Behr, R. & Iyengar, S. (1985). Television news, real-world cues, and changes in the public agenda. Public Opinion Quarterly, 49, 38-57.

Benton, M., & Frazier, P. J. (1976). The agenda-setting function of the mass media at three levels of informational holding. Communication Research, 3. 261-275. 218

Berelson, B., Lazarsfeld, P., & McPhee, W. (1954). Voting. Chicago: Chicago University Press.

Blood, R. W. (1980) Competing Models of Agenda-Setting: Issue Obtrusiveness vs. Media Exposure._Paper presented at the Annual Meeting of the Association for Education in Journalism (63rd, Boston, MA, August 9-13,1980).

Blood, R. W. (1980). Media agenda-setting theory: Points of departure. Paper presented at the Annual Meeting of the International Communication Association (39th, San Francisco, CA, May, 1989).

Blood, R. W. (1981). Unobtrusive issues in the agenda-setting role of the press. Unpublished doctoral dissertation, Syracuse University, New York.

Blumler, J. G., & McLeod, J. M. (1974). Communication and voter turnout in Britain. In M. Ferguson (Ed.), Sociological Theory and Social Research (pp. 265-312). London: Sage.

Boyle, T. P. (2001). Intermedia agenda-setting in the 1996 Presidential election. Journalism & Mass Communication Quarterly, 78, 26-44.

Breen, M. A. (1968). Ten leading newspapers rated for coverage of the 1967 Detroit riots. Journalism Quarterly, 45, 544-546.

Breen, M. J. (1997). A cook, a cardinal, his priests, and the press: Deviance as a trigger for intermedia agenda setting. Journalism & Mass Communication Quarterly, 74, 348-356.

Briggs, E., Nelson, R. & Soderlund, W. (1980). Regional and linguistic agenda-setting in Canada: A study of newspaper coverage of issues affecting political integration in 1976. Canadian Journal of Political Science 13. 347-356.

Brosius, H.-B., & Weimann, G. (1996). Who sets the agenda?: Agenda-setting as a two- step flow. Communication Research, 23, 561-580.

Brosius, H.-B., & Kepplinger, H. M. (1990). The agenda-setting function of television news: static and dynamic views. Communication Research, 17, 183-211.

Brosius, H.-B., & Kepplinger, H. M. (1992). Beyond agenda-setting: the influence of partisanship and television reporting on the electorate's voting intentions. Journalism Quarterly, 69, 893-901.

Brosius, H.-B., & Kepplinger, H. M. (1992). Linear and non-linear models of agenda- setting in television. Journal of Broadcasting & Electronic Media, 36, 5-23. 219

Brosius, H.-B., & Kepplinger, H. M. (1995). Killer and victim issues: Issue competition in the agenda-setting process of German television. International Journal of Public Opinion Research, 7, 211-231.

Brown, L. (1971). Television. New York: Harcourt Brace Jovanovich.

Bryant, J. (Ed.). (1986). Perspectives on media effects Mahwah, NJ: Erlbaum.

Bryant, J., & Miron, D. (2004). Theory and research in mass communication. Journal of Communication, 54, 662-704.

Buddenbaum, J. (2001). The media, religion, and public opinion: toward a unified theory of cultural influence. In D. A. Stout & J. Buddenbaum (Eds.), Relgions and popular culture: Studies in the interaction of worldviews (pp. 19-38). Ames, IA: Iowa State University Press.

Bystrom, D. G., Robertson, T.A., & Banwart, M. C. (2001). Framing the fight: An analysis of media coverage of female and male candidates in primary races for governor and U.S. Senate in 2000. American Behavioral Scientist. 44. 1999-2013.

Carmines, E.G„ & Zeller, R. A. (1979) Reliability and validity assessment. Newbury Park, CA: Sage.

Carragee, K., Rosenblatt, M., & Michaud, G. (1987). Agenda-setting research: A critique and theoretical alternative. In S. Thomas (Ed.), Studies in communication (Vol. 3, pp. 35-49). Norwood, MJ: Ablex.

Carter, R. F., Stamm, K. R., & Heintz-Knowles, K. (1992). Agenda-setting and consequentiality. Journalism Quarterly, 69. 868-877

Chafee, S. H. (1972). Longitudinal designs for communication research: Cross-lagged correlations. Paper presented to the Association for Education in Journalism annual conference, Carbondale, 111.

Chaffee, S. H., & Hocheimer, J. (1985). The beginnings of political communication research in the United States: Origins of the limited effects model. In M. Gurevitch & M. Levy (Eds.), Mass Communication Review Yearbook (Vol. 5, pp. 75-104). Beverly Hills, CA: Sage.

Chaffee, S. H., & Schleuder, J. (1986). Measurement and effects of attention to media news. Human Communication Research, 13. 76-107.

Chan, K. (1999). The media and environmental issues in Hong Kong 1983-95. International Journal of Public Opinion Research. 11,135-151. 220

Cobb, R. W„ & Elder, C. D. (1971). The politics of agenda-building: An alternative perspective for modern democratic theory. Journal of Politics, 33, 892-915.

Cohen, B. (1963). The press and foreign policy. Princeton, NJ: Princeton University Press.

Coleman, R., & Banning, S. (2006). Network TV news' affective framing of the presidential candidates: Evidence for a second-level agenda-setting effect through visual framing. Journalism & Mass Communication Quarterly, 83, 313- 328.

Cook, F„ Tyler, T„ Goetz, E„ Gordon, M„ Protess, D„ Leff, D., & Molotch, H. (1983). Media and agenda-setting: effects on the public, interest group leaders, policy makers, and policy. Public Opinion Quarterly, 47, 6-35.

Craft, S„ & Wanta, W. (2004). U.S. public concern in the aftermath of 9-11: A test of second level agenda-setting. International Journal of Public Opinion Research, 16,456-463.

Cronbach, L.J., and P. M. Meehl. (1955). Construct validity in psychological tests. Psychological Bulletin. 52.281-302.

Crouse, T. (1972). The boys on the bus. New York: Random House.

Curtin, R., Presser, S. & Singer, E. (2005). Changes in telephone survey nonresponse over the past quarter century. Public Opinion Quarterly, 69, 87-98.

Dalton R. (1998). Test of media-centered agenda setting: Newspaper content and public in a presidential election. Political Communication, 4, 463-481.

Dearing, J. W., & Rogers, E. M. (1996). Agenda-setting. Thousand Oaks, CA: Sage.

DeGeorge, W. F. (1981). Conceptualization and measurement of audience agenda in agenda-setting research. In G. C. Wilhoit & H. de Bock (Eds.), Mass Communication Review Yearbook 2, Beverly Hills: Sage.

DeMause, L. (1986). Why did Reagan do it? Journal of Psvchohistory, 14,: 107-118.

Demers, D. P. (1996). Does personal experience in a community increase or decrease newspaper reading? Journalism & Mass Communication Quarterly. 73. 304- 318.

Demers, D. P., Craff, D., Choi, Y. H., & Pessin, B. M. (1989). Issue obtrusiveness and the agenda-setting effects of national network news. Communication Research, 16, 793-812. 221

Devitt, J. (1999). Framing gender on the campaign trail: Women's executive leadership and the press. Washington D.C.: Women's Leadership Fund.

Downs, A. (1972). Up and down with ecology: the 'issue-attention cycle.' The Public Interest. 28. 38-50.

Eaton H., Jr. (1989). Agenda setting with bi-weekly data on content of three national media. Journalism Quarterly, 66, 942-948, 959.

Edelstein, A. (1993). Thinking about the criterion variable in agenda-setting research. Journal of Communication, 43. 85-99.

Einsiedel, E., Salomone, K., & Schneider, F. (1984). Crime effects of media exposure and personal experience on issue salience. Journalism Quarterly, 61,131-136.

Entman, R. M. (1993). Toward clarification of a fractured paradigm. Journal of Communication, 43,41-53.

Entman, R. M. (2007). Framing bias: Media in the distribution of power. Journal of Communication, 57,163-173.

Erbring, L., Goldenberg, E., & Miller, A. (1980). Front-page news and real-world cues: A new look at agenda-setting by the media. Public opinion quarterly. 24. 16-49.

Everitt, J., & Gidengil, E. (2003). Tough talk: How television news covers male and female leaders of Canadian political parties." In M. Tremblay & L. Trimble (Eds.). Women and Electoral Politics in Canada. (194-210). Don Mills, ON: Oxford University Press.

Eyal, C.H., Winter, J. P., & DeGeorge, W. F. (1981). The concept of time frame in agenda-setting. In G. C. Wilhoit & H. de Bock (Eds.), Mass Communication Review Yearbook 2. Beverly Hills: Sage.

Eyal. C. H. (1979). Time frame in agenda-setting research: A study of the conceptual and methodological factors affecting the time frame context of the agenda-setting process. Unpublished doctoral dissertation, Syracuse University.

Fan, D., & Tims, A. (1988). The impact of the news media on public opinion: American presidential election 1987-1988. International Journal of Public Opinion Research, 1,151-163.

Fan, D., Brosius, H.-B., & Kepplinger, H. (1994). Predictions of the public agenda from television coverage. Journal of Broadcasting & Electronic Media, 38.163-177.

Frey, L. R., Botan, C. H., & Kreps, G. L. (2000). Investigating communication: An introduction to research methods (2nd ed.). Toronto: Allyn & Bacon. 222

Funkhouser, G. R. (1973). The issues of the sixties. An exploratory study of the dynamics of public opinion. Public Opinion Quarterly. 37. 62-75.

Gadziala, S. M., & Becker, L. B. (1980). A new look at agenda-setting in the 1976 election debates. Journalism Quarterly. 57,122-126.

Gamson, W. A., & Modigliani, A. (1998). Media discourse and public opinion: A constructivist approach. American Journal of Sociology. 95, 1-37.

Gaziano, C. (1985). Neighborhood newspapers and neighborhood leaders: Influences on agenda setting and definitions of issues. Communication Research, 12, 568-594.

Geer, J. G. (1991). Do open-ended questions measure salient issues? Public Opinion Quarterly. 55. 360-370.

Gerbner, G. (1964). On content analysis and critical research in mass communications. In L. Dexter & D. White (Eds.), People, Society, and Mass Communications (pp. 476-500). New York: Free Press.

Ghanem, S. (1996). Media coverage of crime and public opinion: an exploration of second- level agenda setting. Unpublished doctoral dissertation, University of Texas, Austin.

Ghanem, S. (1997). Filling in the tapestry: The second level of agenda setting. In M. E. McCombs, D. L. Shaw & D. Weaver (Eds.), Communication and democracy: Exploring the intellectual frontiers in agenda-setting theory (pp. 3-14). Mahwah, NJ: Erlbaum.

Ghanem, S., & Wanta W. (2001). Agenda-setting and Spanish cable news. Journal of Broadcasting and Electronic Media, 45, 277-289.

Gidengil, E. & Everitt, J. (2000). Filtering the female: Television news coverage of the 1993 Canadian Leaders' debates. Women in Politics. 21. 105-31.

Gilberg, S., Eyal, C., McCombs, M. E„ & Nicholas, D. (1980). The state of the union address and the press agenda. Journalism Quarterly, 57, 584-588.

Gilens, M., & Hertzman, C. (2000). Corporate ownership and news bias: Newspaper coverage of the 1996 Telecommunications Act. Journal of Politics. 62.115-146.

Goffmann, E. (1974). Frame analysis. An essay on the organization of experience. Harmondsworth: Peregrine Books.

Golan, G., & Wanta, W. (2001). Second-level agenda-setting in the New Hampshire primary: a comparison of coverage in three newspapers and public perceptions of candidates. Journalism & mass communication quarterly, 78, 247-259. 223

Gonzenbach, W. (1992). A time-series analysis of the drug issue, 1985-1990: The press, the President and public opinion. International Journal of Public Opinion Research. 4. 126-147.

Gonzenbach, W. J., & McGavin, L. (1997). A brief history of time: A methodological analysis of agenda setting. In M. E. McCombs, D. L. Shaw, & D. Weaver (Eds.), Communication and democracy: Exploring the intellectual frontiers in agenda-setting theory (pp. 115-136). Mahwah, NJ: Erlbaum.

Gosta, C., Dahlberg, A., & Rosengren, K. (1981). Mass media content, political opinions, and social change: Sweden, 1967-1974. In K. Rosengren (Ed.), Advances in Content Analysis. Ed. Karl Erik Rosengren (pp. 227-240). Sage Annual Reviews of Communication Research Beverly Hills, CA: Sage.

Graber, D. A. (1984). Processing the news: how people tame the information tide. New York: Longman.

Graber, D. A. (2007). Media power in politics. 5th ed. Washington DC: CQ Press.

Gross, K., & Aday, S. (2003). The scary world in your living room and neighbourhood: Using local broadcast news, neighborhood crime rates, and personal experience to test agenda setting and cultivation. Journal of Communication, 53,411-426.

Gurevitch, M., & Blumler, J. G. (1990). Political communication systems and democratic values. In J. Lichenberg, (Ed.), Democracy and the Mass Media (pp. 269-289). Cambridge: Cambridge University Press, 1990.

Hardy, B., & Jamieson, K. (2005). Can a poll affect perception of candidate traits? Public Opinion Quarterly. 69.725-743.

Heldman, C., Carroll, S. J. & Olson, S. (2000). Gender differences in print media coverage of Presidential candidates: Elizabeth Dole's bid for the Republican nomination. Prepared for delivery at the Annual Meeting of the American Political Science Association, Washington, D.C., August 31 - September 3, 2000.

Heeter, C„ Brown, N., Soffin, S., Stanley, C.,, & Salwen, M. (1989). Agenda-setting by electronic text news. Journalism Quarterly, 66,101-106.

Hester, J. B., & Gibson, R. (2003). The economy and second-level agenda setting: a time- series ganalysis of economic news and public opinion about the economy. Journalism Quarterly, 80, 73-90.

Hill, D. (1985). Viewer characteristics and agenda-setting by television views. Public Opinion Quarterly. 49. 340-350. 224

Hirsch, P. (1980). The 'scary world' of the nonviewer and other anomalies: a reanalysis of Gerbner et al.'s finding on cultivation analysis. Communication Research. 7, 403-456.

Huang, L. N. (2000). Examining candidate information search processes. Journal of Communication, 50, 93-114.

Huegel, R., Degenhardt, W., & Weiss, H. (1989). Structural equation models for the analysis of the agenda-setting process. European Journal of Communication, 4, 191-210.

Iyengar, S. (1979). Television news and issue salience: A Re-examination of the agenda- setting hypothesis American Politics Research, 7. 395-416.

Iyengar, S. (1987). Is Anyone Responsible? How Television Frames Political Issues. Chicago: University of Chicago Press.

Iyengar, S. (1988). New directions of agenda-setting research. In J. A. Anderson (Ed.), Communication yearbook (Vol. 11, pp. 595-602). Newbury Park, CA: Sage.

Iyengar, S., & Simon, A. (1993). News coverage of the Gulf crisis and public opinion. Communication Research, 20. 365-383.

Jacoby, W. (2000). Issue framing and public opinion on government spending. American Journal of Political Science. 44. 750-767.

Jasperson, A., Shah, D. V., Watts, M„ Faber, R. J., & Fan, D. (1998). Framing and the public agenda: Media effects on the importance of the federal budget deficit. Political Communication, 15.205-224.

Johnson, B., Scott-Sheldon, L„ Snyder, L, Noar, S„ & Huedo-Medina, T. (2008). Contemporary approaches to meta-analysis in communication research. In A. Hayes, M. Slater & L. Snyder (Eds.), The Sage sourcebook of advanced data analysis methods for communication research (pp.311-348). Los Angeles, Sage.

Johnson, T., Byrd, J., & Wanta, W. (1995). Exploring FDR's relationship with the press: A historical agenda-setting study. Political Communication, 12, 157-172.

Kahn, K. F. (1994). The distorted mirror: Press coverage of women candidates for statewide office. Journal of Politics, 56, 154-73. Kahn, K. F., Goldenberg, E. N. (1997). The Media: Obstacle or ally of feminists. In S. Iyengar & R. Reeves (Eds.), Do the Media Govern? Politicians, Voters and Reporters in America (pp. 156-65). Thousand Oaks, CA: Sage. 225

Katz, E, Blumler J., & Gurevitch. M. (1974). Utilization of Mass Communication by the Individual." In J. G. Blumler & E. Katz (Eds.), The Uses of Mass Communications: Current Perspectives on Gratifications Research (pp. 19-32). Beverly Hills, CA: Sage.

Kepplinger, H., Donsbach, W„ Brosius, H. B., & Staab, J. F. (1989). Media tone and public opinion: a longitudinal study of media coverage and public opinion on Chancellor Kohl. International Journal of Public Opinion Research, 1, 326-42.

Kim, J. K., Shoar-Ghaffari, P., & Gustainis J., (1990). Agenda-setting functions of a media event: The case of 'Amerika'. Political Communication. 7, 1-11.

Kim, S., Scheufele, D., & Shanahan, J. (2002). Think about it this way: Attribute agenda- setting function of the press and the public's evaluation of a local issue. Journalism and Mass Communication Quarterly. 79,7-25.

King, P. (1997) The press, candidate images, and voter perceptions. In M. E. McCombs, D. L. Shaw & D. Weaver (Eds.), Communication and democracy: Exploring the intellectual frontiers in agenda-setting theory (pp. 29-40). Mahwah, NJ: Erlbaum.

Kiousis, S. (2000). Beyond salience: Exploring the linkages between the agenda setting role of the mass media and mass persuasion. Unpublished doctoral dissertation, Unviersity of Texas, Austin.

Kiousis, S. (2004). Explicating media salience: a factor analysis of New York Times issue coverage during the 2000 presidential election. Journal of Communication, 54. 71-87.

Kiousis, S., & McCombs, M. (2004). Agenda-setting effects and attitude strength: Political figures during the 1996 Presidential election. Communication Research. 31. 36- 57.

Kiousis, S., Bantimaroudis, P., & Ban, H. (1999). Candidate image attributes: Experiments on the substantive dimension of second-level agenda setting. Communication Research. 26.414-428.

Klapper, J. (1960). The effects of mass communication. New York: Free Press.

Kleinnijenhuis, J., Rietberg, E. (1995). Parties, media, the public, and the economy: Patterns of societal agenda-setting. European Journal of Political Research, 28, 95-118.

Kosicki, G. M. (1993). Problems and opportunities in agenda-setting research. Journal of Communication, 43, 100-127. 226

Rrippendorff, K. (1995). On the reliability of unitizing continuous data. Sociological Methodology. 25.47-76.

Krippendorff, K. (2003). Content analysis: An introduction to its methodology (2nd ed.). Thousand Oaks, CA: Sage.

Krosnick, J., & Kinder, D. R. (1990). Altering the foundations of support for the present through priming. American Political Science Review. 84,497-512.

Kuhn, T. (1970). The structure of scientific revolutions (2nd ed.). Chicago: University of Chicago Press.

Lang, G. E., & Lang, K. (1983). The battle for public opinion: The President, the press, and the polls during Watergate. New York: Columbia University Press.

Lang, K., & Lang, G. E. (1983). The 'New' rhetoric of mass communication: A longer view. Journal of Communication. 33,128-140.

Lang, K., & Lang, G. E. (1959). The mass media and voting. In E. Burdick & A. J. Brodbeck (Eds.), American Voter Behavior (pp. 215-235). Glencoe, II: Free Press.

Lasorsa, D., & Wanta, W. (1990). Effects of personal, interpersonal and media experiences on issue saliences. Journalism Quarterly, 67. 804-813.

Lazarsfeld, P., Berelson, B., & Gaudet, H. (1968). The people's choice: how the voter makes up his mind in a Presidential campaign. New York: Columbia University Press.

Lee, G. (2005). Agenda-setting effects in the digital age: uses and effects of online media. Unpublished doctoral dissertation, University of Texas, Austin.

Leff, D., Protess, D., & Brooks, S. (2001). Crusading journalism: Changing public attitudes and policy-making agendas. Public Opinion Quarterly. 50, 300-315.

Lippman, W. (1997). Public Opinion (Rev. ed.)._New York: Free Press Paperbacks.

Lopez-Escobar, E., Lennon, F., & Llamas, J. (1998). Two levels of agenda setting among advertising and news in the 1995 Spanish elections. Political Communication. 15,225-238.

Lopez-Escobar, E., Llamas, J. P., & McCombs, M. E. (1998). Agenda setting and community consensus: First and second level effects. International Journal of Public Opinion Research. 10. 335-348. 227

Lowry, D., Nio, T., & Leitner, D. (2003). Setting the public fear agenda: A longitudinal analysis of network TV crime reporting, public perceptions of crime, and FBI crime statistics. Journal of Communication, 53,61-73.

MacKuen, M. B. (1981). Social communication and the mass policy agenda. In M. B. MacKuen & S. L. Coombs (eds.), More than news: Media power in public affairs (pp. 19-144). Beverly Hills, CA: Sage.

MacKuen, M., & Coombs, S. (1981). More than news. Beverly Hills: Sage.

Maher, M. T. (1996). Media framing and public perception of environmental causality. Southwestern Mass Communication Journal, 12, 61-73.

Maher, M. T. (2001). Framing: An emerging paradigm or a phase of agenda setting? In S. D. Reese, O. H. Gandy, Jr., & A. E. Grant (Eds.), Framing Public Life (pp. 83- 94). Mahwah, NJ: Erlbaum.

Maher, T. M. (2001). Framing: An emerging paradigm or a phase of agenda setting? In S. Reese, O. Gandy Jr., & A. Grant (Eds.), Framing Public Life (pp. 83-94). Mahwah, NJ: Erlbaum.

Manheim, J. B. (1986). A model of agenda dynamics. In M. L. McLaughlin (Ed.), Communication yearbook (Vol. 10), pp. 499-516), Newbury Park, CA: Sage.

Manheim, J. B., & Albritton, R. B. (1984). Changing national images: International public relations and media agenda setting. American Political Science Review, 78, 641-657.

Massing, M. (1984). The network newscasts: Still hot off the presses. Channels of Communications, 3. 47-52.

Matthes, Jorg. (2005). The need for orientation towards news media: revising and validating a classic concept. International Journal of Public Opinion Research, 18,422-444.

Mazur, A. (1987). Putting Radon on the public risk agenda. Science, Technology and Human Values, 12, 86-93.

McCombs, M. E. (1981a). Setting the agenda for agenda-setting research: an assessment of the priority ideas and problems. In G. C. Wilhoit & H. de Bock (Eds.), Mass Communication Review Yearbook 2, (pp. 209-211). Beverly Hills: Sage.

McCombs, M. E. (1981b). The agenda-setting approach. In D. Nimmo & K. Sanders (Eds.), Handbook of political communication (pp. 121-140). Beverly Hills, CA: Sage. 228

McCombs, M. E. (1977). Newspaper versus television mass communication effects across time. In D. L. Shaw & M. E. McCombs (Eds.), The emergence of American political issues: The agenda-setting function of the press (XX-XX). St. Paul: West Publishing.

McCombs, M. E. (1992). Explorers and surveyors: expanding strategies for agenda setting research. Journalism Quarterly, 69, 813-824.

McCombs, M. E. (1993) The evolution of agenda-setting research: Twenty-five years in the marketplace of ideas. Journal of Communication, 43, 58-67.

McCombs, M. E. (1997). Building consensus: The news media's agenda-setting roles. Political Communication. 14,433-443.

McCombs, M. E. (2004). Setting the agenda: The mass media and public opinion. Maiden, MA: Blackwell.

McCombs, M. E., & Ghanem, S. (2001). The convergence of agenda-setting and framing. In S. Reese, O. Gandy Jr., & A. Grant (Eds.), Framing Public Life (pp. 67-81). Mahwah, NJ: Erlbaum.

McCombs, M. E., & Reynolds, A. (2002). News influence on our pictures of the world. In J. Bryant & D. Zillman (Eds.), Media effects: Advances in theory and research (2nd Ed., pp. 1-18). Mahwah NJ: Erlbaum.

McCombs, M. E., & Shaw, D. (1974). A progress report on agenda-setting research. Paper presented at the Annual Meeting of the Association for Education in Journalism (57th, San Diego, California, August 18-21,1974)

McCombs, M. E., & Shaw, D. (1976). Structuring the unseen environment. Journal of Communication, 26.18-22.

McCombs, M. E., & Shaw, D. (1980). An up-to-date report on the agenda-setting function. Paper presented to the International Communication Association, Acapulco, Mexico.

McCombs, M. E., & Shaw, D. L. (1972). The agenda-setting function of the mass media. Public Opinion Quarterly. 36.176-185.

McCombs, M. E., & Weaver, D. (1985). Toward a merger of gratifications and agenda- setting research. In K. Rosengren, L. Wenner & P. Palmgreen (Eds.), Media gratifications research (pp. 95-108). Beverly Hills, CA: Sage.

McCombs, M. E., & Zhu, J.-H. (1995). Capacity, diversity and volatility of the public agenda. Public Opinion Quarterly. 59,495-525. 229

McCombs, M. E., Danielian, L., & Wanta, W. (1995). Issues in the news and the public agenda: The agenda-setting tradition. In T. L. Glasser & C. T. Salmon (Eds.), Public opinion and the communication of consent (pp. 281-300). New York: Guilford.

McCombs, M. E., Einsidel, E., & Weaver, D. (1991). Contemporary public opinion: Issues and the News. Hillsdale, NJ: Erlbaum.

McCombs, M. E., Llamas, J. P., Lopez-Escobar, E., & Rey, F. (1997). Candidate images in Spanish elections: Second-level agenda-setting effects. Journalism & Mass Communication Quarterly, 74,703-717.

McCombs, M. E., Lopez-Escobar, E., & Llamas, J. P. (2000). Setting the agenda of attributes in the 1996 Spanish general election. Journal of Communication, 50, 77-92.

McCombs, M. E., Shaw, D., & Weaver, D. (Eds.). (1997). Communication and democracy: Exploring the intellectual frontiers in agenda-setting theory. Mahwah, NJ: Erlbaum.

McGregor, J., Fountain, S., & Comry, M. (2000). From contest to content: The impact of public journalism on New Zealand election campaign coverage. Political Communication, 17,133-148.

McLeod D. M., & Detenber, B. H. (1999). Framing effects of television news coverage of social protest. Journal of Communication, 49. 3-23.

McLeod, D. M„ Kosicki, G. M., & McLeod, J. M. (2002). Resurveying the boundaries of political communications effects. In J. Bryant & D. Zillman (Eds.), Media effects: Advances in theory and research (2nd ed., pp. 215-267). Mahwah NJ: Erlbaum.

McLeod, J. M., Kosicki, G.M., & Pan, Z. On understanding and misunderstanding media effects. In J. Curran & M. Gurevitch (Eds.), Mass Media and Society (pp. 235- 66). London: Edward Arnold, 1991.

McLeod, J. M., & Reeves, B. (1980). On the nature of mass media effects. In S. B. Withey & R. P. Abeles (Eds.), Television and social behavior: Beyond violence and children (pp. 17-54). Hillsdale, NJ: Erlbaum.

McLeod, J. M., Becker, L., & Byrnes, J. E. (1974). Another look at the agenda-setting function of the press. Communication Research, 1.131-166.

McQuail, D. (2000). McQuail's Mass Communication Theory 4th ed.Jhousand Oaks, CA: Sage. 230

Megwa, E., & Brenner, D. (1988). Toward a paradigm of media agenda-setting effect: Agenda-setting as a process. Howard Journal of Communications, 1, 39-56.

Meijer, M-M., & Kleinnijenhuis, J. (2006). Issue news and corporate reputation: Applying the theories of agenda setting and issue ownership in the field of business communication. Journal of Communication, 56, 543-559.

Meyer, P. (1989). Defining and measuring credibility of newspapers: Developing an index. Journalism Quarterly, 66. 567-74, 588.

Meyers, R. (1992) Public issues, agenda-setting, and argument: A theoretical perspective. In S. A. Deetz (Ed.), Communication yearbook (Vol. 15, pp. 397-409). Newbury Park, CA: Sage.

Miller, J. M., & Krosnick, J. (1997). "Anatomy of News Media Priming." In S. Iyengar & R. Reeves (Eds.), Do the media govern? Politicians, voters and reporters in America (pp. 258-75). Thousand Oaks, CA: Sage.

Miller, R. E., & Wanta, W. (1996). Race as a variable in agenda setting. Journalism and Mass Communication Quarterly, 73, 913-925.

Miller, R. E., & Wanta, W. (1996). Sources of the public agenda: The President-press- public relationship. International Journal of Public Opinion Research, 8, 390- 402.

Min, Y., Ghanem, S., & Evatt, D. (2007). Using a split-ballot survey to explore the robustness of the 'MIP' question in agenda-setting research: A methodological study. International Journal of Public Opinion Research. 19, 221-236.

NADBank Inc. (2005) Why Newspapers? Toronto, ON.

Neuman, W. R. (1990). The threshold of public attention. Public Opinion Quarterly, 54, 159-176.

Neuwirth, K., Frederick, E., & Mayo, C. (2002). Person-effects and heuristic-systematic processing. Communication Research, 29. 320-359.

Noelle-Neumann, E., & Mathes, R. (1987). The "event as event" and the "event as news": The significance of "consonance" for media effects research. European Journal of Communication, 2. 391-414.

Page, B., Shapiro, R., & Dempsey, G. (1987). What moves public opinion? American Political Science Review. 81. 123-138.

Palmgreen, P. (1984). Uses and gratifications: A theoretical perspective. Communication Yearbook. 8. 20-55. 231

Pan, Z., & Kosicki, G. M. (1993). Framing analysis: An approach to news discourse. Political Communication, 10, 55-75.

Pan, Z., & Kosicki, G. M. (1997). Priming and media impact on the evaluations of the President's performance. Communication Research. 24, 3-30.

Patterson, T„ & McClure, R. (1976). The Unseeing Eve: The Myth of Television Power in National Elections. New York: G. P. Putnam & Sons.

Peter, J. (2003). Country characteristics as contingent conditions of agenda setting: The moderating influence of polarized elite opinion. Communication Research. 30. 683-712.

Peter, J., & de Vreese, C. (2003). Agenda-rich, agenda-poor: A cross-national comparative investigation of nominal and thematic public agenda diversity. International Journal of Public Opinion Research. 15,44-64.

Peterson, S. (1981). International news selection by the elite press: A case study. Public Opinion Quarterly. 45.143-163.

Petty, R. E., & Cacioppo, J. T. (1986). Communication and persuasion: Central and peripheral routes to attitude change. New York: Springer-Verlag.

Ploughman, P. (1984). The creation of newsworthy events: An analysis of newspaper coverage of the man-made disaster at Love Canal. Unpublished doctoral dissertation, State University of New York at Buffalo, New York.

Price, V., & Tewksbury, D. (1997). News values and public opinion: A theoretical account of media priming and framing. In G. A. Barnett & F. J. Baoster (Eds.), Progress in communication sciences: Advances in persuasion (vol. 13, pp. 173-212). Greenwich, CT: Ablex.

Pritchard, D. (1986). Homicide and bargained justice: The agenda-setting effect of crime news on prosecutors. Public Opinion Quarterly, 50,143-159.

Pritchard, D., & Berkowitz, D. (1993). The limits of agenda-setting: The press and political responses to crime in the United States, 1950-1980. International Journal of Public Opinion Research, 5, 86-91.

Protess, D. L., Leff, D. R., Brooks, S. C., & Gordon, M. T. (1985). Uncovering rape: The watchdog press and the limits of agenda-setting. Public Opinion Quarterly. 49. 19-37.

Reese, S. D., & Danielian, L. (1989). Intermedia influence and the drug issue: Conferring on cocaine. In P. Shoemaker (Ed.), Communication Campaigns About Drugs (pp. 29-46). Hillsdale NJ: Erlbaum. 232

Rice, R. E., & Atkin, C. K. (2000). Public communication campaigns (3rd. ed.). Thousand Oaks CA: Sage.

Rimmer, T., & Weaver D. (1987). Different questions, different answers? Media use and media credibility. Journalism Quarterly, 64. 28-36,44.

Roberts, D., & Bachen, M. (1981). Mass communication effects. Annual Review of Psychology, 32. 307-356.

Roberts, M. (1992). Predicting voting behavior via the agenda-setting tradition. Journalism Quarterly. 69. 878-892.

Roberts, M., & McCombs, M. E. (1994). Agenda-setting and political advertising: Origins of the news agenda. Political Communication. 11. 249-262.

Roberts, M., Wanta, W., & Dzwo, T-H. (2002). Agenda setting and issue salience online. Communication Research, 29,452-465.

Robinson, J., & Levy, M. (1986). The main source: Learning from television news. Beverly Hills, CA: Sage.

Roessler, P. (1999). The individual agenda-designing process: How interpersonal communication, egocentric networks, and mass media shape the perception of political issues by individuals. Communication Research, 26. (1999): 666-700.

Roessler, P., & Schenk, M. (2000). Cognitive bonding and the German reunification: Agenda-setting and persuasion effects of mass media. International Journal of Public Opinion Research. 12. 29-47.

Rogers, E. M., Dearing, J. W., & Bregman, D. (1993). The anatomy of agenda-setting research. Journal of Communication, 43. 68-84.

Rogers, E. M., Hart, W. B., & Dearing, J. W. (1997). A paradigmatic history of agenda- setting research. In S. Iyengar & R. Reeves (Eds.), Do the media govern? Politicians, voters and reporters in America (pp. 225-236). Thousand Oaks, CA: Sage.

Salwen, M. B. (1988). Effect of accumulation of coverage on issue salience in agenda setting. Journalism Quarterly. 65. 100-106, 130.

Salwen, M. B. (1995). News of Hurricane Andrew: The agenda of sources and the sources' agenda. Journalism Quarterly. 72. 826-840.

Salwen, M. B., & Matera, F. R. (1992). Public salience of foreign nations. Journalism Quarterly. 69. 623-632. 233

Scheafer, T. (2007). How to evaluate it: The role of story-evaluative tone in agenda setting and priming. Journal of Communication, 57, 21-39.

Scheberle, D. (1994). Radon and asbestos: A study of agenda-setting and causal stories. Policy Studies Journal, 22, 74-86.

Scheufele, D. (1999). Framing as a theory of media effects. Journal of Communication, 49, 103-22.

Scheufele, D. (2000). Agenda-setting, priming, and framing revisited: Another look at cognitive effects of political communication. Mass Communication and Society, 3,297-316.

Schmitt, J. (1995). NAFTA, the press and public opinion: The effect of increased media coverage on the formation of public opinion. International Journal of Public Opinion Research. 7. 178-184.

Schoenbach, K., & Semetko, H. A. (1992). Agenda-setting, agenda-reinforcing or agenda- deflating? A study of the 1990 German national election. Journalism Quarterly, 69,837-846.

Semetko, H., & Canel M. (1997). Agenda-senders versus Agenda-setters: Television in Spain's 1996 election campaign. Political Communication, 14, 459-479.

Shaw, D. L., & Hamm, B. J. (1997). Agendas for a public union or for private consumption. In M. E. McCombs, D. L. Shaw & D. Weaver (Eds.), Communication and democracy: Exploring the intellectual frontiers in agenda- setting theory (pp. 209-30). Mahwah, NJ: Erlbaum.

Shaw, D. L., & Martin, S. (1992). The function of mass media agenda setting. Journalism Quarterly, 69. 902-920.

Shaw, D. L„ & McCombs, M. E. (Eds.). (1977). The Emergence of American Political Issues. St. Paul, MN: West.

Shaw, D. L., McCombs, M. E., Weaver, D. H., & Hamm, B. (1999). Individuals, groups, and agenda melding: A theory of social dissonance. International Journal of Public Opinion Research, 11, 2-24.

Sheafer, T., & Weimann, G. (2005). Agenda building, agenda setting priming individual voting intentions, and the aggregate results: An analysis of four Israeli elections. Journal of Communication. 55.347-465. 234

Shoemaker, P. J. (Ed.). (1989). Communication campaigns about drugs: Government, media and the public. Hillsdale, NJ: Erlbaum.

Shoemaker, P. J., & Reese, S. D. (1990). Exposure to what? Integrating media content and effects studies. Journalism Quarterly. 67.649-652.

Shoemaker, P. J., & Reese, S. D. (1996). Mediating the message: Theories of influence on mass media content (2nd ed.). White Plains, NY: Longman.

Shoemaker, P. J., Wanta, W., & Leggett,. D. (1990) Drug coverage and public opinion, 1972-1986 In Shoemaker, P. J. (Ed.), Communication campaigns about drugs: Government, media and the public (pp. 67-80). Hillsdale, NJ: Erlbaum.

Smith, K. A. (1987). Newspaper coverage and public concern about community issues. Journalism Monographs, 101,1-32.

Smith, T. W. (1980). America's Most Important Problem: A trend analysis, 1946-1976. Public Opinion Quarterly. 44,164-180.

Sohn, A. B. (1984). Newspaper agenda-setting and community expectations. Journalism Quarterly. 61. 892-897.

Soroka, S. (2000b). Schindler's List's intermedia influence: Exploring the role of "entertainment" in media agenda-setting. Canadian Journal of Communication. 25,211-230.

Soroka, S. (2000). Expanding the agenda-setting agenda: Issue dynamics in Canada, 1985- 1995. Unpublished doctoral dissertation, University of British Columbia, British Columbia.

Soroka, S. (2001). Media, public opinion, and foreign policy. Paper presented to the American Political Science Association, San Francisco, CA.

Soroka, S. (2002). Agenda-setting dynamics in Canada. Vancouver: UBC Press.

Soroka, S. (2002). Issue attributes and agenda-setting by media, the public and policymakers in Canada. International Journal of Public Opinion Research, 14, 264-285.

Stempel, G. H., III. (1989). Content Analysis. In G. Stempel & B. Westley (Eds.), Research methods in mass communication (pp. 119-131). Upper Saddle River NJ: Prentice-Hall.

Stone, G. C. (1987). Examining newspapers: What research reveals about American newspapers. Beverly Hills, CA: Sage. 235

Stone, G., & McCombs, M. E. (1981). Tracing the time lag in agenda-setting. Journalism Quarterly. 58, 51-55.

Swanson, D. L. (1988) Feeling the elephant: Some observations on agenda-setting research. In J. A. Anderson (Ed.), Communication yearbook (Vol. 11, pp. 603- 619). Newbury Park, CA: Sage.

Takeshita, T. (1993). Agenda-setting effects of the press in a Japanese local election. Studies of Broadcasting, 29,193-216.

Takeshita, T. (2005). Current critical problems in agenda-setting research. International Journal of Public Opinion Research, 18, 275-296.

Tankard Jr., J. W. (2001). The empirical approach to the study of media framing. In S. Reese, O. Gandy Jr., & A. Grant (Eds.), Framing Public Life (pp. 95-106). Mahwah, NJ: Erlbaum.

Tipton, L. P., Haney, R. D., & Basehart, J. R. (1975). Media agenda-setting in city and state campaigns. Journalism Quarterly, 52, 15-22.

Tolman, E. C., (1948). Cognitive maps in rats and men. Psychological Review, 55, 189- 208.

Tuchman, G. (1978). Making news: A study in the construction of reality. New York: Free Press.

Trenaman, J., & McQuail, D. (1961) Television and the political image. London: Methuen.

Trimble, L & Arscot, J. (2003). Still counting: Women in politics across Canada. Peterborough, ON: Broadview Press.

Trumbo, C. (1995). Longitudinal modeling of public issues: an application of the agenda- setting process to the issue of global warming. Journalism Monographs, 152.

Wagner, J. (1983). Media do make a difference: The differential impact of mass media in the 1976 presidential race. American Journal of Political Science, 27,407-430.

Walgrave, S., & Van Aelst, P. (2006). The contingency of the mass media's political agenda setting power: Toward a preliminary theory. Journal of Communication, 56,88-109.

Wang, T. (2000). Agenda-setting online: an experiment testing the effects of hyperlinks in online newspapers. Southwestern Mass Communication Journal, 15, 59-70.

Wanta, W. (1988). The effects of dominant photographs: An agenda setting experiment. Journalism Quarterly. 65. 107-111 236

Wanta, W. (1997). The public and the national agenda: How people learn about important issues. Mahwah, NJ: Erlbaum.

Wanta, W., & Foote, J. (1994). "The President news media relationship: A time series analysis of agenda-setting. Journal of Broadcasting and Electronic Media, 38, 437.

Wanta, W„ & Ghanem, S. (2006). Effects of Agenda-Setting. In J. Bryant & R. Carveth (Eds.), Meta-analyses of media effects pp.. Mahwah, NJ: Erlbaum.

Wanta, W., & Roy, M. J. (1995, August). Memory decay and the agenda-setting effect: An examination of three news media. Paper presented at the Association for Education in Journalism and Mass Communication, Washington, DC.

Wanta, W., & Wu, Y-H. (1992). Interpersonal communication and the agenda-setting process. Journalism Quarterly. 69. 847-855.

Wanta, W., & Wu, Y-H. (1992). The effects of credibility, reliance, and exposure on media agenda-setting: A path analysis model. Journalism Quarterly, 71,90-98.

Wanta, W., & Wu, Y-H. (1993). The agenda-setting effects of international news coverage: An examination of differing news frames. International Journal of Public Opinion Research. 5, 250-264.

Wanta, W., & Wu, Y-H. (1994). Time lag differences in the agenda-setting process: An examination of five news media. International Journal of Public Opinion Research. 6. 225-240.

Wanta, W., Golan, G., & Lee, C. (2004). Agenda setting and international news: Media influence on public perceptions of foreign nations. Journalism Quarterly. 81, 364-377.

Wanta, W., King, P., & McCombs, M. E. (1995). A comparison of factors influencing issue diversity in the U.S. and Taiwan. International Journal of Public Opinion Research, 7, 353-365.

Wanta, W„ Stephenson, J., Turk, J., & McCombs, M. E. (1989). How the President's state of the union address influenced news media agendas. Journalism Quarterly, 66. 537-541.

Watt, J. H., Mazza, M., & Snyder, L. (1993). Agenda setting effects of television news coverage and the effects decay curve. Communication Research. 20,404-435.

Weaver, D. (1977). Audience need for orientation and media effects. Communication Research. 8. 361-376. 237

Weaver, D. (1984). Media agenda-setting and public opinion: Is there a link? In R. Bostrom & B. Westely (Eds.), Communication yearbook, 8, (pp. 680-691). Beverly Hills, CA: Sage.

Weaver, D. (1991). Issue salience and public opinion: Are there consequences for agenda- setting? International Journal of Public Opinion Research, 3.53-68.

Weaver, D. (1991). Political issues and voter need for orientation. In D. Protess & M. McCombs (Eds.), Agenda-setting: readings on media, public opinion and policy-making (pp. 131-139). Hillsdale, NJ: Erlbaum.

Weaver, D. (1994). Media agenda setting and elections: Voter involvement or alienation? Political Communication, 11, 347-356.

Weaver, D. (2007). Thoughts on agenda setting, framing and priming. Journal of Communication, 57, 142-147.

Weaver, D„ Graber, D., McCombs, M, & Eyal, C. (1981). Media agenda-setting in a presidential election: Issues, images, and interests. New York: Praeger.

Weimann, G. (1994). The influentials: People who influence people. Albany, NY: SUNY Press.

Weimann, G., & Brosius, H.-B. (1994). Is there a two-step flow of agenda-setting? International Journal of Public Opinion Research, 6, 323-341.

Weiss, H-J. (1992). Public issues and augmentation structures: An approach to the study of the contents of media agenda-setting. In S. A. Deetz (Ed.), Communication yearbook (Vol. 15, pp. 374-396). Newbury Park, CA: Sage.

Westley, B. H. (1976). What makes it change? Journal of Communications, 26,43-47.

Williams Jr., W., Shapiro, M., & Cutbirth, C. (1983). The impact of campaign agendas on perceptions of issues in the 1980 campaign. Journalism Quarterly, 60,226-231.

Willnat, L. (1997). Agenda setting and priming: conceptual links and differences. In M. E. McCombs, D. Shaw & D. Weaver (Eds.). Communication and Democracy (pp. 51-66). Mahwah, NJ: Erlbaum.

Winter, J. P., & Eyal, C. H. (1981). Agenda-setting for the civil rights issue. Public Opinion Quarterly, 45, 376-383.

Winter, J. P., Eyal, C., & Rogers, A. (1982). Issue-specific agenda setting: the whole as less than the sum of the parts. Canadian Journal of Communication 8,1-10. 238

Wood, B., & Peake, J. (1998). The dynamics of foreign policy agenda setting. American Political Science Review. 92.173-184.

Wooley, J. T. (2000). Using media-based data in studies of politics. American Journal of Political Science. 44.156-173.

Yagade, A., & Dozier, D. M. (1990). The media agenda-setting effect of concrete versus abstract issues. Journalism Quarterly. 67. 3-10.

Yioutas, J., & Segvic, I. (2003). Revising the Clinton/Lewinsky scandal: the convergence of agenda setting and framing. Journalism and Mass Communication Quarterly, 80,567-582.

Young, J., & Weaver, D. (2005). Another look at what moves public opinion: Media agenda setting and polls in the 2000 U.S. election. International Journal of Public Opinion Research. 18. 174-197.

Zaller, J. R. (1997). A Model of Communication Effects at the Outbreak of the Gulf War. In S. Iyengar & R. Reeves (Eds.). Do the media govern? Politicians, voters and reporters in America (pp. 296-311). Thousand Oaks, CA: Sage.

Zamaria, C., & Fletcher, F. (2008) Canada online! The internet, media and emerging technologies: Uses, attitudes, trends and international comparisons 2007. Toronto: Canadian Internet Project 2008.

Zhu, J.-H. (1992). Issue competition and attention distraction: A zero-sum theory of agenda setting. Journalism Quarterly. 69. 825-836.

Zhu, J.-H., & Boroson, W. (1997). Susceptibility to agenda setting: a cross-sectional and longitudinal analysis of individual differences. In M. McCombs, D. Shaw and D. Weaver (Eds.), Communication and democracy: Exploring the intellectual frontiers in agenda-setting theory (pp. 69-84). Mahwah, NJ: Erlbaum.

Zhu, J.-H., Watt, J., Snyder, L., Yan, J., & Jiang, Y. (1993). Public issue priority formation : Media agenda-setting and social interaction. Journal of Communication. 43, 8-29.

Zucker, H. G. (1978). The variable nature of news media influence. In B. D. Ruben (Ed.), Communication Yearbook (Vol. 2, pp. 225-245). New Brunswick, NJ: Transaction Books.