A CRITICAL REVIEW OF AGENDA-SETTING METHODOLOGY AND THE EVALUATION OF A MEDIA-CENTRIC MODEL ANDREW LAING A DISSERTATION SUBMITTED TO THE FACULTY OF GRADUATE STUDIES IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY GRADUATE PROGRAM IN COMMUNICATION AND CULTURE YORK UNIVERSITY, TORONTO, ONTARIO SEPTEMBER 2009 Library and Archives Bibliothgque et 1*1 Canada Archives Canada Published Heritage Direction du Branch Patrimoine de I'gdition 395 Wellington Street 395, rue Wellington Ottawa ON K1A0N4 Ottawa ON K1A 0N4 Canada Canada Your file Votre r6f6rence ISBN: 978-0-494-64956-5 Our file Notre reference ISBN: 978-0-494-64956-5 NOTICE: AVIS: The author has granted a non- L'auteur a accorde une licence non exclusive exclusive license allowing Library and permettant a la Bibliotheque et Archives Archives Canada to reproduce, Canada de reproduire, publier, archiver, publish, archive, preserve, conserve, sauvegarder, conserver, transmettre au public communicate to the public by par telecommunication ou par Nnternet, preter, telecommunication or on the Internet, distribuer et vendre des theses partout dans le loan, distribute and sell theses monde, a des fins commerciales ou autres, sur worldwide, for commercial or non- support microforme, papier, electronique et/ou commercial purposes, in microform, autres formats. paper, electronic and/or any other formats. The author retains copyright L'auteur conserve la propriete du droit d'auteur ownership and moral rights in this et des droits moraux qui protege cette these. Ni thesis. Neither the thesis nor la these ni des extraits substantiels de celle-ci substantial extracts from it may be ne doivent etre imprimes ou autrement printed or otherwise reproduced reproduits sans son autorisation. without the author's permission. In compliance with the Canadian Conformement a la loi canadienne sur la Privacy Act some supporting forms protection de la vie privee, quelques may have been removed from this formulaires secondaires ont ete enleves de thesis. cette these. While these forms may be included Bien que ces formulaires aient inclus dans in the document page count, their la pagination, il n'y aura aucun contenu removal does not represent any loss manquant. of content from the thesis. M Canada iv ABSTRACT Agenda-setting ranks among the most studied phenomena in communications research. A factor behind its popularity has been the nature of its research design that incorporates both media content and public opinion in a manner that yields an observable and empirically testable effect. Despite the model's importance, there has been little systematic evaluation of the underlying research design and, in particular, the role of media within the media/transfer of salience/public opinion equation. How media should be conceptualized within the research design has growing importance to the broader study of media effects due to the divergence of media sources and the erosion of mainstream media influence. The thesis addresses these issues by first conducting a meta-analysis that focuses on the research designs in agenda-setting research published in the last 25 years in major journals. The meta-analysis supports the hypothesis that media is often underconceptualized in research designs, and offers a new approach—the media-centric model—that addresses key weaknesses concerning the proximity of media to audience/respondents and other factor that undermine research design validity. The media- centric model was tested over a five-month period in the Waterloo region of Ontario. The model includes formulas for weighting of media content based on market research and other audience demographic data. The study confirmed the presence of an agenda-setting effect, and also confirmed the importance of proximity of news content and, in particular, the importance V of local news media coverage that is often overlooked in agenda-setting research. The presence of the agenda-setting effect was not significantly strengthened by the application of audience demographic data, but other factors from the model that enhanced proximity of content did indicate significant results, but were qualified by the nature of key news events. The results provide an important contribution to conceptualizing the role of media not only in agenda-setting research designs but also in other areas of study into primary media effects within the context of an expanding universe of mediated sources of news and information. vi TABLE OF CONTENTS Abstract iv List of Figures viii List of Tables ix Acknowledgements xi Introduction 1 Chapter One: Reviewing the development of the media agenda setting research model 11 A. Defining agenda setting 11 B. Historical development of the model 14 C. Key elements of the model 25 1. Media agenda 28 2. Transfer of salience 41 3. Public agenda 47 4. Time 52 D. Validity in agenda-setting research design 55 1. External validity and media sampling 56 2. Internal validity and the transfer of salience 59 Discussion 62 Chapter Two: Meta-analysis of agenda-setting research design 65 A. Outlining the meta-analysis of research designs used in agenda- setting 65 B. The consideration of time: Cross-sectional versus longitudinally designed studies. 70 C. Media sample validity and proximity 71 D. Operationalizing the transfer of salience 76 E. Controlling for contingent conditions 79 F. Evaluating media sampling, transfer of media salience and contingent factors within a single index 83 G. Defining and constructing the media-centric agenda-setting research model 91 vii Discussion 95 Chapter Three: Implementing the media-centric model 100 A. Site selection 100 B. Media sample selection 102 C. Content analysis design and data handling 113 D. Weighting media results 114 E. Sampling public opinion 117 F. Media consumption and demographics 119 Discussion 120 Chapter Four: Analyzing results from the media-centric model 123 A. Establishing testing criteria: media-centric model versus a non- media-centric model 123 B. Overview of media and public opinion data 126 C. Cross-sectional tests of the media-centric model 133 D. Longitudinal tests of the media-centric model 144 1. Economy 147 2. Health care 152 3. Environment 154 4. War/conflict 157 5. Consumer prices 159 Discussion 162 Conclusion 168 Appendix A: Meta-analysis codebook 180 Appendix B: Media-centric model: media analysis codebook 187 Appendix C: Coding reliability tests 194 Appendix D: Media-centric model: public opinion questionnaire 196 Appendix E: Media weighting formulae 209 Bibliography 218 Vlll LIST OF FIGURES Figure 1-1 Basic agenda setting model 28 Figure 1-2 How some researchers explain the transfer of salience process 59 Figure 1-3 Agenda-setting process defined by stages in developing the research design 62 Figure 2-1 Proximity rankings of media samples for agenda-setting research designs, by national versus local media samples 74 Figure 2-2 Distribution of agenda-setting research articles by methodological index 85 Figure 2-3 Average number of media outlets surveyed per agenda- setting study 89 Figure 4-1 Matrix of tests of the media-centric model 125 Figure 4-2 Medimediaa, coveragby weeke, oJanuarf economiy 1—Junc storiee 6s, b200y loca8 l and national, 149 Figure 4-3 Standardized scoring of economy as a top issue by local and national media over a ten-week period 150 Figure 4-4 Tracking local and national media coverage of health care issues by week: January 1—June 6, 2008 153 Figure 4-5 Tracking local and national media coverage of environmental issues by week: January 1—June 6, 2008 155 Figure 4-6 Local media coverage and public opinion—environment identified as a top local issue, by survey week 156 Figure 4-7 Tracking media coverage of war/conflict issues, by week: January 1—June 6, 2008. 158 Figure 4-8 Public opinion and media coverage concerning war/conflict issue: 56 day time lag 159 Figure 4-9 Standardized scoring of public opinion ranking of consumer issues, media coverage, and gasoline prices in Toronto, by week 162 ix LIST OF TABLES Table 2-1 Results of agenda-setting meta-analysis 67 Table 3-1 Newspaper readership by respondents per week 105 Table 3-2 Television news consumption by respondents, per week 108 Table 3-3 Internet site consumption by respondents, per week 112 Table 4-1 Top national issues by respondents 128 Table 4-2 Top local issues by respondents 128 Table 4-3a Top provincial and national issues by media coverage, by month 129 Table 4-3b Top local issues by media coverage, by month (N=4876) 129 Table 4-4 JanuarDistributioy 1—Junn of toe p7 ,topic 2008s by broadcast news channel: 135 Table 4-5 Rank-order correlation of top topics by broadcast news channels 135 Table 4-6 Breakdown of major topic by scope 137 Table 4-7 Rank-ordemedia and rorigi correlationn of conten: topt national and local issues by 139 Table 4-8 Rank order correlation of education and interpersonal discussion as control variables on local and national media coverage of issues 141 Table 4-9 Rank-order correlation by national issues—weighted versus non weighted media data 142 Table 4-10 Rank-order correlation of top issues by high/low news consumers, by media type 144 Table 4-11 Respondent's selection of top issues by week 145 Table 4-12 mediCorrelatioa n by top issue over time by media weights—all 147 Table 4-13 Correlation of economy as a top issue by time lag and media type 149 X Table 4-14 Correlation of the economy as a top issue by lag periods, media type and level of media usage by respondents 151 Table 4-15 Correlation of health care as a top issue by lag periods, media type and level of media usage by respondents 154 Table 4-16 Correlation of the environment as a top issue by lag periods, media type and level of media usage by respondents 157 Table 4-17 Correlation of consumer issues as a top issue by lag periods, media type and level of media usage by respondents 160 Table 5-1 Percentage share of media usage per by respondents 173 xi ACKNOWLEDGMENTS I wish to first and foremost express my thanks to my thesis supervisor Professor Fred Fletcher for his guidance, clarity of thought, and a host of his other many qualities I've drawn upon during my doctoral studies.