Cleveland State University EngagedScholarship@CSU ETD Archive 2013 Reviewing the Critics: Examining Popular Video Game Reviews Through a Comparative Content Analysis Ben Gifford Cleveland State University Follow this and additional works at: https://engagedscholarship.csuohio.edu/etdarchive Part of the Communication Commons How does access to this work benefit ou?y Let us know! Recommended Citation Gifford, Ben, "Reviewing the Critics: Examining Popular Video Game Reviews Through a Comparative Content Analysis" (2013). ETD Archive. 578. https://engagedscholarship.csuohio.edu/etdarchive/578 This Thesis is brought to you for free and open access by EngagedScholarship@CSU. It has been accepted for inclusion in ETD Archive by an authorized administrator of EngagedScholarship@CSU. For more information, please contact [email protected]. REVIEWING THE CRITICS: EXAMINING POPULAR VIDEO GAME REVIEWS THROUGH A COMPARATIVE CONTENT ANALYSIS BEN GIFFORD Bachelor of Arts in Journalism Cleveland State University, Cleveland, OH May, 2009 submitted in partial fulfillment of requirements for the degree MASTER OF APPLIED COMMUNICATION THEORY AND METHODOLOGY at the CLEVELAND STATE UNIVERSITY May, 2013 THESIS APPROVAL SCHOOL OF COMMUNICATION This thesis has been approved for the School of Communication and the College of Graduate Studies by: ________________________________________________________________________ Thesis Committee Chairperson – print name ________________________________________________________________________ Signature School of Communication ________________________________________________________________________ (Date) ________________________________________________________________________ Committee Member – print name ________________________________________________________________________ Signature School of Communication ________________________________________________________________________ (Date) ________________________________________________________________________ Committee Member – print name ________________________________________________________________________ Signature School of Communication ________________________________________________________________________ (Date) ii In memory of Dr. Paul Skalski, You made friends wherever you went, and you are missed by all of them. iii ACKNOWLEDGEMENTS First, I would like to acknowledge to efforts of my original adviser, Dr. Paul Skalski. As a champion of video games, your input was instrumental in the construction of this thesis. Without your guidance, I might never have thought of studying video games academically in the first place. I wish you could have seen this to its conclusion. To Dr. Kimberly Neuendorf, without your mentorship on content and statistical analyses, this thesis would not have been possible. Of equal importance, I would like to thank you for taking on the lead advisory role at a moment’s notice. In the short time you have held this role, your feedback has been very fair and immensely helpful. Only because of your willingness to push forward in light of unfortunate circumstance has this thesis been possible. I would also like to extend thanks and appreciation to Dr. Bob Abelman. The wonderful experience I had in your media criticism class as an undergraduate helped to cement my interest in and enhance my understanding of criticism as its own distinct body of literature. I owe much of my interest in this topic to your teachings, and your advice has undoubtedly served to strengthen and enrich this thesis. To Dr. Anup Kumar, although your participation in this thesis stems from tragic conditions, I am very glad to have your involvement. Your input during the review process has been both welcome and helpful, but your influence in my writing, like Dr. Abelman’s, extends back to my undergraduate career. Finally, to my parents, my siblings, and my wife, your occasional good-hearted nagging was just enough to push me to the finish. iv REVIEWING THE CRITICS: EXAMINING POPULAR VIDEO GAME REVIEWS THROUGH A COMPARATIVE CONTENT ANALYSIS BEN GIFFORD ABSTRACT The purpose of this study is to evaluate the current critical climate in popular online video game reviews (i.e., video game criticism written for a general audience). So far, most of the research published in this area focuses on how the reviews reflect the games themselves, rather than strictly examining the content of the reviews in this growing body of literature. This study uses computer-aided text analysis (CATA) supplemented with human coding to identify typological differences between film and video game reviews, as well as differences in theory usage and critical thought and style. Video game reviews are more concerned with the price of the work being reviewed, supporting the notion for a utility theory of video games. Game reviewers also tend to find redeeming qualities even in very flawed games, suggesting they are either overly passionate and/or concerned about keeping advertisers happy. Although not at the exceedingly high levels as previous studies, the author finds support for using usability heuristics (e.g., responsiveness of controls, use of in-game tutorials) to review games. Neither body of popular criticism examined delves deeply into theoretical frameworks for auteur or feminist theories, but discussion is provided as to how the reviewers could address these issues should they choose to do so. v TABLE OF CONTENTS Page ABSTRACT……………………………………………………………………………….v LIST OF TABLES………………………………………………………………………..ix LIST OF FIGURES………………………………………………………………………..x CHAPTER I. INTRODUCTION………………………………………………………………1 II. LITERATURE REVIEW………………………………………………………5 2.1 Definition of Video Games…………………………………………….5 2.2 Background……………………………………………………………6 2.3 Defining Criticism……………………………………………………10 2.4 History of Film Criticism……………………………………………..14 2.5 Auteur Theory………………………………………………………..14 2.6 Feminist Theory……………………………………………………...16 2.7 Usability Principles for Video Game Design…………………………18 2.8 Utility Theory of Video Games……………………………………….21 2.9 Previous Research……………………………………………………25 2.10 Rationale……………………………………………………………26 2.11 Comparing Video Games and Film………………………………….28 2.12 Research Questions…………………………………………………33 III. METHODS…………………………………………………………………..36 3.1 Conceptualization and Operationalization of Variables……………...36 3.2 Human Coding……………………………………………………….40 vi 3.3 Sampling……………………………………………………………..40 3.4 Reliability Check……………………………………………………..44 IV. RESULTS……………………………………………………………………47 4.1 Information About Results…………………………………………...47 4.2 Analysis for RQ1……………………………………………………..47 4.3 Analysis for RQ2……………………………………………………..49 4.4 Analysis for RQ3……………………………………………………..50 4.5 Analysis for RQ4……………………………………………………..51 4.6 Analysis for RQ5……………………………………………………..52 4.7 Analysis for RQ6……………………………………………………..54 4.8 Analysis for RQ7……………………………………………………..61 V. DISCUSSION…………………………………………………………….......65 5.1 Overview……………………………………………………………..65 5.2 Limitations…………………………………………………………...72 5.3 Directions for Future Research……………………………………….74 5.4 Conclusion……………………………………………………………76 REFERENCES…………………………………………………………………………..77 APPENDICES…………………………………………………………………………...90 A. ON THE BORDERS OF THE CLASSIC GAME MODEL………………....91 B. USABILITY PRINCIPLES FOR VIDEO GAME DESIGN…………………92 C. DICTIONARIES……………………………………………………………..96 D. HOW METASCORES ARE CALCULATED……………………………...107 E. CODING MANUAL FOR EVALUATING THE CRITICS………………..110 vii F. HOW TO OBTAIN REVIEWS……………….……………………………118 G. LIST OF REVIEWS USED IN STUDY…………………………………....126 H. LIST OF SAMPLE REVIEWS FOR RELIABILITY CHECK…………….136 I. MEAN COMPARISON AND CORRELATION OF DICTIONARY SCORES………………………………………………………………..138 J. MEAN COMPARISON AND CORRELATION OF CRITICAL THOUGHT AND STYLE…………………………………………………………....140 K. MEAN COMPARISON AND CORRELATION OF REVIEW PURPOSE..142 L. AVERAGE METASCORES OVER TIME………………………………...144 M. DIFFERENCES BETWEEN FILM REVIEW TYPES……………………..145 viii LIST OF TABLES Table Page 1 Inter-coder Reliability Check…………………………………………………….44 2 Mean Comparison of Dictionary Scores Between and Correlations to Review Types……………………………………………………………………..48 3 Percent Occurrence and Correlations of Critical Thought and Style to Review Types…………………………………………………………….50 4 Percent Occurrence and Correlations of Review Purpose to Review Types……..55 5 Percent Occurrence and Correlations of “Auteurism” to Review Types…………56 6 Frequency of Usability Principles for Video Game Design in Popular Gaming Criticism and Correlation to Publication Score…………………58 7 Mean Comparison of Review Scores and Length Between Review Types………61 8 Mean Comparison of Dictionary Scores Between General Readership, Industry Insider, and Independent Film Reviews…………………………………145 9 Significant Differences in Critical Thought and Style Categories Between General Readership, Industry Insider, and Independent Film Reviews…146 ix LIST OF FIGURES Figure Page 1 Chart comparing consoles in 1981, taken from the first issue of Electronic Games Magazine…………………………………………………………………..8 2 Mean Comparison of Dictionary Scores Between Review Types………………138 3 Correlations of Dictionaries to Review Types…………………………………..139 4 Percent Occurrences of Critical Thought and Style By Review Type………….140 5 Correlations of Critical Thought and Style to Review Types …………………..141 6 Percent Occurrences of Review Purpose Between Review Type………………142 7 Correlations of Review Purpose to Review Types……………………………...143 8 Average Metascores Over Time……………………………………...…………144 x CHAPTER I INTRODUCTION There is no question video games have become