CODED VISUALIZATION: THE RHETORIC AND AESTHETICS OF DATA-BASED CULTURAL INTERFACE A Dissertation Presented to The Academic Faculty by Tanyoung Kim In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the School of Literature, Media, and Communication Georgia Institute of Technology May 2013 COPYRIGHT 2013 BY TANYOUNG KIM CODED VISUALIZATION: THE RHETORIC AND AESTHETICS OF DATA-BASED CULTURAL INTERFACE Approved by: Dr. Carl DiSalvo, Advisor Dr. John Stasko School of Literature, Media, and School of Interactive Computing Communication Georgia Institute of Technology Georgia Institute of Technology Dr. Jay David Bolter Dr. Johanna Drucker School of Literature, Media, and Graduate School of Education and Communication Information Studies Georgia Institute of Technology University of California, Los Angeles Dr. Ian Bogost School of Literature, Media, and Communication Georgia Institute of Technology Date Approved: April 2, 2013 ACKNOWLEDGEMENTS It is difficult to explain how much gratitude and thankfulness I feel to have support from these people. First of all, I especially thank my thesis readers. Without my advisor, Carl DiSalvo’s guidance, encouragement, critiques, and patience, I would never have finished this dissertation. John Stasko has also supported me endlessly since I first took his class that opened my eyes to visualization. Jay Bolter and Ian Bogost, the great digital media scholars, have guided me not to miss critical points in my dissertation. I am grateful that Johanna Drucker, who wrote the most significant books for my research, served as one of my committee members. I also thank other fantastic faculty members in the Digital Media Program. Brian Magerko supported me in successfully performing my first research in Atlanta. Janet Murray has always inspired me with her ingenious insights and knowledge. Ali Mazalek’s class stirred my interest in computational design. A former faculty member, John Sharp, taught a class on aesthetics that helped me to find paths when I was overwhelmed with this grand topic. My dearest friend, Heekyoung Jung encouraged me to survive the entire Ph.D. journey. She has supported me not only with her loyal friendship but also with advice as a great researcher in a close field. My fellow Ph.D. students are also invaluable to my survival: Hye Yeon Nam, Jean Ho Chu, Hank Blumenthal, Jill Fantauzzacoffin, and Nettrice Gaskins. My life in i Atlanta would have been miserable without Shannon Yao and Christine Wu, the sweetest people I know. Public Design Workshop members gave me advice along the long journey: Jon Lukens, Thomas Lodato and Tom Jenkins. I also appreciate Rebecca Rolfe for reading the manuscript and correcting my bad English. The best geek ever, Melanie Richard always solved any technical problems I had. Another geek, Tae Won Cho never said that I was bothering him with my stupid coding questions. Jeff Rosenblum and Clarin Ellard made me physically healthier and stronger. I just cannot list all other friends who supported me. At last, I thank my family for loving and enduring me. ii TABLE OF CONTENTS Page ACKNOWLEDGEMENTS i LIST OF TABLES ix LIST OF FIGURES x SUMMARY xiv CHAPTER 1 Introduction 1 2 Identifying problem spaces 7 2.1. Sketching the landscape 7 2.1.1. Digital media with embedded data 8 2.1.2. Messages embedded in the data-based digital media 15 2.2. Current approaches 19 2.2.1. Information Visualization for presentational purposes 19 2.2.2. Graphic design with unrefined data 21 2.2.3. Human-centered computing research for persuasion 25 2.2.4. Data journalism 27 2.3. Research questions 29 2.3.1. Deficiency in research of related fields 29 2.3.2. Q1: New affordances of computational visualization 31 2.3.3. Q2: Designing visualization as a cultural interface 34 2.4. Design research as methodology 34 2.4.1. Design knowledge as outcome of design research 35 2.4.2. Situating visualization in design research 38 iii 2.4.3. Modern graphic design for digital media discourses 40 2.4.4. Digital media studies through critical design practices 42 3 The rhetoric of data visualization 46 3.1. Theories for the rhetoric of visualization 46 3.1.1. Communication theories 47 3.1.2. Visual rhetoric 48 3.1.3. Theories of Rhetoric in design 50 3.1.3.1 Buchanan on rhetoric 50 3.1.3.2 Other research on rhetoric and visual communication design 52 3.2. Rhetoric of non-digital visualization 53 3.2.1. Social involvement of printed media in the 20th century 54 3.2.1.1 Use for propaganda 54 3.2.1.2 Visual symbols for educational purposes 57 3.2.2. Rhetorical analysis of historical examples 59 3.2.3. Classical artifices for deceiving visualizations 62 3.2.3.1 Selection of favorable data set 62 3.2.3.2 Use of decorative graphics 64 3.2.3.3 Other miscellaneous tricks 65 3.3. Expansion of rhetoric in digital environment 67 3.3.1. Views on digital rhetoric 68 3.3.2. Procedural rhetoric 71 3.3.3. Digital media literacy and political engagement 74 3.3.4. Participatory culture 75 3.4. Influence of computation on the rhetoric of visualization 77 3.4.1. Current view on rhetoric in visualization research 77 iv 3.4.2. Users' creating and gathering data 80 3.4.3. Experimental forms by digital production 84 3.4.4. Rhetoric of interactivity 86 3.4.4.1 Interaction for further exploration 86 3.4.4.2 Procedural rhetoric through dialogues 87 3.4.5. Sharing beyond experiences within visualization 89 3.4.5.1 Off-visualization Engagement 89 3.4.5.2 Coherent identity throughout multiple platforms 90 4 The aesthetics of data visualization 92 4.1. Current views on aesthetics 92 4.1.1. Limited understanding of aesthetics in InfoVis 93 4.1.2. Early perspectives from computer graphics 96 4.1.3. Aesthetics with instrumental concerns in HCI 97 4.1.4. New media theories for aesthetics 101 4.2. Defining aesthetics for data visualization research 105 4.2.1. Look-and-feel linking beauty and utility 106 4.2.2. Aesthetic communication for trustworthiness 108 4.2.3. Situated aesthetics 110 4.3. Aesthetic roots in modern graphic design 113 4.3.1. Formalization of modern visual language 114 4.3.1.1 Early data graphics 114 4.3.1.2 Bauhaus 116 4.3.1.3 International Typographic Style 118 4.3.1.4 Science of transparent typography 121 4.3.2. Rationalism grounding the connection 122 v 4.3.3. Examples of remediation 125 4.4. Influence of computation on the aesthetics of visualization 130 4.4.1. Encyclopedic cultural contents 133 4.4.2. Remediated variable forms for modular data 134 4.4.3. Aesthetics of pre-designed forms 135 4.4.4. Interaction bridging functionality and storytelling 138 4.4.5. Platform for secondary data exploitation 139 4.4.6. Direct visualization for cultural visual media 140 5 Characterizing coded visualization 143 5.1. What code means in visualization 143 5.1.1. From computation to code 144 5.1.1.1 Code as the quality of digital process 144 5.1.1.2 Code as symbolic messages 145 5.1.2. Linking aesthetics and rhetoric 147 5.2. Differences form previous concepts of visualization 148 5.2.1. Rhetorical use of aesthetic visualization 149 5.2.2. Beyond an analysis tool in cultural analytics 151 5.2.3. Cultural interface 153 5.3. Coded visualization for research-through-design 155 5.3.1. Process of coded visualization 156 5.3.1.1 Previously suggested process of visualization 156 5.3.1.2 Expanded pipeline of visualization 159 5.3.2. Design criteria for coded visualization 161 5.3.2.1 Participatory and mash-up cultural data 163 5.3.2.2 Disagreement and conflict in contents 165 vi 5.3.2.3 Cultural references into computational forms 166 5.3.2.4 Interaction for narratives and provocation 167 5.3.2.5 Proximate to current civic events or issues 168 5.3.3. Examples of coded visualization 168 5.3.3.1 Reconstruction2012 169 5.3.3.2 170 5.3.3.3 Home and Away: Iraq and Afghanistan War Casualties 170 5.3.3.4 The NameVoyager 172 6 The Political Grid Project 174 6.1. Background: Twitter as a public sphere 175 6.1.1. Social influences of Twitter 175 6.1.2. New Approaches to political engagement and expression 179 6.2. Designing coded visualization 181 6.2.1. Technology and site structure 182 6.2.2. Contrary contents from a major civic events 184 6.2.3. Voting on tweets as computational comments 186 6.2.4. Visualizations of collective opinion 187 6.3. Discussion and critiques 193 6.3.1. User participation 193 6.3.2. Rethinking the design criteria of coded visualization 195 6.3.2.1 Encouragement of participation 195 6.3.2.2 Presentation of contents for direct comparison 196 6.3.2.3 Cultural coding besides colors 196 6.3.2.4 Extensive navigation throughout multiple visualizations 197 vii 6.3.2.5 Targeting a longer term of political issues 198 6.4. Further design research agendas 199 6.4.1. Critical design challenging the objectivity of forms 200 6.4.2. Visualization as an immersive narrative space 200 6.4.3. Unique forms for user engagement 201 7 Conclusions 203 References 210 viii LIST OF TABLES Table 1. Design spaces and matching the new rhetoric and aesthetics 163 Table 2. Design space and matching design criteria of coded visualization 163 Table 3.