Evaluating Civic Technology Design for Citizen Empowerment by Erhardt Graeff B.S., Rochester Institute of Technology (2006) B.S., Rochester Institute of Technology (2007) M.Phil., University of Cambridge (2008) S.M., Massachusetts Institute of Technology (2014) Submitted to the Program in Media Arts and Sciences in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Media Arts and Sciences at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY June 2018 c Massachusetts Institute of Technology 2018. All rights reserved. Author............................................................................ Program in Media Arts and Sciences May 4, 2018 Certified by........................................................................ Ethan Zuckerman Associate Professor of the Practice Program in Media Arts and Sciences Thesis Supervisor Accepted by....................................................................... Tod Machover Academic Head Program in Media Arts and Sciences 2 Evaluating Civic Technology Design for Citizen Empowerment by Erhardt Graeff Submitted to the Program in Media Arts and Sciences on May 4, 2018, in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Media Arts and Sciences Abstract Civic technology should empower us as citizens. Despite its breadth as a field, civic technology often takes its lead from Silicon Valley companies that espouse design goals potentially hazardous to participatory democracy. In this dissertation, I explore: How might we design civic technologies for citizen empowerment and evaluate their impact on this goal? With their growing role as mediators of democracy, it is insufficient for civic technology designers to evaluate their designs in terms of ease of use and increased engagement with their platform. Research from political and developmental psychol- ogy shows the importance to lifelong civic engagement of learning experiences that cultivate a citizen's perception they can make change (political efficacy) and their belief in having responsibilities to the public good (civic identity). To achieve these positive feedback loops, we need a richer framework for civic technology design. This dissertation proposes two solutions: 1) empowerment-based design principles for civic technology and 2) a prototype toolkit for evaluating the impact of civic tech- nology on political efficacy. Because empowerment is contextual, the proposals here focus on tools and platforms built to support \monitorial citizenship," an increasingly popular form of civic engagement aimed at holding institutions accountable. To see these solutions in action, I report on a case study of SeeClickFix, a civic technology company that builds tools enabling citizens to report infrastructure problems to local governments. Two surveys of political efficacy and a randomized experiment with active users of SeeClickFix, followed by interviews with SeeClickFix staff, indicate the validity and utility of evaluating political efficacy as a measure of empowerment as well as the limitations of testing for incremental improvements. Thesis Supervisor: Ethan Zuckerman Title: Associate Professor of the Practice Program in Media Arts and Sciences 3 4 This doctoral thesis has been examined by the following committee: Ethan Zuckerman. Chair, Thesis Committee Associate Professor of the Practice Massachusetts Institute of Technology Peter Levine . Member, Thesis Committee Lincoln Filene Professor of Citizenship and Public Affairs Tufts University Evan Lieberman . Member, Thesis Committee Total Professor of Political Science and Contemporary Africa Massachusetts Institute of Technology Winter Mason . Member, Thesis Committee Data Scientist Facebook 5 6 Acknowledgments This dissertation is dedicated to my son Nikolai. He can earn as many or as few academic degrees as he wishes; I simply hope he is blessed with a lifetime of curiosity, inquiring both deeply and widely. Thank you to my wife, Kate. You make all things in my life possible; I would be lost without you. This is as much your PhD as it is mine. Thank you to my Mom, Dad, and sister Lydia. You have supported me through an absurd amount of education. Thank you to my advisor Ethan. You have pushed me to be more creative, impactful, and thoughtful since before I was your student. You are a dear friend and colleague. I am going to miss teaching, writing, speaking, and organizing alongside you, but I know that this dissertation also marks a beginning with opportunities to collaborate even more. Thank you to my committee: Peter, Evan, and Winter. Peter, apart from Ethan, you have been the most influential mentor in my life over the past few years; I have so much more to learn from you and look forward to our next chance to work together. Evan, I couldn't be more thankful for your willingness to join this unorthodox disser- tation and lend your expertise; I have grown a lot as a scholar thanks to your feedback. Winter, your important work at Facebook directly inspired this dissertation|it lit- erally would not exist without you; I'm proud to call you a friend and colleague. Thank you to J. Nathan Matias and Andrew Ho. Nathan, your friendship, col- legiality, and dedication to service continue to inspire me and illuminate my path toward doing impactful work. I miss you terribly, and yet you always find time to help me out no matter where you are or how busy. This dissertation does not exist without your eye for statistics and experimentation and your unending support. An- drew, your tutelage in my general exams and in coursework through all things data analysis and psychometrics, including teaching Nathan and I how to be thoughtful and rigorous social scientists, laid the foundations for this study. Thank you to my MIT Center for Civic Media family, who have given me the great 7 gifts of a loving community and of a sense of direction toward what empowerment really means: Abdulrahman, Alexis, Ali, Anushka, Bex, Catherine, Charlie, Chelsea, Cindy, Dan, Ed, Emilie, Ethan, Heather, Gordon, Jia, Joy, Jude, Lorrie, Luis, Matt C., Matt S., Mariel, Molly, Natalie, Nathan, Pablo, Petey, Rahul, Sands, Sasha, Whitney, Willow, and many more! Thank you to Samidh Chakrabarti and Funda Kivran-Swaine for hiring me at Facebook to explore these questions around the world, and for being on the vanguard of ethical civic technology design. Thank you to Kate Krontiris for your path-breaking work on civic technology and impact and for inviting me to be a part of the movement as a fellow researcher. I am continually inspired by you; I strive to be as thoughtful and generous in as many aspects of my life as you are. Thank you to the kind baristas at Peet's Coffee in Harvard Square, Starbucks in Davis Square, and the Tower Caf at Tufts University's Tisch Library for remembering my name and preferences and putting up with me during long stretches of writing and editing. And thank you to the acquisitions and circulation staff at the MIT Libraries who acquired and reserved over 80 books for me over the course of 6 years. Even if this dissertation accomplishes nothing else, I know it helped curate a collection of materials on civic engagement and civic technology at MIT that will serve future students investigating similar questions. 8 Contents List of Figures 13 List of Tables 15 1 The Need for Empowerment-based Design and Evaluation 17 1.1 Defining the Problem . 19 1.1.1 The Story of We the People . 19 1.1.2 The Logic of Civic Technology . 24 1.1.3 The Civic Empowerment Gap . 28 1.2 A Framework for Democracy, Citizen Empowerment, and the Role of Civic Technology . 30 1.3 Research Questions . 35 1.4 Scope of this Dissertation . 35 1.5 A Preview of Chapters 2{7 . 36 2 Citizenship, Civic Learning, and Empowerment in the Current Age of Civic Technology 39 2.1 Education and Equality . 39 2.2 Theorizing Civic Learning and Political Socialization . 42 2.3 The Need for Digital Literacy . 45 2.4 New Media, New Civics? . 48 2.5 From Voice to Empowerment . 51 9 3 Defining Monitorial Citizenship 55 3.1 History of the term \Monitorial Citizen" . 56 3.2 Contemporary Definitions and Examples of Monitorial Citizenship . 58 3.3 Locating Monitorial Citizenship within Contemporary Debates of Demo- cratic Theory and Practice . 61 3.3.1 Postmodern Citizenship . 61 3.3.2 Counter-Democracy . 62 3.3.3 Eyes on the Street . 63 3.3.4 Strong Democracy and the Participatory Citizen . 64 3.3.5 Everyday Politics and Public Work . 64 3.3.6 Connective Action . 65 3.3.7 The Actualizing Citizen . 66 3.3.8 Participatory Politics . 66 3.3.9 Sousveillance . 67 3.3.10 Monitory Democracy . 67 3.3.11 Participatory Governance . 68 3.3.12 Social Monitoring . 69 3.3.13 Accountability Technologies . 69 3.4 Why use \Citizenship" . 70 3.5 SeeClickFix as Monitorial Citizenship . 71 4 Empowerment-based Design Principles for Civic Technology and Monitorial Citizenship 73 4.1 Point of Departure: Feminist HCI . 76 4.2 Principle 1: Be Inclusive at Every Stage . 77 4.3 Principle 2: Give Users Agency . 81 4.4 Principle 3: Provide Opportunities for Reflection and Discourse . 84 4.4.1 Feedback Loops . 85 4.4.2 Deliberation . 86 10 4.5 Principle 4: Foster and Respect Communities . 87 4.6 Principle 5: Tell Stories with Data . 90 4.6.1 Leverage facts and data . 90 4.6.2 Personal experiences are data too . 91 4.6.3 Thinking narratively . 92 4.7 Principle 6: Anticipate Breakdown and Evaluate Rigorously . 93 4.7.1 mySociety and Facebook Research Programs . 94 4.7.2 Representation in Research . 96 4.8 On Institutionalization . 96 4.9 Evaluating SeeClickFix according to Empowerment-based Design Principles . 99 4.9.1 Inclusivity and Agency . 99 4.9.2 Reflection and Discourse . 100 4.9.3 Communities . 101 4.9.4 Storytelling with Data . 101 4.9.5 Evaluation . 102 5 Toward an Empowerment-based Research Toolkit (Methods) 103 5.1 Why Study Political Efficacy? .