Evaluating Interactive Documentaries: Audience, Impact and Innovation in Public Interest Media by W ASSACHUSE1TS INSTITUTE OF TECHNOLOGY Sean Peter Flynn SEP 212015 B.A., Cinematic Arts (Critical Studies) University of Southern California, 2013 LIBRARIES Submitted to the Program in Comparative Media Studies/Writing in partial fulfillment of the requirements for the degree of Master of Science in Comparative Media Studies at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY September 2015 Sean Flynn. All rights reserved. The author hereby grants MIT permission to reproduce and to distribute publicly paper and electronic copies of this thesis document in whole or in part in any medium now known or hereafter created. Signature of Author: Signature redacted Department of Comparative Media Studies July 24, 2014 Signature redacted Certified by: William Uricchio Professor of Comparative Media Studies Thesis Supervisor Accepted by: Signature redacted V7, T.L. Taylor Director of Graduate Studies Comparative Media Studies 1 Evaluating Interactive Documentaries: Audience, Impact and Innovation in Public Interest Media by Sean Flynn Submitted to the Department of Comparative Media Studies On July 24, 2015, in partial fulfillment of the requirements for the degree of Master of Science in Comparative Media Studies Abstract Public interest media organizations are increasingly interested in experimenting with interactive and participatory approaches to documentary storytelling enabled by digital technologies. However, due to the experimental nature of these interactive documentaries, it is not yet clear whether the more active user engagements they require translate into outcomes like sustained attention, greater narrative comprehension, enhanced learning, empathy or civic engagement - never mind larger societal impacts like improved public discourse, behavior change or policy change. The shifting definitions and measures of complex, multi-dimensional concepts like "engagement" and "impact" is a challenge for public interest media organizations migrating to digital platforms - particularly at a time when audience activities have become far more transparent and funders place greater emphasis on "data-driven" impact measurement. This thesis explores the "theories of change" that inform institutional investments in documentary and examines how three public interest media organizations - the National Film Board of Canada, POV and the New York Times - are approaching interactive documentary production, attempting to define what constitutes success or impact - and how to measure it. I argue that we need new theories of change and evaluation frameworks that expand definitions of "impact" and "engagement," balancing public service mission with the strategic goals of audience development and the circuitous processes of artistic and technological innovation. This means looking beyond quantitative mass media era metrics, which fail to account for important qualitative dimensions of the user experience. I propose a new set of qualitative and quantitative measures that might better reflect the social and artistic aspirations of the interactive documentary, test assumptions in ways that can inform project design, and embrace the potentials of technology to transform the methods, ethics and process of documentary storytelling in the digital age. Thesis Supervisors William Uricchio, Professor, Comparative Media Studies Ethan Zuckerman, Director, Center for Civic Media 3 4 Acknowledgements I am profoundly grateful to have had the opportunity to work on this thesis with the mentorship, guidance and support of two brilliant advisors, William Uricchio and Ethan Zuckerman, who consistently challenged me to ask deeper and better questions and craft more compelling, forceful arguments. Thank you to the many practitioners I interviewed at the National Film Board of Canada, POV, the New York Times and elsewhere, whose insights helped give form and clarity to my understanding of a field that is still in formation itself. I'm particularly indebted to Kat Cizek, Adnaan Wasey and Tom Perlmutter for generously sharing their time, and to Sarah Wolozin for creating countless opportunities to connect with artists and scholars at the forefront of the field. There are so many others who provided the support - in big ways and small - that allowed me to begin this journey. I only have space to name a few, but I want to thank Beth Murphy and Kevin Belli for my first adventures in documentary filmmaking, AnnieLaurie Erickson for your strength and support, Jesse Shapins for your enthusiasm and advice, and the Fulbright Program and Michael Renov for opening the doors to continuing my education at times when I worried they might be closed. The most rich and rewarding part of my MIT experience was undoubtedly becoming part of a diverse community of creative and critical thinkers. The friendship and camaraderie I found in the OpenDocLab Fellows and in my fellow 15ers - Heather, Chelsea, Desi, Liam, Suruchi, Ainsley, Erik, Jesse and Yu - constantly inspired me and kept me going through the many ups and downs of graduate school. By the same token, I am grateful to my dear friends and colleagues Ben Fowlie and Caroline von Kuhn for giving me the space and flexibility to continue doing a job that I love while juggling the responsibilities of school. Thank you to my parents, who have always given me the freedom to pursue my dreams and ambitions, even when they have taken me in unconventional directions. And most of all, thank you to Jess. Your sense of humor, your critical insights, your unfailing optimism and your fierce love sustained me through every moment of self- doubt and allowed me to see this process as an opportunity to find my voice and become a better version of myself. I can't thank you enough for being my person. 5 Contents INTRODUCTION: Documentary in the Digital Age ................................................... 7 CHAPTER 1: Theories of Change ................................................................................ 20 CHAPTER 2: The National Film Board of Canada ....................................................... 59 CHAPTER 3: POV....................................................................................................................88 CHAPTER 4: The New York Times............................................................................... 115 CONCLUSION: A Decision at Every Turn ..................................................................... 139 6 INTRODUCTION: Documentary in the Digital Age "Documentaryis a clumsy term, but let it stand." -John Grierson, "First Principles of Documentary" (1933) "The internet *is* a documentary." -Kat Cizek, Webby acceptance speech (2008) Nearly nine decades have passed since John Grierson famously coined the term "documentary" in his 1926 review of Robert Flaherty's nonfiction film Moana. As a mode or genre of film, the word remains as clumsy and imprecise today as it was in Grierson's time, a fact borne out by the endless critical debates over documentary's claim to represent "reality" or "truth."1 Yet the term has come to signify not just a genre or mode of filmmaking but also a century-old traditionof socially engaged storytelling, produced and distributed across a wide variety of media - from cinema and television screens to print, radio and gallery walls. Borrowing from the artistic language of cinema, the investigative practices of journalism and advertisers' strategies for achieving social influence, documentary has always been a hybrid, alternative media form with a wide range of aesthetic and rhetorical functions.2 Although it has rarely found large audiences, financial profit or a stable institutional home, the documentary tradition has persisted for nearly a century, sustained by generations of practitioners, funders, critics and audiences 1 Winston, Claiming the Real. 2 Renov, Theorizing Documentary, 21. 7 who believe in its distinct social and artistic value. For Grierson and generations of filmmakers who have joined the tradition he helped to establish, documentary has represented an attempt to expand the artistic boundaries and political possibilities of cinema by recording lived experience, creatively shaping it into narrative form, and offering perspectives on the world that help audiences become more informed, engaged and compassionate citizens. The fluidity of the documentary form has become particularly evident in an era defined by tectonic shifts in the media environment. During the past two decades, networked digital media technologies - from the Web to social media to smartphones - have enabled an exponential growth in the amount of nonfiction media content being produced and the number of channels through which it is distributed. The rise of a "participatory culture" 3 has fundamentally transformed the relationships between professional media producers and their audiences, a distinction that some have argued has lost some of its relevance as media users play increasingly active roles in both producing and circulating content. Among legacy media organizations, this fragmentation of the media landscape and growing "audience autonomy"4 has produced uncertainty about the ability of their productions to attract and influence digital audiences in a competitive marketplace where attention is an increasingly scarce resource.5 Organizations committed to public service, which have typically provided the institutional support for much 3 Clinton et al., "Confronting the Challenges of Participatory