Design Research for Media Development a GUIDE for PRACTITIONERS

Design Research for Media Development A GUIDE FOR PRACTITIONERS

By: In Collaboration with: Design Research

AUTHOR: CREATIVE DIRECTOR: Panthea Lee Jennifer Thibault for Media EDITORS: DESIGNER: Kate Petty, Ethan Wilkes Mollie Ruskin

PRODUCTION MANAGER: ADVISORS: Development Patrick Ainslie Zack Brisson, Amanda Noonan A GUIDE FOR PRACTITIONERS

© 2013 Reboot Design LLC. The material in this work is subject to copyright. However the authors encourage the dissemination of knowledge. We invite you to contact us should you seek to reproduce this work in whole or in part. For all queries on rights and licenses, we can be reached at [email protected]. Reboot // 45 East 20th St, 5th Floor, New York, NY USA 10003 +1 212 388 1010 // www.theReboot.org

ACKNOWLEDGMENTS With special thanks to Mark Frohardt, Amanda Noonan, and Sam De Silva from the Internews Center for Innovation and Learning, Oren Murphy from Internews Asia, and Charmaine Anderson, Arif Khan, Muhammad Ghawas, and the entire Internews Pakistan Country Office. The case studies and examples throughout this book are drawn from a design research investigation commissioned by Internews in Fall 2012. It took place in the Federally Administered Tribal Areas (FATA) and Khyber Pakhtunkhwa (KP) regions of Pakistan, with fieldwork executed by a dedicated field team. Reboot is grateful to our researchers Amjad Ali, Qaiser Ali, Sumaira Asghar, Shakir Ullah Dawar, Arif Khan, Farooq Ali Khan, Jasim Khan, Kamran Khan, and Rizwanullah Khan—thank you for your hard work, your perseverance, and your commitment to finding the real answers. Finally, to all those in FATA and KP who welcomed our team and shared their By: In Collaboration with: lives with us—thank you. Contents

2 Foreword 4 About This Guide 10 Introducing Design Research 16 PHASE 1: Define 50 PHASE 2: Plan 66 PHASE 3: Collect 88 PHASE 4: Synthesize 110 PHASE 5: Design 122 Looking Ahead 124 About Reboot 125 About ICIL

1 // FOREWORD

Foreword

In 1980, I was working on the Thai- relationships with the communities I was drawn to Internews because We were most delighted to find in Cambodian border in a refugee camp they are seeking to serve. This is their work has long embodied a Reboot an organization that shares of over 130,000 displaced individu- a process that goes well beyond similar ethos. Program design is our values. Reboot brings significant als. This was my first experience in simple question and answer surveys. firmly rooted in the principles of expertise in formal, as well as the international humanitarian and This is a process that demands an strong communications and an intuitive, human-centered design. development space. A number of ethnographer’s ear and a journalist’s intimate relationship with the end This guide was developed to assist organizations were present providing inquisitiveness to uncover the behav- user. Our evolution into human- those working within Internews to assistance, and all were effective in ioral insights that drive a community. centered design, therefore, is only challenge our embedded assump- their own way. But one organization natural. This approach allows us tions and deepen our relationships impressed me in particular for the I was also fortunate to work with to further refine our engagement with the communities we serve. This degree of proximity they maintained Fred Cuny of Intertect, a master of with local populations and com- guide is also written for the broader with those they were working to this approach, during the return of munities to develop solutions fitted community of practice that hopes to serve. The doctors and nurses of Kurdish refugees to Northern Iraq in to their daily needs and aspirations. understand the continuously chang- Médecins Sans Frontières would 1991. Although called in to advise This approach also allows us to move ing information ecosystems that often just sit with their patients and commanders of the multinational beyond a program design process increasingly define and constrain local staff, beyond the scope of their forces responsible for the repatria- that simply informs our initial development practice. work and questions—beyond the tion, Fred, as always, first went deep engagement to one that ensures traditional relationship of aid worker into the camps to develop plans deep and lasting relationships We hope you find the ideas and and aided. based on an ever-evolving under- throughout the life of a project, methods of this guide valuable in standing of the needs and capacity keeping objectives based on shift your work and welcome your views In the over three decades since that of local leaders and individuals. Fred -ing priorities relevant and on target. and suggestions. first encounter, the organizations was deeply committed to solutions and individuals I have seen success- that included the “end user” in both Mark Frohardt fully fulfilling their development mis- conception and implementation. Executive Director sion are those who have built close Internews Center for Innovation & Learning

2 Design Research for Media Development 3 // about this guide

About This Guide

This is a hands-on reference guide This resource is our effort to share Throughout this guide, several special features will for media development practitioners. our process with our Internews help you better understand and implement design research: colleagues and among the broader It is based on principles and practices media development community, in of design research that have been the hopes that it will inspire practice long used by the private sector, and and discussion and have an impact grounded in the experience Reboot far beyond our immediate goals. has gathered in designing and implementing international develop- This is not a comprehensive guide ment projects around the world. to program design. As a field, media development has a rich history of FATA in Focus use real-life examples Tools signal a resource to help This guide was born out of a col- innovation, thoughtful leadership, from an Internews project in the FATA practitioners implement the design laboration between Reboot and and tried-and-true processess— region of Pakistan to illustrate research process. Sample templates Internews, through its Center for many pioneered by Internews. design research principles in action. and examples from the Internews They show how the approach both Pakistan Case Study, can be found at Innovation and Learning, and its Indeed, many ideas in here will requires adaptability to context and www.design.internews.org/tools. Pakistan Country Office. Together, we likely be familiar to readers working offers rich contextual insights. sought to understand the complexity in this systems-oriented field. Thus, of the information ecosystem in the our hope is that this guide will help Federally Administered Tribal Areas practitioners integrate design (FATA) region of Pakistan. Our ulti- research principles and practices A companion website for this guide can We are eager for feedback and mate goal was to design contex- naturally into their existing work, be found at www.design.internews.org. to hear about your experiences in This online platform will be frequently applying these approaches. Please tually appropriate programs that stretch their creativity, and find new updated with resources to support send thoughts to [email protected] improved access to information solutions. practitioners seeking to integrate and [email protected]. by communities in this region. design research into their work.

4 Design Research for Media Development 5 // about this guide // Principles in Practice

Harnessing Design Research in FATA

This guide features a running case studie to illustrate how design research can apply to media development. All examples are drawn from an investigation conducted in the Federally Administered Tribal Areas (FATA) region of Pakistan in September 2012.

About FATA Socially, economically, and politically, Politically, FATA has traditionally Yet opportunities for change are But the region remains media dark, FATA is the least developed region been ruled, not governed, by on the horizon. The upcoming 2013 with variable access to mobile in Pakistan. Sixty-six percent of the unelected tribal leaders. The region general election will be the first time networks and Internet and tight population lives below the poverty is strongly conservative and cultur- that political parties are able to form restrictions on journalists. Although line;1 its key industries are farm- ally isolated, constraints that are and operate in FATA, meaning the there has been an increase in media ing, smuggling, and illegal drugs;2 heightened by geographic and first chance for FATA communities access and channels in recent years, unemployment is estimated at 60 security factors. United States mili- to have their interests represented little is known about the informa- to 80 percent;3 and the literacy rate tary operations using unmanned on the national stage. To leverage tion and media behaviors of FATA is 22 percent (and 1.5 percent for aerial vehicles (drones) have tar- this moment, independent and communities. females).4 geted the region since 2004, in an good quality information about the attempt to defeat Taliban and Al- political process will be vital to its Qaeda militants. Drone strikes have residents. killed up to an estimated 3,400 people.5

6 Design Research for Media Development 7 // about this guide // Principles in Practice HARNESSING DESIGN RESEARCH IN FATA

Project Background Project Approach Given the FATA’s communications the elections? Furthermore, what did It was determined that design Due to the security restrictions, landscape and its population’s low they think about politics in Pakistan? research—a research and analytical Reboot and Internews staff did literacy rates, Internews was inter- And finally, how should informa- process that combines ethnograph- not travel in FATA. Reboot and ested in the potential of interactive tion be presented? What sources of ic, journalistic, and systems think- Internews managed field research voice response (IVR) technology to information were perceived as trust- ing approaches—could help answer from Islamabad, and conducted get information about political pro- worthy, and how did people evaluate these complex questions, and help training and synthesis workshops cesses to FATA communities, particu- the credibility of information? Internews identify opportunities to in Islamabad and Peshawar. larly around the upcoming elections. improve the information ecosystem But as teams began planning such Internews realized a more nuanced for FATA communities. an initiative, they realized they need- understanding of the information The research findings will be ed answers to a few key questions to and communications landscape in A design research investigation released in early 2013. enable strong program design. the region was necessary to ensure was conducted over four weeks programs would be appropriate and in September 2012. A team of 10 To start, it was critical to understand impactful on the national stage. To researchers from FATA conducted what were the technology behaviors leverage this moment, independent the fieldwork, with training and man- of FATA communities, and whether and good quality information about agement from Reboot. The study IVR was even a realistic option. And the political process will be vital to covered 7 agencies in FATA and beyond the choice of communica- its residents. And, in recent years, Khyber Pakhtunkhwa (KP) and 125 tions channel, what kind of informa- there has been an increase in media respondents were interviewed. tion did people want? What did FATA access and channels within the FATA. communities already know about

1 Shuja Nawaz and Arnaud De Borchegrave, FATA - A Most Dangerous Place, (Washington, D.C.: Center for Strategic and International Studies, 2009). 2 Shahid J. Burki, “Economics and Extremism,” DAWN.com, n.d. 3 “FATA: Inside Pakistan’s Tribal Regions,” New America Foundation, n.d. 4 Government of Pakistan, United Nations Children’s Fund, and the World Food Programme, Federally Administered Tribal Area (FATA) Multiple Indicator Cluster Survey (Peshawar: Government of Pakistan, 2009) 5 “The Year of the Drone,” Counterterrorism Strategy Initiative, New America Foundation, 2012.

8 Design Research for Media Development 9 // INTRODUCING DESIGN RESEARCH

Introducing Key Principles of Design Research

1. Understand humans and institutions. The best solutions meet the needs of users, while respecting Design their capacities and constraints, and they do so at the community and institutional levels. Understanding both, therefore, is key.

2. Discard assumptions. “Design by ToR (Terms of Reference)” is tough. The best Research solutions come from the real world, so learn to recognize personal biases and assumptions, as well as those of others, to see the way forward.

3. Remain flexible. It is impossible to predict the solution (or even the best path for finding the solution) at the start of a project. Good program design requires That is why design research is a Change is the norm, so be open-minded and adaptive. two things: empathy for people valuable and important starting point for media development. Design and understanding of place. 4. Consider diverse stakeholders. research is more than an exercise Design research is a valuable A program is only as good as the sum of its parts. Examine in knowledge gathering; it is a pro- tool to achieve both. the needs and capacities of stakeholders along your program active, rigorous, and intellectual delivery chain, such as regulators, media outlets, journalists, process tailored to create concrete Design research is useful whether and audiences, to ensure the design accommodates each programmatic interventions. of them. you are designing an iPhone in Palo Alto or a media development program Blending practices from ethnography, in Pakistan. But for projects like 5. Be responsible. journalism, and systems thinking, the latter—whose designers often Anticipate and mitigate the unintended negative impacts design research helps practitioners come from vastly different back- a project may have on communities. From selecting understand the causes, relationships, grounds than their users, and where respondents to introducing the research, consider how and human dimensions of complex actions and words may impact local dynamics. the context is extremely complex— contexts—and then provides tools developing empathy and under- to incorporating this knowledge into standing requires greater effort and the design of innovative and realistic unique skillsets. interventions.

10 Design Research for Media Development 11 // INTRODUCING DESIGN RESEARCH

Design research is a process of The process of design is fluid and discovery, not of verification. It is creative, it is constantly refined particularly useful in the early stages and adjusted for different contexts— of a project, where opportunities are design research is similarly versatile. vast and the paths to seize them To that end, while design research Design Research is Not.... are unclear. It is a process to discard is inherently valuable for program assumptions, and a means to design, it can be integrated into Design research is a powerful tool for understanding complex design interventions that are truly fit other approaches already used by human systems and surfacing key factors that are critical for for context. media development professionals. program design. Just as no two contexts are alike, Because, ultimately, while success- there is no one formula for good It is not: ful programs require funding, tech- research and design. A substitute for quantitative research. nology, and management, understanding of context is paramount. Beyond the field of media develop- Design research is not intended to yield statistically significant findings. Yet it complements quantitative research by surfacing ment, design research has been key insights that are invisible to less nuanced methods. As nec- widely utilized to answer a range of essary, qualitative insights can be probed further for represen- questions relating to governance tativeness through survey work. and international development pro- Integrate gramming. These include: design A replacement for stakeholder engagement. research Design research emphasizes deep engagement with communi- • How can we design a social into M&E. ties and stakeholders with the aim of eliciting data to inform accountability system to program design. It is not intended to replace a program’s maximize citizen engagement stakeholder consultation and sensitization activities. This guide focuses on the in the design and delivery of basic applications of design research services, and compel government A solution for every program. response based on citizen input? in program design and develop- Design research is most useful when the challenge or context ment, but the process is equally is new, complex, or involves diverse stakeholders. Certain valuable for M&E efforts. • How can we help diverse and often inaccessible human trafficking types of programs, such as infrastructure development or victims find the means to escape policy advocacy, may demand other approaches, with design The ability to assess and explain research playing a secondary, or even non-existent, role. project outcomes in terms of their situations? context is essential to learning— • How can the international community particularly when projects fall short. leverage new technologies to support good governance, social cohesion, and economic development in a country that has undergone a popular revolution?

12 Design Research for Media Development 13 // INTRODUCING DESIGN RESEARCH The Design Research Process

There are five key phases in the design research process. The following chapters will explain each of these in detail, along with practical advice and case studies.

PHASE 1 PHASE 2 PHASE 3 PHASE 4 PHASE 5 Define Plan Collect Synthesize Design

Articulate a program’s Develop an intellectual Work to understand Make sense of the Distill findings into program ultimate objective, and framework for informa- the people the research, and draw design recommendations define the questions that tion that is needed to program seeks to patterns and insights and guidelines for use by must be answered to help meet the design serve, and the actors from the data. Ensure the implementing team. achieve it. Set clear goals challenge, and an that will be serving that findings are It is also helpful to present to establish boundaries operational plan for them. Dive into the useful and actionable findings in a format for what falls outside how to get that context. for program design. accessible to wider immediate relevance or information. audiences to benefit the current capacity. larger community of practice.

Design research can and should has its limitations when examining inform programs spread across be used in conjunction with other complex systems. To design a good areas with geographic variance, or approaches, such as participatory survey, you need to know what you programs that will be expanding to a rural appraisal (PRA), social network are looking for and how that information significant scale. analysis, the Delphi Method, or may manifest in the particular context. audience research surveys. In fact, Qualitative research, on the other Resources, human and financial, practitioners will find similarities, or hand, excels at building rich portraits as well as time will also be factors even shared tools, between some of of a people or place, and at mapping in selecting research approaches. these approaches. the relationships between people, Thankfully, design research scales places, cultures, and institutions. well. It can be effective as a small- In combining research approaches, it But insights surfaced by design scale process as well as a much is a matter of sequencing and scale. research may merit additional deeper expansive investigation. Quantitative research, for example, surveying, especially if they will

14 Design Research for Media Development 15 Phase 1 // DEFINE

Define the Challenge

Phase 1:

All development projects These kinds of assumption-based start with a challenge. But too projects risk failure, no matter how Define often, a project will define that many times or where else they challenge by presupposing a have worked before, because they solution. Design research helps do not consider how people in a specific context actually behave or us start with the right questions. the unique characteristics of the environment. Even if a proposed If, for example, a challenge people solution, such as the SMS information face is poor information or media service, may suit a particular context, access, practitioners might ask: it is impossible to know how to design “How can we support independent a service that is broadly accessible, community radio?” or “How might has locally relevant content, and we develop an SMS-based informa- which can be cost-effectively and tion delivery service?” sustainably maintained.

16 17 Phase 1 // DEFINE // Research Framework Research Framework

Framing and defining a design chal- Once a precise and concise design The research framework is a living In practice, priorities tend to shift lenge, therefore, is key for program challenge has been determined, the document that defines your research from what was originally put on design, and for the design research team should be able to determine themes and questions, but its focus paper. All practitioners know about that will inform it. Start from the the information it will need to be will evolve and narrow over time, first the importance of flexibility and wrong perspective, or with a set of able to tackle the challenge. This is as feedback is gathered from stake- striking a balance between planning assumptions, and you can spend where design research comes in. A holders, and then even more so over and operations. The research frame- your whole project working on the strong solution for any context starts the course of field research. work creates a way to adjust to shift- wrong thing. with a strong understanding of that ing donor and stakeholder priorities, context. Inspired by the design chal- While not a record of insights or find- while also keeping the on-the-ground Thus, a good design challenge lenge, we are now ready to build our ings, it is reflective of an increas- realities in focus. should: Research Framework. ingly sophisticated understanding of the context, honing in closer to the Speak to a user need. actual truth as the research unfolds. Over time, more targeted and more Do not speak to an organizational need, a technological opportunity, or detailed questions arise—this is a a donor directive. good sign.

Be open-ended in terms of how that need might be addressed. It should not predefine a solution.

Include the constraints that the design needs to satisfy. These may include meeting certain timelines, collaborating with certain Use the partners, or building upon existing framework Research programming. to align Framework stakeholders.

It can help stakeholders The complete develop shared understanding Research Framework of the project goals, processes, from the Internews and expected outcomes. During Pakistan Case Study check-ins, use it to reflect the can be found at: evolving understanding of the www.design.internews. operational context, and thus org/tools. program priorities.

18 Design Research for Media Development 19 PhasePhase 1 //1 //DEFINE DEFINE // // Research Research Framework Framework Principles in Practice

Key Components of a Research Framework:

Project Objectives. Specific goals, both concerning the questions to explore and the projected deliverables based on research findings. Defining the Challenge: A Collaborative Process Key Opportunities and Challenges. From institutional priorities, desk research, and expert consultations.

Guiding Themes and Questions. Coming up with a robust and for low-literacy users, like many Also grounded in desk research and expert consultations, realistic design challenge was FATA populations, and the upcoming these drive the field research. no easy task. Our project had elections were certainly a factor in planning timelines. Target Respondents and Sampling Plan. eight people representing four teams from two organizations Outline of different user and stakeholder groups to consult, Ultimately, however, the team spread across four cities (and and how they will be recruited. agreed to keep the prompt as broad that is not including travel). as possible: “Identify opportunities Methodology. This is what we did: Summary of the research methods to be used, to improve information access along with the rationale, role, and specific focus of each. and relevancy for communities in Based on Internews’ existing work the FATA.” While technology and Operational Plan. and team members’ experience new media were of interest, they Field logistics, including locations, timelines, research working in the region, everyone were excluded from the challenge schedule, team composition, and roles. agreed that FATA communities needed definition to not prematurely bias better information channels to get findings. And while Internews had Question Guides and Scripts. information about the world, and to timelines and programming that Concrete plan for the research team to explore key share information about themselves. could be strategically integrated with themes in respondent interviews. But there were differing opinions the project, it wanted to untether as to how this should be done, and the research of institutional priorities what timelines it should be accom- to truly benefit from the generative plished within. nature of design research. The challenge, therefore, was intentionally free of Interactive voice response (IVR) constraints to encourage research technology showed promise as that may contribute to the broader a news and communications tool development community.

20 Design Research for Media Development 21 Phase 1 // DEFINE // Research Framework Principles in Practice

STEP 1: Map the Issues

To understand a challenge, examine it, systematically, from all angles. Write down your design challenge, then write down all the questions you might have to answer to be able to tackle the challenge.

Then, group the questions to determine key themes that will drive the research.

For information- and media-related research, practitioners Determining Key Themes may want to consider:

Trends over time. Contours of trust and influence. In addition to current attitudes and To the extent that trust and The work in Pakistan aimed to We began by defining ‘information behaviors, examine longitudinal influence are relative, dynamic understand the behaviors usage’ and realized we were interested patterns—how have cultural, political, concepts, explore the drivers and of FATA communities relating in the following question: and technological changes impacted boundaries of trust. This should to information usage—a tall user behavior over time? Such a be done within communities, as How are FATA populations access- well as between communities and order, to be sure. focus can help anticipate future ing, sharing, evaluating, and creating shifts, and allow program designs a) institutions, Through a mapping exercise, information, and how might their that accommodate them. b) diverse information sources, we determined the key themes habits change over the next two and Given the pace of technological years? c) the media. that the study would focus on. change, practitioners must understand the forces driving change, not To the extent that effective Beyond current habits, we wanted only how that change is manifest interventions always, at some to understand the needs, capacities, today. level, rely on trust, understanding constraints, and sensitivities that the nature of trust in a particular influence the information-seeking Multiplicity of media. context is paramount. People’s usage of and relationship and media consumption behaviors to media is complex and diverse, of FATA communities. and research should examine diverse information sources Through a mapping exercise, we (formal and informal), as well determined three key themes that as the complex, ever-changing would drive the research, presented relationships people have to them. here as an excerpt from the Research Framework of the Internews Paki- stan Case Study.

22 Design Research for Media Development 23 Phase 1 // DEFINE // Research Framework DEFINING KEY Principles in Practice THEMES

Theme 1: Theme 2: Understanding Trust & Influence Impacts of Displacement

How do FATA communities evaluate not specifically oriented towards What has been the impact of con- how FATA populations access, evalu- the credibility and trustworthiness disseminating information but that flict and internal displacement on ate, and trust different information of different information sources may do so as a secondary activity the information-seeking and media sources and media channels. To the and media channels, and how do or naturally as a result their role in behaviors of FATA communities? extent possible, the research will these factors impact the influence communities (e.g. service providers map how these attitudes and behav- and reach of different source and such as schools or clinics where Since 2007, FATA has experienced iors change over people’s timeline of channels? people gather, or entrepreneurs that significant population migration as displacement. wield economic influence). There a result of government operations Both formal (e.g. newspapers) and will be a specific focus on emerging against non-state armed actors and informal (e.g. community leaders) sources of information. sectarian violence. The research sources of information will be seeks to understand how migration considered, as well as sources and displacement has impacted

Key questions include: Key questions include:

• What are the social norms around gathering, sharing, • How do displaced FATA populations access information? and assessing different types of information? How does this compare to communities that still reside within the FATA?

• What are the metrics FATA communities use to determine • How does the experience of migration and displacement, whether a particular piece of news or information—or a source— both conflict-related and seasonal or work-related, impact how FATA is credible, trustworthy, or worth passing on to their networks? populations evaluate the credibility and importance of different media sources? • Specifically, what are the uses and perceptions of secular versus religious information sources? • How does information pass to and from FATA residents and connections outside? • Based on the above metrics, along with reported and observed user experiences, can we map the relative influence and reach of different types of information sources among FATA communities, how information flows between them, and how these patterns have changed over time?

24 Design Research for Media Development 25 Phase 1 // DEFINE // Research Framework DEFINING KEY THEMES

STEP 2: Review Existing Research

Theme 3: Desktop research is a critical early As part of the literature review, Usage of New Media & Technology input to the design research process. include work in the specific geo- Literature reviews, expert consulta- graphic context as well as the sector As technology is vastly changing Beyond analyzing indicators such tions, and discussions with institutional (and review both case studies and the communications landscape in as technology penetration (e.g. partners ground the potential future evaluations). Pakistan, and will continue to do so, radio, television, or mobile), size of in the wisdom of what has been the research will have a particular audiences for particular programs, done before. In consultations, seek out three focus on perceptions of and behaviors dollars invested in specific channels, different types of experts: around new media and digital com- or other market metrics, the study Desk research sets the stage for 1) Operational experts, munications tools. will examine the human factors that design. With such an expansive impact technology behaviors and body of innovation and achievement 2) subject matter experts, and Given the potential of information choices. in the field of development, any work 3) institutional experts. and communications technologies should build on what has been done. (ICTs) to enable easier and broader access to information, the study will seek to understand specific technologies that have high potential to reach FATA communities, as well as challenges around and limitations Operational experts have worked extensively in a of ICTs. geographic context. Their insight will be especially key in planning the logistics of a research trip and developing a risk management plan. Key questions include: Subject Matter experts have implemented or evaluated • What are current behaviors around the use of technology to access, share, similar or complementary programs, and can share their create, and evaluate information? Can we extrapolate larger trends around opinions and understanding about how and why their technology usage beyond specific channels and media sources? research and program approaches succeeded (or fell short).

• How does literacy, in all its forms — textual, technological, and media — Institutional experts from organizations, donors, or other impact usage of technology among FATA populations? What are ways stakeholders can help focus goals and deliverables. They can in which FATA communities currently circumvent their own capacity help understand what outputs are most useful and feasible, constraints around technology to get and share the information they need? and how best to present them. • What are the threats to the use of technology, and how do FATA populations respond? Does the cultural appropriateness of technology vary with its intended purpose (e.g. mobile phones for communications versus mobile phones for listening to music or the news)?

26 Design Research for Media Development 27 Phase 1 // DEFINE // Research Framework Principles in Practice

Consulting Experts Before fieldwork, the team consulted with experts on key project themes and to gain better understanding of the operational Category Role Affiliation Profile context. After fieldwork, experts helped interpret, analyze, and validate research findings. This is redacted sample of those consulted. Subject Academic University DC-based Pakistan expert Matter Researcher with on-the-ground perspective, with focus on political stability and Islamic politics. Advise on research framing questions, Category Role Affiliation Profile and institutional interests.

Operational M&E Donor Well versed in operating M&E Subject Co-Founders Local ICT4D Islamabad-based technology Officer Organization programs in FATA, familiar Matter Firm and civic participation firm that with Internews. Advise on has deployed mobile-based operational and risk planning information services across and management. Pakistan. Advise on technology behaviors and service design. Operational Chief of International Extensive on-the-ground experi- Party Development ence in FATA, and with media- Subject M&E International Worked with Internews on research Contractor related programming. Advise on Matter Program Development and M&E in several countries. operations, logistics, and security. Advisor Contractor Advised on complementary and adjacent research initiatives to Operational/ CEO Local Works on audience research with draw from and feed into. Subject Research Internews in Pakistan. Extensive Matter Organization experience media behavior in the Institutional Senior Donor Deep understanding of media region. Advise on gaps in current Leadership Organization landscape in FATA, has managed research. Internews programming. Advised on donor priorities and interests Operational/ Various Staff Local Experts in freedom of information in the region and for the research. Subject Civil Society and expression issues in Pakistan, Matter Organization and the cultural and political fac- Institutional Various Across Advised on Internews’ needs and tors that influence the conversation. Internews Operational goals around research, perceived Advise on research questions, and Staff and Country opportunities and gaps in current the role of technology in shaping Teams research, program development, the information landscape. and M&E processes to help refine outputs. Subject Journalist Various news Pakistani journalist with extensive Matter outlets experience covering FATA and Given the aims of the research, local or locally based experts international news. Advised on research framing, questions, were prioritized over international ones. Institutional and international and managing sensitivities. perspectives still very much informed the research to ensure it would be useful to a range of stakeholders.

28 Design Research for Media Development 29 Phase 1 // DEFINE // Research Framework

STEP 3: Their values and needs will impact This list becomes your respondent program outcomes, thus their per- group, and each group’s needs and Identify the Stakeholders spectives must be understood and challenges will need to be addressed. accommodated, as appropriate, in While the map to it is not yet clear, Media development seeks to improve human livelihoods through information. the program design. the places where these many view- So, it is logical that humans are at the heart of good program design. They points and needs intersect is where both inform and inspire program designs, and their needs are the metrics by List all the stakeholders who the the most sustainable interventions which programs should be evaluated. program must consider. Starting will be found. And be realistic: There Yet there are many actors that impact the design, delivery, with the end-users, list all those who will be actors your program will not and ultimate effectiveness of a program. influence their behavior, as relevant have access to or influence over. to the program: What people and Still include them as variables to be These may include: institutions do they interact with? aware of, and to recognize the gaps Which other stakeholders have the in your own data. End-Users Community Leaders greatest impact on their actions? Do Target individuals and communities Individuals with both formal and this for all the stakeholders identified. Do not get intimidated. This process (often called “beneficiaries”) the informal distinctions as leaders can be overwhelming. Any map of activities and outputs of the within their community and that program are intended to benefit. have particular influence. Think through the perspective of the conflicting interests of dozens each listed stakeholder: How do of people and institutions will look Service Providers Contextual Influencers they impact the experience of the unsystematic and unsustainable. Individuals and organizations that Individuals or organizations that end-users? What are their priori- Getting through the mess is what have impact or control over the serve as functional touchpoints for ties and goals? How do they view the research is for. reaching end-users—not all may be environment or the individuals in which a program is targeting the other stakeholders on the list? formally recognized as information How can they contribute to program sources or service providers. These Policymakers objectives? may include radio stations, primary Creators of the environments in schools, or agricultural extension which a program exists through workers. legislation, rhetoric, or other Administrators forms of power. Functional managers that oversee Institutional Partners Predict users’ the actions of service providers. Institutional donors, private sector needs, not Know who to partners, or other organizations solutions. include when. that define, resource, and influence program design. Think broadly and fundamentally Naturally, stakeholders include about users’ needs. For example, funders, partners, colleagues, users may not need “a radio and others who are contributing station” or “better information”; to or otherwise directing the rather, they may need security, work. Include their perspectives stability, or connection with when planning the research, loved ones, all of which can be but once fieldwork begins, addressed in multiple ways. your allegiance is to end-users.

30 Design Research for Media Development 31 Phase 1 // DEFINE // Research Framework Principles in Practice

A Note on Sample Size

In the age of “Big Data”, it is hard to resist the idea that survey data, and lots of it, is better. It is true that quantitative data is a valuable tool for many aspects of program design: Large datasets can help identify sweeping trends across a community, or determine how human well-being has changed in a region over time. The strength of such analysis comes in part, however, by reducing nuance. Defining Target Respondents

To complement data analysis, design research is intrinsically about understanding complexity through human nuance. This means aiming for a focused and Pakistan has a vigorous and diverse media ecosystem, and manageable sample size. The goal is not statistical below are the stakeholder groups that our research targeted, representation; there is no need to touch every potential as well as the target percentage of the overall respondent type of stakeholder relevant to the program. Rather, the population for each group, based on our mapping exercise. goal is enough representativeness to draw meaningful insights on patterns of behavior and context. Early on, we agreed that the focus Further, researching information The most valuable insights often come from making would, first and foremost, be on distributors (5%) and regula- surprising thematic connections between seemingly information consumers (60%): FATA tors (5%) would help Internews disparate behaviors and cultural norms. These can only communities who seek information identify the opportunities and be generated by human evaluation of collected data. through diverse outlets. Under- challenges for information dis- Further, as the best analysts and program designers are standing their behaviors and atti- semination. Finally, media-related usually those that conducted the field research, their tudes—and their underlying ratio- organizations (5%) including NGOs, facility with the entire data set is important. As powerful nale—and capacities would allow watchdog groups, and others will as large data sets can be when run through NVivo or Atlas Internews to develop content and help guide the research process, TI, human creativity is also a powerful tool—and for it to programs that will reach and be both in data collection as well as work well, the capacities of human memory should be understood by these populations. analysis. On the following pages is considered when determining sample size. Understanding a broad range a condensed version of the study’s of information producers (25%), stakeholder analyses: both formal and informal, and the motivations, capacities, and self- conceptions of each actor would be critical to mapping information flows and networks of influence and trust among FATA communities.

32 Design Research for Media Development 33 Phase 1 // DEFINE // Research Framework DEFINING TARGET Principles in Practice RESPONDENTS

60% 25%

Information Consumers Information Producers

Examples of Entities Key Questions Examples of Entities Key Questions

• FATA populations in and outside of • How do they access, share, and • Journalists: local and foreign • What are their goals in participating FATA, a diverse sample in terms of assign value to different types in the information landscape, and • Bloggers gender,* age, economic background, of information? in producing information? education, location (both rural and • Traditional leadership • What is the role of information/news • How does each actor see its role, urban) in the context of life in FATA? (Does • Informal community leaders and how does it compare to other • Particular attention will be paid to it play role in creating change? How • Civil society and/or special interest actors in the ecosystem? FATA populations perceive informa- a) influencers, as their habits and groups (e.g. youth organizations, • What sort of training have they had tion/media? What types of informa- choices have outsize impact on NGOs, fundamentalist groups) to produce information, and what are tion do they want to access?) their larger communities • Religious media outlets (e.g. mullah the standards they hold themselves • What is their access to and usage of b) youth, as their behaviors can radio) and leaders (e.g. imams)* to? technology? suggest trends in information • Government officials • What is their sense of the security consumption and technology • How do they evaluate the credibility (e.g. Political Agents)* issues involved in their work? habits and trustworthiness of information How do they mitigate security risks? sources? • Government and state-controlled c) low-literacy populations, as ICTs media outlets (Pakistani and foreign)* • What other information sources have the potential to uniquely • For influencers: What is the do they draw upon and use? serve them relationship between these individu- • University journalism departments • What do they see as the biggest als and those they influence? How • Press clubs constraints, for themselves and do they gain and use their influence? their listeners/viewers? * Note: It may be difficult to access How does information and com- • Political parties and activists female populations. Little is known munication impact or facilitate their • Businesses and/or entrepreneurs • How do they evaluate their success about this demographic in the influence? as information producers? region, much less specific media • For youth: What are the tools they • For businesses: How do they and technology behaviors. * = May not be possible use to access and share information? approach marketing in FATA, and is it any different from how they • For low-literacy populations: advertise in the rest of the country? What are the ways in which they are accessing information, despite their lack of literacy?

34 Design Research for Media Development 35 Phase 1 // DEFINE // Research Framework DEFINING TARGET Principles in Practice RESPONDENTS

5% 5% 5%

Information Information Media-Related Distributors* Regulators/Controllers* Organizations*

Examples of Entities Examples of Entities Examples of Entities

• Media groups • Government agencies • NGOs and civil society groups (e.g. Jang, Dawn, Lakson) (e.g. FATA Secretariat, Pakistan (e.g. Rural Media Network Pakistan, Pakistan Centre for Development • Local media outlets Broadcasting Corporation, Pakistan Electronic Media Regulatory Authority) Communication) • International media outlets • Traditional leadership and tribal • Journalist unions • Television broadcasters elders • Crowdsourced content sites (e.g. Hosh Media) • Aggregated content sites (e.g. Teabreak)

Key Questions Key Questions Key Questions

• How do they determine the • What are their key goals and • What do they think are the key * For this study, it was determined content that they will include in concerns regarding to the production opportunities and challenges in that perspectives of information their outlet? How do they prioritize and dissemination of information? securing greater information distributors, media-related access and better quality of different pieces of content, and • How do they determine what types organizations, and information information for FATA communities? choose what to feature? of information can or cannot be regulators/controllers would • Do they track responses to the shared, and how they will be shared? • What are their thoughts on our largely be drawn from existing content they put out? If so, what research framing and plan, as well research and expert consultations. tends to be more/less popular? as our findings/analyses? Field research would largely target Do they have ideas as to why? information consumers and producers.

36 Design Research for Media Development 37 Phase 1 // DEFINE // Research Methods Research Methods

Many instruments are available to Naturally, practitioners must choose help researchers for getting a holistic instruments based on what is fea- view of a target community. sible in the environment. Consider The Value of Variety culture as well as security, timelines, Group interviews provide a broad and other challenges. Any research program worth its salt will use several but basic understanding of key different instruments. Multiple methods can help research themes, and surface Over time, the methods used may researchers triangulate data surfaced by different tools threads that require further probing. change, either to focus on an and address the inherent deficiencies within each tool, emerging theme or to overcome helping overcome research bias. Individual, in-context depth challenges (of bias, logistics, or interviews allow a deeper under- other) that crop up in the field. Often, Everyone carries biases, and different instruments surface standing of people’s frustrations, a discovery in the course of research different biases in both researchers and respondents. For vhopes, and day-to-day lives, especial- makes one or more instruments example, due to natural social dynamics, respondents ly when used over time. obsolete; for example, a service in a group interview may be eager to appear more pious, trial of a government information more hard-working, or otherwise impress each other with During a home stay, as the guest line is impossible if the phone line their answers; individual follow-up interviews may elicit of a respondent, researchers gain has been shut down for the past six more self-reflective answers. Similarly, during an in-depth greater empathy and insights into months. Governments and devel- interview, a respondent may be embarrassed to admit how people think and operate, as opment organizations do not always challenges they have experienced in using a service, for social barriers are broken down have updated information on the fear of appearing incompetent, but a service trial or an through sustained engagement. service availability, especially for observation of that same person using the service can rural areas, so plan accordingly and raise these frustrations loudly and clearly. Service trials yield first-hand expe- remain flexible. rience of how programs and services Using several instruments helps correlate findings, check work—and thus insights about the biases, and better understand the subtext of people’s relationship between institutions answers. and people—which is critical for effective program design. Keep it interesting. The common thread to all of these instruments is immersion. Design Always question whether a research tools open structured space research tool is the right tool for where researchers can discuss, the situation, and whether the observe, test, experience, and mix keeps things engaging for understand the mechanics and the team. Research should be emotions of daily life in the way that stimulating, and using a range respondents do. of tools help unlock creativity, which is critical for program design.

38 Design Research for Media Development 39 Phase 1 // DEFINE // Research Methods

Research Tools

GROUP INDIVIDUAL INTERVIEWS DEPTH INTERVIEWS Facilitated discussions with a Semi-structured conversations that group of respondents. Respondents probe into an individual’s attitudes may be grouped for any number of and thought processes. Best when reasons: Similar demographics or conducted in private and in a setting profiles, or diverse viewpoints. that respondent is comfortable with, such as their home or workplace.

Used To: Challenges: Used To: Challenges: • Gain a broad understanding of • Group environments will impact • Gain a deep and nuanced under- • Getting rich data is time consuming. a new issue area and uncover respondents’ responses. The desire standing of how a person thinks and The team may have to visit a respon- key themes to probe further. to appear in a certain light amongst behaves. dent several times to establish the trust required for them to speak • Understand the dynamics within ones peers can influence answers. • Learn about the environments in openly and honestly. a community, quickly profile • Certain individuals may dominate, which respondents live, work, and play. several individuals, and identify and if they have some kind of • Develop relationships with potential • People may not want to reveal their respondents to follow up with at formal or informal status within target users, with whom the team can true feelings or may simply misreport greater depth. the community, their presence may return to help design, prototype, and details for any number of reasons. Thus, it is important to validate • Establish shared research prevent others from speaking their test solutions. findings across several respondents. experiences and common minds. reference points among the team. Requires: • A private setting to minimize the Requires: influence of other people. Be • A facilitator who is socially • A significant amount of time, Keep it perceptive, firm, able to synthesize inclusive. at least one hour. small. quickly, and has the ability to spark and sustain discussion. • A skilled interviewer that has deep The value of group interviews di- familiarity with the research For individual interviews, minishes if there are more than 10 topics, and ideally an accompanying there should be ideally one or respondents. But when entering a note-taker/photographer. two, and certainly no more than new context, group interviews that three researchers, present. are open to all signals interest in all perspectives, and helps diffuse Any larger a group and it can tension when specific individuals become intimidating for the are chosen for interviews. respondent.

40 Design Research for Media Development 41 Phase 1 // DEFINE // Research Methods

OBSERVATION HOME STAYS Observing the activities in a Extended (and usually overnight) particular context, or the actions stays in the homes of your target of an individual or group of users, where interview and individuals, to gain insight into observation methods integrate the subtle forces that shape a with natural social interaction. place or a people.

Used To: Challenges: Used To: Challenges: • Gain personal, experiential under- • Time constraints often do not allow for • Deepen understanding of a target • Homestays can raise expectations standing of a context, community, long-term observation—only when the user group by observing the finer among respondents—after living for or individual helps build an intuition researcher starts getting bored do they details and nuances of their lives. several days (or even a single night) about that context; this then enables start noticing the minute contextual People tend to discard with pretenses with someone, the line between you to make better design decisions insights that are most revealing. and act more naturally the longer a researcher and respondent can blur. for that context. Yet project timelines rarely afford the guest spends with them. It is critical to address expectations • Identify otherwise unarticulated needs space for a team member to ‘grow • Achieve a more holistic understanding upfront, before the interaction begins. bored’. Thus, researchers must be and actions that may be subconscious of community dynamics. By living in a • There may be personal risks to the constant, vigilant observers that are to respondents themselves; what community for several days, 24 hours researcher: Depending on the con- attuned to, and documenting, the ebbs people think they do is often different a day, greater exposure to diverse texts, there may be health concerns and flows of the environment. from what they actually do. rituals and opportunities for engage- (e.g. hygiene, food preparation) and ment will present themselves. security challenges, so heed the advice of the local team. Requires: Requires: • Patience. Observation takes a long time. It is impossible to visit a hospital • A high degree of sensitivity to how for an hour and understand all that Try your presence may impact your hosts. happens in that environment, and why. shadowing. The presence of a guest can be a • Confidence. Observation requires financial and even emotional burden, researchers to trust their own instincts, Shadowing is much like Observa- in low-income communities. Thus, and to believe their own insights have tion—it requires immersion and the be sensitive to your hosts’ cues, and value. Such confidence grows with focused study of people. Unlike compensate for any inconvenience experience. Observation, Shadowing asks users you pose (e.g. bringing food as a gift) to be active participants in the or understanding when to abandon research process by “thinking out (e.g. if it seems that it is straining the loud”; that is, by articulating their family’s resources and energy). thoughts as they conduct actions. While an imperfect window into the human mind, it yields oft hard-to- get insights into factors that direct or influence behavior.

42 Design Research for Media Development 43 Phase 1 // DEFINE // Research Methods

SERVICE USER TRIAL DIARIES A personal trial of the services used Self-reported responses to a prompt by the respondents. Gain firsthand or set of questions, which can be experience of how the service recorded as written, photo, audio, functions, to inspire ideas for or video diaries. your own service or program.

Used To: Challenges: Used To: Challenges: • Develop an understanding of service • Service providers and frontline staff • Understand a user’s particular habit, • Incentivizing participants to maintain and program norms in a particular often get suspicious when a foreigner behavior, or thought process over time. the habit is difficult. Consider how context. How people currently do is taking notes or asking questions • Learning thoughts that users may to make the exercise fun rather than things points to how they might want about their service offering, and may not want to reveal through other cumbersome, and how to provide or expect to do things in the future, be hesitant to allow you to take notes. means, such as face-to-face small incentives to encourage but also how they might want things Ask your local team to conduct the interviews, but are comfortable sustained participation. to be different. service trial, and to then document writing down and sharing. • Not all respondents will understand • Examine different models for service their experience after. • See the world from the user’s eyes. what is required of them, in terms delivery and program implementation— By having them record their experi- of the content or the techniques for from communications and marketing, ences, emotions, thoughts, and ac- documentation. To mitigate this risk, to customer service, to grievance tivities, you get a sense of the world consider how to make the exercise redressal—to inform the design from a wholly new perspective. as simple as possible, and provide of a program. training and support. But ultimately, be prepared to receive unusable Requires: data. Some respondents will misinter- Requires: pret the prompt or tasks required. • A degree of technical capacity among • A keen eye for details. Each and users—whether that is the ability to every step of the service experience read and write, to use a camera or should be noted—the instructions and other types of records—or the ability communications provided, the efforts to pick up the required skill quickly. or materials required, the service agents, the actions taken, any confu- • Enthusiastic, committed participants. sion or frustration, the outcomes, all The quality of the data will depend observed and recorded in minute on how much respondents are willing detail. to invest. This exercise is particularly well-suited to young people, who often enjoy self-documentation.

44 Design Research for Media Development 45 Phase 1 // DEFINE // Research Methods

SURVEYING KEY INFORMANT A series of structured questions INTERVIEWS focused on key indicators— Interviews with individuals that demographics, casualties, usage have domain or firsthand expertise of particular tools—that may in areas you are interested in to impact our understanding of test hypothesis and design ideas. the research questions.

Used To: Challenges: Used To: Challenges: • Used to contextualize findings, • Survey interviews can become • Kick off the research preparation, • There can be a tendency to overem- determine patterns of behavior, and cumbersome to respondents – do and help inform or design the phasize the expertise of ‘experts’. Par- map the extent of research patterns your best to make sure you are not research framing, themes, questions, ticularly when timelines are short and through the research sample. robotic in asking survey questions, and approaches. questions are many, it is tempting to • Develop a basic understanding of or try weaving them throughout the • Test or validate ideas that have rely on their experience. Use them in key demographics or norms around interview. emerged through research, particu- a targeted way. They are not substi- a specific topic. Many areas where • The nature and structure of surveys larly early seeds or ideas for solutions. tutes for talking to the real experts: the end-users. development work occurs lack can make it hard to get nuanced • Understand contextual dynamics and reliable data about their populations: answers, sometimes leading to mis- constraints that the end-users may It is impossible to find disaggregated understandings. For example, for the not know about (e.g. regulations, data for the community level, or question “Do you use SMS?”, human resources in a particular area) certain communities may have been a respondent may answer “Yes” but which may enable or inhibit a Think beyond overlooked in larger surveys. because they receive SMS notifica- particular design idea. traditional tions from their mobile network ‘experts’. provider. Thus, be creative and Requires: resourceful in verifying survey Requires: responses to ensure both researcher Key informants can be anyone, • A light questionnaire, no more than • Forward planning. Depending on the and respondent take an answer to but prioritize those with hands- 20 questions, that can be used as a profiles you would like to interview, par- warm-up tool at the start of an inter- mean the same thing. on experience over pedigree. ticular institutional audiences, it may To design mobile information view, or at the end of an interview to take weeks to set up an interview, wind down. systems, key informants may and it is worth weighing whether or include: Mobile network opera- • Thoughtful questions that will add con- not it seems worth the time. tors and software developers text to research themes—do not collect • An understanding of the backgrounds (to assess operational feasibility), data for the sake of collecting data. and potential biases of each key infor- government officials (to under- mant. Some may want to talk to you stand regulatory issues), mobile simply to be helpful, other may have phone repairmen (to understand Respondent Indexes institutional or personal agendas they mobile consumer behavior), and want to advance by participating in bloggers and prolific Tweeters Respondent Indexes are spreadsheets that capture survey results. your research. (to understand the changing They are a helpful check in design research because they can help information landscape). identify broad trends within a sample or the validity of a specific finding, which can inform or constrain program design.

46 Design Research for Media Development 47 Phase 1 // DEFINE // Research Methods Principles in Practice

ARTIFACT COLLECTION Collect materials from a particular environment to understand the artifacts that context produces.

Using Research Tools

Used To: Challenges: • Understand people through how they • It is not possible to collect every conceive of, design, distribute, and artifact the research team wants to. In FATA, while we used many were far richer and more diverse use different tools and materials. To the extent possible, ‘take’ an of the research tools highlighted than at the nightly hujra, a traditional • Derive inspiration and guidance— artifact by capturing it via photog- gathering of men. The barbershop, raphy. From all angles, at the most above, individual depth inter- artifacts help understand what forms, views, observation and service he realized, was a place for both messages, and other decisions are minute details. trials proved most useful for our the rich and the poor, and where understood by or resonate with a men helped each other—whether investigation. population. educated or not—learn about the day’s news. Requires: Individual depth interviews formed the majority of the respondent inter- • A very large rucksack. Researchers also tried many actions due to the sensitivity of the different services, from government research questions. Respondents ID registration to health clinics to were understandably hesitant to banking services. One service trial speak about religion, politics, and of a telephone political engagement technology in public. Most inter- platform from the Pakistan Tehreeke views, therefore, were conducted in Insaf (PTI) party stood out to the private and one-on-one, and record- team. The platform allows people ing was not always an option. to pledge membership to PTI. After registering, recruits immediately Research is not always about asking receive a phone call with an auto- questions though. In FATA, researchers mated recording from PTI Chairman enjoyed observation and shadowing and former cricket legend Imran activities, despite initial skepticism: Khan expressing gratitude for your “But people will think we are from support in his prime ministerial the [Pakistani intelligence agency]!” candidacy. From the ease of regis- One afternoon a researcher spent tration, to the sequencing and con- at a barbershop in Dera Ismail Khan tent of messages, all are instructive proved especially illuminating. He for communications campaigns in observed that the conversations the region.

48 Design Research for Media Development 49 Phase 2 // PLAN

PHASE 2: Plan the Plan Investigation

Every project has issues that team. Presenting these plans in a are unique to its objectives single place and an understandable and context. Plan an investiga- way creates a springboard for diving tion that accommodates for—or, into fieldwork. better yet, takes advantage of— This chapter also discusses respon- these factors. It will yield better, sible community engagement. Any more honest insights, which development project can have are the foundations of a good potentially negative impacts on the program. community or context it seeks to serve. Self-awareness and proactive After defining the research chal- management of this risk is critical lenge and a rich set of supporting throughout the process. questions, it is time to determine the methods and plans to get those Ultimately, designing an investiga- questions answered. But beyond tion is fantastic training for program research tools, it covers the practi- design. Think of it as a trial run, and cal aspects of your fieldwork: A field be as serious and committed as you travel plan, risk management, and would be for ‘the real thing’. the composition of your research

50 51 Phase 2 // PLAN // Operational Research Plan Operational Research Plan

Once you know what questions to Sometimes, several of these factors, Team ask, it is a matter of determining the such as target locations or team composition nuts and bolts of how to get those composition, may have been prede- should inform questions answered. Where are you termined by the project or institution- location going to go? When will you go? Who al priorities. But whenever possible, selection, and will you take? push to make sure the operational vice versa. details are just right—they can make or break a study. Usually one factor is more rigid, so plan accordingly. For example, if a project requires certain locations, recruit suitable profiles for those locations. Similarly, if there are individuals that must be on the team, assess STEP 1: locations in terms of risk to those individuals and adjust as Determine Locations necessary.

A few principles to selecting research sites: Potential for impact. Ability to map connections. Team security. Time requirements. Focus efforts in areas where It is important to interact with When selecting research locations, The amount of time spent in each your institutional expertise and citizens from a range of locations consider potential impact on team location will largely depend on resources can make greatest in a particular region. Conducting security. Team members’ race, deadlines, budgets, and staff avail- impact. That is not to say a research across diverse areas tribal affiliation, gender, and other ability, but always spend at least study should categorically avoid helps compare different features characteristics may impact their one week in a given focus area, areas where resources may have between them, and examine the ability to safely operate in different meaning a particular city, a rural historically been concentrated. connections and relationships locations. region, or other relevant definition. Any less and it is difficult to under- There may be good reasons between populations in a region. (For more on risk management, stand the rhythm of a place. to do so—such as leveraging see page 56). infrastructure in an urban area A full week (or more) helps the to test a pilot, or scaling success- team build trust and credibility, and ful programs—but consider whether provides opportunities to follow-up there are traditionally neglected with the same respondents when areas where you can have strong new questions arise. Often, people impact. are more open and honest in a second or third conversation.

52 Design Research for Media Development 53 Phase 2 // PLAN // Operational Research Plan Phase 2 // PLAN // Operational Research Plan

STEP 2: Choosing the Right Locations Organize Field Logistics

Due to the immense deprivation in the region, FATA allowed Design is grounded in immersion A few simple tips: and empathy, and it is impossible significant opportunities for positive change. But choosing Accommodations to achieve those while riding in the right locations for research was critical to success. SUVs, sleeping in four-star hotels, or Rent a house, preferably in a residential neighborhood, or opt sipping lattes in the capital. In field for a local guesthouse. Have core operations, do as the local do where and local team members stay to- possible—ride motorbikes, sleep in gether to minimize the distance Our investigation covered rural and connections within the village, even guesthouses, and become immerse between international and local staff. remote areas of Pakistan, which if they were from a nearby town— in the patterns of local life. A few are more information-deprived, and researchers were only deployed simple tips: Travel where ICTs have greater opportunity to areas where they had extended Use local transportation to experience to make an impact, but also urban social networks. They used snowball life as your users do. While project timelines or team security may not locations to be able to compare find- sampling to identify respondents. allow local transportation everywhere, ings between locations. Urban areas, This ensured more honest responses, use them selectively. When hiring due to factors such as the concentra- and the security of our researchers. vehicles, select drivers carefully and tion of resources and early technol- Consider a cars that do not draw attention—both ogy adopters, also surfaced key infor- are critical to operational security. satellite phone. mation and media behaviors that are Each researcher or team of research- likely to spread across the region in ers was assigned to a certain city Communications the next few years. Finally, the inves- or region for two weeks. Even those When working in particularly An early key task is to ensure tigation covered both locations in assigned to a larger geographic area remote areas, consider investing the entire team has local mobile numbers, and phones that work in FATA and KP to examine the impacts returned to the same towns or villag- in a satellite phone for the team. Having an available phone line even the context. Each phone should have of migration and displacement on es multiple times. FATA’s challenges have made its people more guarded. when there are no mobile networks all other team member’s numbers FATA populations and their informa- saved, as well as any emergency Returning to the same respondents can be invaluable, even if only for tion habits. peace of mind. numbers. Depending on the project, allowed researchers to build trust locations, and other obligations for and rapport, ask follow-up ques- the team, get USB modems (internet As there was some operational tions that surfaced through nightly dongles) for the team. flexibility, field locations were select- synthesis, and better understand the ed based on researchers’ back- dynamics within a particular place. grounds. Given the sensitivity of the research questions and the tensions in the region—particularly in small villages, residents were highly suspicious of outsiders that had no

54 Design Research for Media Development 55 Phase 2 // PLAN // Operational Research Plan

STEP 3: Anticipate and Mitigate Risks

Fieldwork comes with risks, thus there must be a realistic risk management Determine, where possible, the likely causes and likelihood of risk and the plan for the health and safety of the team, and to make the most of time on potential impacts. For each identified risk, assign an owner that will work the ground. Here are a few things to consider when planning. to monitor that risk factor and mitigate its occurrence. When challenges do arise, the owner is also responsible for liaising with other operational stakeholders to manage its negative impact. Project Logistics Security From visa applications to The safety of a project team and government approvals to other the communities they engage organizational factors, there are with is paramount. Here, both many logistical issues that may individuals’ physical and mental impede project progress. well-being and the security of collected information and data Human Resources Risk are critical components. Security Management Issues include obvious risks that is relative for each situation and Keep a Plan are easier to manage, such as individual, and changes in the low profile. the number or quality of team security situation can be swift, members. Issues may also be thus requires constant and vigilant Avoiding unnecessary attention is From the Internews Pakistan Case less obvious and more difficult attention. key to risk management. Consider Study (available at www.design. to anticipate or manage, such as Research Challenges how choices in accommodations, internews.org/tools), this plan shows team members’ ability to cope vehicles, dress, and even dining how to develop an operational frame- with the stress and pressure of Many factors can impact the spots may be interpreted by various work for anticipating and mitigating intense fieldwork, and to deal with feasibility, quality, or completeness actors in the field context. Where project risks, and discusses some of sensitive and difficult conditions. of the research. Some of this risk possible, be discreet and keep a the challenges anticipated and faced can be mitigated through planning low profile. in FATA. Field Logistics and training, others simply require Risks here are many, and not close monitoring throughout the always predictable. From poor fieldwork. connectivity to unexpected operational hurdles, fieldwork always presents a range of issues that must be dealt with as they arise.

56 Design Research for Media Development 57 Phase 2 // PLAN // Research Team Research Team

Choosing and preparing a team for Managing Uncertainty fieldwork is a critical process. The research framework and the opera- Ensure tional planning should help establish continuity Due to the security and operational concerns in FATA, the size of the team and needed in process. strict precautions were taken to ensure no members skills, talents, and traits. of the team were placed at risk. It is imperative that at least one Create a short job description for of the program design team is each role to be filled—translators, also on the field research team. A highly experienced field manager, on the security situation in each of project managers, and technical Rizwanullah, was recruited for the their assigned areas. leads—whether they will be internally Too much gets lots in knowledge transfer, and there is no substitute project to support researchers, coor- filled, externally recruited, or one for the inspiration of immersion. dinate with Internews’ security team Communications in general present- person may play several roles. It is and the project leads, and to trouble- ed challenges for the team. Given the important to have clarity on all roles, shoot as necessary. Fortunately, with sensitivity of the region, researchers and how they fit together. the exception of some brief stints of and the project team determined illness, the project risks identified operating protocol that included: Most teams will consist of a core set of members drawing from an organi- did not materialize. a) all phone numbers of foreign zation’s, staff, partners, or clients. and/or female colleagues were These individuals will be supported But no amount of planning can saved as local and male names; prevent the unexpected. During by a local team that is native to the fieldwork, protests erupted across b) to reach field researchers, program context. Pakistan in response to an anti- Reboot would send a text mes- Islam film that had been posted sage and wait for the researcher on YouTube. The government shut to call once they were in a pri- down mobile networks in 15 cities vate or otherwise safe area—it for one day to prevent protestors could be dangerous to speak from organizing. Without mobile English in public; and signal in Islamabad, the field manager c) SMS messages written or re- instructed all staff to stay at home. ceived in English were to be Some researchers were resistant, immediately deleted from citing the need to meet research tar- researchers’ mobile phones. gets, but the field manager insisted. Many others were developed to Once mobile networks were restored, accommodate for the context, researchers were able to coordinate which required extra patience with the project leads, and collabora- and effort from all team members. tively revise plans for the next day’s research and synthesis plans, based

58 Design Research for Media Development 59 Phase 2 // PLAN // Research Team

STEP 1: STEP 2: Identify the Core Team Recruit a Local Team

Internally, making the decision To that end, be sure to have a con- A project lives and dies by its local intelligence. People who can blend about who to staff on the project versation with core team members team. Local researchers are vital for in, and who are naturally empathetic team, whether staff, stakeholders, about expectations, potential chal- linguistic translation, relationship and easy conversationalists are clients, donors, or consultants, can lenges and risks, and what they facilitation, cultural interpretation, indispensable for design research, be challenging. should expect. Interview them and and program design. A project simply when earning the trust of respon- be critical: Ask why they want to will not happen without them, and in dents is half the battle. Often, institutional dynamics and join the fieldwork, what they think many scenarios, the research team constraints will impact this decision they will bring to it, what challenges will be following their lead. You need Of course, ethnicity, tribe, language, process. But be careful of how many they anticipate, and how they think to have people you can trust, and religion, gender, and other factors team members are experts in the those will impact research. Look for who trust you. that may influence a researcher’s focus topic or region, as well as how colleagues who are thorough and ability to interact with target respon- many hail from that region. They may thoughtful enough to be a benefit in Invest heavily in recruitment. It will dents must also be considered. In have strong personal opinions that, the field. Also consider the person- pay off. certain regions, having a team mem- depending on their role and person- alities of core team members, and ber from the wrong tribe can make ality, can bias the entire team and the dynamics between the organiza- How to recruit a local team? Look it impossible to engage respon- research. tions and interests they represent. for the same thoughtfulness, com- dents. In others, it will be impossible The team needs to be able to get mitment to work, and understand- for male researchers to interview Soft skills to look for. While the along even after an 18-hour day, and ing of goals as with the core team. female respondents. technical skills you will need will vary to honestly push back on each other. Look for soft skills and emotional by project, you want to generally look for people who are naturally curious, Ask potential members how they culturally sensitive, and who are think design research will impact both well-traveled and can travel the program, or others they may be well. It is also important that people working on. Some may think of it are ready and willing to work hard. just as a standard research or field Field days are not like office days. trip—look for those who understand Days can start as early as dawn and the value of the approach, and the end as late as sunrise, and sleep larger implications of the work. Keep it Team up. deprivation is often the norm. Col- small. leagues need to understand the commitment before jumping on As a rule of thumb, plan for the Plan for one local team member for board. smallest team possible. This will each core member. This enhances keep the team agile and flexible. overall effectiveness and efficiency, A large team can change the as it allows the team to break into dynamics of engagement—you will independently functioning pairs and end up looking like a tour group. cover more ground.

60 Design Research for Media Development 61 Phase 2 // PLAN // Research Team Principles in Practice

Local Team Profiles

Here are a few standard profiles that can be valuable to staff on a local team. Not all will be necessary, or at all phases. Consider the various stages of research, and recruit and sequence accordingly. Institutional fixer. Recruiting in Pakistan Most programs require consultation and coordination with institutional and/or government stakeholders. Look for a team member with networks among key institutions, and who can get an audience with the right actors. The characteristics and skills required on a team will change University students from relevant fields. for each project. Here is a sample of what qualifications we sought Students or recent graduates who are keen to bring when recruiting for our investigation in FATA. The soft skills an applied understanding to their training can be great were determined through a series of interviews (two to three per researchers. Look for process expertise (e.g. anthropology researcher, plus reference checks). or sociology majors) over technical (e.g. communications or journalism majors). • Must be a Pakistani national, • Must be computer-literate and Early-career journalists. with deep personal familiarity proficient in the use of word Journalists usually have a nose for stories, and know how with the FATA and/or KP regions. processing and Internet software to draw them out of people. Their curiosity and listening skills are valuable assets. Most are street-smart, and they • Must have interest and ability applications. know how to get a team into desired situations and out of to travel within the FATA and KP. • A background in the social hairy ones. Early-career journalists are key, as seasoned • Must be fluent in both Pashtun sciences is strongly preferred. ones have a greater tendency to introduce their biases or Prior experience with ethnographic past experience into the work. and Urdu, and speak and write English at an advanced level. or other types of qualitative Local facilitator. research is also preferred. • Must have the ability to operate in A team member from a particular geographic area can a wide variety of social contexts, • Knowledge of or practical experi- be valuable, and allow the research team to leverage their and with respondents from ence in media, communications relationships and trusted status within communities it or other related fields is strongly enters. diverse socioeconomic backgrounds, particularly low-income preferred, but not required for communities. the right candidate. • Must be sensitive to and respectful • Must be available from day before of a wide array of political and training begins to one week after religious beliefs and customs. field research concludes.

62 Design Research for Media Development 63 Phase 2 // PLAN // Research Team Principles in Practice

Anatomy of a Star Researcher

For our research in FATA, we were hiring to fill important and specific skills and attributes in our local team.

We were lucky to find researchers One of our best was a design homosexuality in Pakistan, and was Qaiser, like many of our best who had both the hard and soft researcher named Qaiser who clearly comfortable treating fragile researchers, felt deeply a sense of skills and the personal investment ticked all the boxes: He had a back- topics with skill and sensitivity. injustice about how communities that make great researchers. All 10 ground in English Literature and are misrepresented (or excluded) researchers recruited for the study Political Science, so we were glad to Importantly, Qaiser was personally in mainstream narratives. His drive were Pashtun, as this tribe almost have his language skills and geopo- invested in the research, and wanted for the hard work and getting it right exclusively dominates the FATA litical understanding. He was from to see positive change in his com- was personal, and that showed up region. Given restrictions on women’s FATA, but had worked with several munity. This is a common trait of all in his outputs and analysis. mobility and ability to appear in public, NGOs and traveled outside of Paki- our best researchers. While there is one female researcher worked in the stan; he both knew the local customs an obvious danger of bias, that is a urban center of Peshawar to capture and had an analytical eye born from risk we mitigate in every researcher women’s perspectives, although his exposure to other cultures and (including ourselves), and it is far this gap in data is recognized in the viewpoints. outweighed by the positive impact research. While several researchers of personal motivation. To attend lived in cities and preferred Western But what really sold us about Qaiser our final synthesis workshop, Qaiser dress, all wore traditional Pakistani was his obvious passion for research. turned down a rare and prestigious dress throughout the fieldwork, and In his first interview, he spoke ener- opportunity to travel internationally several grew out their facial hair, to getically about his experience with a civil society group he had comply with regional norms, and to researching difficult topics, including worked with—a testament to his put respondents at ease. commitment to our work.

64 Design Research for Media Development 65 Phase 3 // COLLECT

Collect the Data

At the heart of design research operational constructs that enable is immersion: Entering as fully strong field research out of the way, phase 3: as possible into a context to bet- this chapter will cover the basics ter understand its people and of good field data collection and, their environment. Immersion particularly, the most basic tool of design research: The interview. builds empathy, and empathy is foundational to effective pro- There are dozens of methods and Collect gram design. instruments available—each study will use several for triangulation and Primary data collection is how the validation—but the in-depth interview team identifies the human needs, is vital to most investigations. aspirations, motivations, and The skills that make a good interview constraints that programs must are core to many different aspects design for, and the institutional of fieldwork. capacities that will shape the program and enable its success. Finally, this chapter will conclude with how to document the research expe- The research framework and plans rience—through notes, recordings, covered in the previous few chapters and photographs—to optimize their pave the way for effective field utility for the design process to research. With the analytical and follow.

66 67 Phase 3 // COLLECT // Fieldwork Reminders Final Fieldwork Preparations

Research framework completed? Check. Team recruited and STEP 1: trained? Check. Travel and accommodation booked? Check and Remember Responsible Engagement check. As a team prepares to begin field research, there are several practical activities that can make the fieldwork easier. Every project has the potential to cause or exacerbate tensions in the communities in which it operates. It is critical that teams Get approvals. Join the conversation. understand the dynamics of each context they work in. Beyond It is often necessary to get approval Prior to research, get immersed enabling more accurate results, such understanding helps of research activities from govern- in the conversations in the region. ment authorities or community Start reading local media, from ensure projects do not spark local conflicts that teams them- leaders. Investigate whether this official outlets to local blogs to selves may or may not see. is required in your research context, personal Twitter feeds, to gain a and plan accordingly. sense of what people are talking Some tips to keep in mind: about and their daily concerns. Talk to experts. Actively manage expectations. Critically evaluate Ask about local politics and Develop basic language skills. the respondent sample. dynamics during conversations with Being able to exchange at least In all community interactions, make operational experts and colleagues a few pleasantries in the local it very clear what the project will Within any community, individuals who have worked in the region. language is invaluable for both and will not do. For many, inter- that are more powerful or articu- Seek tips about how to introduce understanding and being welcomed national organizations represent late, or are easier to access, may and manage your team and work, into a community. At the very least, resources. Jockeying for access to dominate a team’s attention, limit- and any rules of engagement, both learn: Hello; my name is; how are a project team may cause conflict ing its understanding of a larger formal and informal, to know about. you; excuse me; sorry; thank you; among communities. As a design community. The team may also be and goodbye. It is smart to practice research study may not lead to biased to certain respondent pro- a few other phrases, as demon- direct impacts on the community files. Be vigilant in examining how, strating effort goes a long way. where the research is conducted, and with who, time is being spent, this point must be emphasized in and ensure that diverse perspec- community interactions. tives are being represented.

Carry this mindset through the Plan several introductions. fieldwork as well. Plan to arrive The affiliations attached to any several days before fieldwork team or project greatly impact on begins to acclimatize, straighten out Read a book. how that team or project is received. Prepare several options for communications, set up the local introducing the work, in consulta- research team in-person, and take tion with the local team, who will care of other details. Once fieldwork You have read all the white papers, know best. Always be honest, but begins, all focus should be on the surveys, and books about the be careful to avoid any framings or research. history or political economy of the descriptions that might jeopardize region in which you will be working. your team or your work. Now pick up some good fiction. The nuance of a novelist’s detail can be priceless.

68 Design Research for Media Development 69 Phase 3 // COLLECT // Fieldwork Reminders

STEP 2: Prepare Question Guide and Scripts

Question guides should be developed If a team is eager to rely on a question with the research team, after all guide, they will likely let the guide team members have been suffi- intrude on a natural interview. To ciently onboarded to the project and avoid robotic, by-the-book interviews, trained in the research methods. spend more time discussing key Collaboratively building the guides themes with the team instead of can cement project themes in team giving detailed direction. Give them Engaging Responsibly members’ minds, and practicing a few topline questions, and then with the guides helps them warm up discuss how specifically to get for fieldwork. those questions answered. This will engage them in the process For the work in FATA, how researchers engaged Question guides are used to focus more deeply, and help imprint the communities was important for managing and guide interviews, but they should questions into their minds. not be overly rigid. For researchers expectations and avoiding conflict. accustomed to asking pre-formulated With a more experienced team, writing lists of questions, the transition detailed guides together may be a to facilitating discursive conver- powerful brainstorm exercise and a Depending on the particular village or the work. Some respondents who sations can be uncomfortable. In useful tool for interview preparation. respondent, researchers introduced had hoped for more direct benefits design research, the interviewer is But, the team should be comfortable themselves as from a media from the researchers were not an active participant, a detective changing or even ignoring the guide organization, a research organization, satisfied by this response and did seeking clues that will unlock the when the context calls for it. or an NGO—the latter was rare, given not participate. While this led to next piece of the conversation—and suspicions of NGO activity in FATA disappointment from communities the next insight. and security concerns for the team. and discomfort for the local team, it If pushed for a particular name, is far preferable to raising expecta- The type and level of detail in each researchers assessed the situation tions that would never be met. Role-play with guide will depend on team members’ and provided Reboot or Internews, the question backgrounds and previous experi- both of which were true, but each of Some of our researchers appreciated guides. ences. It is counterintuitive, but which would impact their reception the access the project gave them there is an inverse relationship differently. to influential people they may not Have the research team interview between how comprehensive and each other about their lives using otherwise have license or reason to detailed the question guides should the question guides. Having to Respondents were told that the speak to. By careful, daily monitoring be and how comprehensive and answer the questions honestly will purpose of the research was to of respondent profiles, project leads detailed a team may want them to be. help everyone think through how help organizations in the region were able to identify this bias early sensitive, relevant, and probing better understand the lives of FATA on and redirect researchers’ efforts the questions are, and help communities. There was no mention and attention. identify gaps. of projects that may come out of

70 Design Research for Media Development 71 Step 3 // COLLECT // The Art of the Interview The Art of the Interview

Drafting a comprehensive question guide is elusive. Design research offers dozens Where the research is going to take you is never certain, of methods and instruments for but there are a few principles: collecting data. Good practice demands the use of several different Think ahead. Emphasize open-ended questions. tools, but this section will dive deep Include suggestions for sub-topics or nuggets to probe within into the most fundamental of them each question. Offer suggestions for follow-up questions. all: The qualitative interview. Always prepare for each interview context and be prepared with ways Craft questions that draw out stories. The interview is foundational to sev- to make respondents feel more at Or, ask respondents to provide specific examples that show, eral methods, including home stays, ease: Bring a small gift if you are not just tell, you their feelings in response to a particularly service trials, and group interviews. entering someone’s home. Dress respectfully and carefully: Too for- topic. Here, we will also use the interview mal, and you may make them feel Use examples in questions. to illustrate core principles of design awkward; too casual, they may feel Include current events or happenings to ask about, which research. disrespected. will ground conversations and make them more concrete and tangible. (Be careful, however, not to provide examples Henry Ford famously said, “If I had as possible responses!) asked my customers what they wanted, they would have asked for Start broad, then refine. a faster horse.” This is a key point: Do not close off options before exploring the possibilities People do not always know the best around a theme. As key threads emerge through research, solutions to meet their needs, so do the question guides, too, will change and become more not expect it of them. focused.

The interview is not designed to The team should read the questions for sensitivity and generate solutions; rather, its pur- language, and then translate the questions into the local pose is to generate nuanced data language together, to develop a shared understanding of about people’s lives through per- how to communicate themes and topics. This is particularly sonal stories and experences. Such important when technical vocabulary is involved, or when stories reveal key insights about language ability is varied. In these scenarios, budget behaviors, uses of technology, com- ample time for the team to practice and build confidence. munity dynamics and other factors that inform program design. For the researcher to effectively gather this rich human data, a good interview should have a semi-structured arc.

72 Design Research for Media Development 73 Step 3 // COLLECT // The Art of the Interview

STEP 1: Introduce the Project

Informed Consent Spend the first part of the interview introducing the process and goals Obtaining informed consent is a researcher’s ethical of the session, and getting informed “Be equal” and responsibility, ensure each team member understands consent from respondents: Who are “Do as they do”. what it means and the process of obtaining it. you, what do you want to ask about, why you want to ask these questions, To get consent, first explain clearly how collected and how long the session will take. Try to be “as equal as possible” responses and data will be used—the goals of the project with respondents—from style of and the channels the data may appear in. It is also Let respondents know that their dress to the model of mobile helpful to provide conditions and examples about how information will be protected and phone used during fieldwork— and “do as they do”. it will not be used. For example, for most projects, it is anonymized, and describe the ways unlikely that any piece of information will be linked back in which the information they provide will be used. (For more on protecting Mirroring body language, actions, to a respondent. This is an important pledge, so ensure a and language goes a long way process is in place for removing inappropriate or sensitive respondent information, see Taking towards putting people at ease. material before it enters the pool of project data. Good Notes on page 82.) Actions such as sitting on the floor when they do or eating if they eat In some contexts, even if the respondent has given Give respondents a chance to ask help bridge the very obvious consent, he or she may not always understand the full questions, but maintain an open distances between researchers project context, or its implications. In these cases, the questions policy throughout the and respondents. team should “think for” the respondent and do this is a interview. Respondents have the “moral check” at the end of each interview: Put yourself right to change their mind at any in the respondents’ shoes and make sure usage of their time, so give them an opportunity to information—words or photos—will not jeopardize their revoke their consent—and delete all safety or security, or their dignity. It does not matter if their data. Bring a translated consent respondents will never see the research report or findings; form with you, as well as a copy for treating them with respect is not just the right thing to do, them. it is core to developing the empathy necessary to create a good development program. Ask for permission to begin.

74 Design Research for Media Development 75 Step 3 // COLLECT // The Art of the Interview

STEP 2: Build Rapport

Developing rapport with respondents Types of Questions is critical to any interview. Start with small talks and some general, There are several types of questions that can be used— open-ended questions about the Be human. or should be avoided—in an interview. This covers the respondents’ lives, questions that basic ones, and their utility. are easy for them to answer: How was their day? How many children Offer information about yourself, Open-Ended Questions. do they have? Where are their as appropriate: The interview Questions that encourage a full, meaningful answer using children? should feel like a dialogue, not the respondent’s own knowledge and attitudes. Helpful for an interrogation, and sharing exploring broad themes with respondents. (e.g. “How do Once a respondent is warmed personal details and finding you get information about events in your community?”) up, begin by asking open-ended points of commonality can help questions that are broadly relevant put respondents at ease. Follow-Up Questions. to the research themes. The start of Questions that solicit for details about a previous response. an interview is when the researcher Helpful for gaining specific examples about statements made, or clarifying confusing and/or contradictory collects the ‘ingredients’ that will responses. (e.g. “You said earlier that you watch TV to feed the rest of the session. Let the get information about your community, but I do not see a respondent drive this process. What TV in the house. Can you tell me where you watch TV?”) do they want to talk about, what are they passionate about, on what Leading Questions. questions do they divert their eyes Questions that suggest an answer or contain information and seem uncomfortable? the researcher may want confirmed—the bias is often subtle. Generally avoided. (e.g. “Satellite TV channels seem to have greater independence from government You will get the best interview if you control. How do you feel about using these channels for follow a respondent’s lead, so start news, compared to government channels?”) by feeling around for the topics that ‘open up’ each respondent and Close-Ended Questions. take good notes on the threads and Questions where the answers are offered, limiting the specific stories to ask about. This respondent’s answers. Generally avoided, but can be will provide the fodder for the rest of helpful for forcing a break in conversation and redirecting an interview. (e.g. “Do you watch television?” or “What TV the interaction. channels do you watch: Al Jazeera, CNN, or Geo TV?”)

76 Design Research for Media Development 77 Step 3 // COLLECT // The Art of the Interview

STEP 3: STEP 4: Dive Deep Wrap Up

Once a respondent has warmed up, Towards the end of the interview, At the end of an interview, give the deep research begins. Based on and depending on the dynamics, it respondents ample opportunity to what the respondent has said about may be possible to ask about more ask questions. Help them review any his or her experiences, select a few sensitive topics, or to confirm or photos taken during the session and Be friendly key topics to zero in on as relevant challenge findings from previous to delete any they do not like. (More but neutral. to the research questions. interviews or even that particular on this on page 85.) session. Did the respondent contra- Maintaining a neutral demeanor Each interview is different, but the dict him or herself, or something that If the study involves surveying, will help respondents feel more best ones draw people to tell detailed was previously learned? If adequate check if the questionnaire elements comfortable opening up or stories about their lives and experi- trust has been established, ask the were fully covered; if they were not, discussing sensitive topics. ences. If a respondent is speaking tough questions. ask the remaining questions. Listen, do not judge. Be engaged, in general terms, ask follow-up and show emotion when appropriate, but do not overdo it. Be a questions that prompt specific End with a sincere thank you. If app- sympathetic counselor, not a ropriate, depending on the context examples. Actions and stories reveal best friend. more about what people actually and institutional affiliation, provide do, rather than what they think they a thank-you note with the team’s should do. contact information. If it was a par- Ask the ticularly insightful or honest inter- follow-up. Do not be afraid to follow unexpected view, encourage the respondent paths or turns in the conversation to get in touch if they have further that may end up revealing unexpect- Respondents will rarely reveal all thoughts, or any questions. ed aspects of the respondent’s expe- that they have to offer in response riences. But avoid letting the inter- to a particular questions. So ask view turn into a one-track complaint follow-up questions: Who, what, where, how and, most importantly, session. A foreign researcher may be why. They show that you are perceived to have access to resources engaged and interested, and they that can “solve all the problems,” often lead to the most revealing and respondents may then focus on information. or hyperbolize difficulties. Remain firm and analytical, and steer the conversation back to broader life experiences, not just solutions they propose for your project.

78 Design Research for Media Development 79 Step 3 // COLLECT // The Art of the Interview

Institutional Interviews

Good design research involves end-user or community research, and institutional research as well. Given the complex and multi-stakeholder nature of any development project, it is critical to understand the needs, capacities, Keeping Open-Ended Questions Open and constraints of institutional stakeholders—especially implementing partners or others that may play a significant role in program implementation.

Institutional interviews, however, can be notoriously We were interested in finding effective ways for disseminating tricky. There are many reasons why institutional actors may choose to provide partial or inaccurate information information to or collecting information from FATA communities. to a research team. These range from misunderstanding The insight from true open-ended questions was crucial. of the project goals, to the desire to secure funding and support, to opposition to the program objectives. Researchers must be prepared to bring a journalistic rigor In our project, questions included: and bias the respondents. Illiterate to institutional research. A few tips on how to do so: “How do you get information about respondents may nod in response your community? And about the to prompts, and say they read the world? How do you communicate newspaper. People that receive Be prepared. Review any background documents you with family and friends?” When faced automated alerts from their mobile can find about the organization or its activities. Practice with such open-ended questions, service provider may agree that they for difficult interviews with teammates. respondents would often look at do indeed use SMS. Be firm but fair. Be prepared to question responses us blankly. There would be a long provided by institutional interviews if they contradict your silence. In that empty space, the We worked hard to train researchers research, or if you believe it to be misleading information. researcher is tempted to follow up to resist the urge to offer possible But also consider the constraints respondents may face. with possible answers: “Do you read responses, and to keep questions Rarely are people trying to undermine a project; rather, the newspaper or listen to the radio truly open. Those that succeeded they are trying to protect their interests. So, challenge for news? Do you call, text, or email were rewarded with answers they them with empathy. your friends?” It is only natural to never could have imagined. The Get multiple viewpoints. As with end-user interviews, want help when we see someone question “How do you communicate always seek out several perspectives from the same struggling or confused. with family and friends?” prompted organization—across functional areas and levels of senior- answers that ranged from messages ity—and from organizations with competing interests, to be But embedded within these friendly recorded on cassette tapes and able to develop a more holistic analysis of the situation. suggestions are assumptions about hand-delivered across borders to the range of possible answers. walkie-talkie gossip marathons that These ‘helpful hints’ turn the open- doubled as Taliban eavesdropping ended questions into leading ones, sessions.

80 Design Research for Media Development 81 Step 3 // COLLECT // The Art of the Interview

Note Taking Template Highlights key sections to include in your notes, and how they can be STEP 5: organized, as well as an excerpt from a set of notes from the Internews Take Good Notes Pakistan Case Study. Available at www.design.internews.org/tools.

Each interview should have a lead Interview Code interviewer and a note-taker. It is An identifying code for the interaction tough to take good notes while inter- that both protects respondent privacy viewing. Immediately after each and allows the team to organize its Recording Interviews interview, both researchers should interviews. General Observations fill in the notes from the session in Deciding whether to record interviews can be tricky. Video greater detail, address any gaps, and General impressions about the respondent and/or audio recordings can be valuable records, but they or environment, especially factors that confirm that they heard the same run risks as well. things. Every night, team members may impact how the responses should should complete their notes from the be interpreted. (e.g. Were there other When to use: Challenges people around? Did the respondent day. If they do not, the finer nuances seem nervous?) • When core team members • Recordings must be tran- of an interaction will be lost. do not speak the local scribed to be useful. The field Themes language and have to rely on team will not have time, so it In addition to basic respondent infor- Key themes or topics from the original local researchers or transla- requires additional resources. mation and interview location, several research framework, or that have tors. Recordings then provide Tight timelines or budgets emerged through fieldwork, that were key aspects of an interview should be an accurate and complete can make this difficult. discussed. Tagging notes with themes record of the interview. captured in notes: throughout makes it easier to process • People get nervous when and search through data later. • When it is inappropriate they are being recorded, and to take notes, for example, the presence of recording Notes when a respondent is device can stifle the interview. Protect What the respondent said. These should sharing personally difficult • In some contexts, recording respondent be objective document of what was information. tools may raise respondents’ and researcher said—not what the researcher inferred. security. • When a researcher is suspicions about researchers’ Quotes working alone, which makes affiliations and intentions. Verbatim quotes that illuminate a crucial conducting an interview and Check all data to make sure all • Reviewing full transcripts of point. These are useful for capturing simultaneously taking good identifying information has been interviews can be a poor use user perspectives and distilling key notes very difficult. scrubbed from notes and photos of time. Field teams usually insights. being entering the pool. This know their key insights. includes names, ID numbers, Analysis Allowing them to build upon phone numbers, and addresses. The researcher’s interpretation or analy- their gut, rather than check- In certain contexts, to protect sis of what was said, including possible ing for every possible detail, both your team and respondents, inaccuracies, inconsistencies with other may be a more efficient use consider using code words when interviews, or relevant context from their of time. capturing sensitive topics (e.g. any- own experiences. thing military-related) and ensuring the team carries no materials with organizational branding.

82 Design Research for Media Development 83 Step 3 // COLLECT // Ethnographic Photography Ethnographic Photography

Photography is a powerful investiga- given area? How did that well-known tion tool, able to communicate a full local NGO design its campaign story about the people and places posters? Can this be instructive for our where a program is meant to work. program’s communications strategy? Images capture the texture of complex What did that union membership dynamics such as service interactions, card to that look like? What does its interpersonal relationships, socio- design or content tell us about how Recording Interviews: economic status, and environmental this particular professional class is conditions in far greater detail than regarded in a certain region? A Management and Validation Tool can be recorded in written notes. Of course, know the cultural dos and Thus the photographs, like the inter- don’ts before snapping away. For views, should dive deep. Aim to take example, in some areas, it is forbidden As part of the research management and quality assurance between 10 and 30 photographs of to take photos of women. Even if a process, researchers were asked to audio record their every research activity or respondent woman herself agrees, the photo- interviews in contexts where it was appropriate. interaction, using the camera as graph could cause problems for her. an instrument, like the interview, to probe and illuminate. Take them from Keep your key themes in mind as you The possibility that their interviews Understanding the relative quality different angles and perspectives, take photographs, as the photos will would be listened to by project leads of different researchers’ outputs and focus on details, such as the help you communicate those themes increased researchers’ motivation to allowed the project team to evaluate make and model of a mobile phone at the end of the research. follow guidelines and not cut corners and rank the data to be considered or the various screens of the mobile in their interviews. in program design. Researchers with applications a respondent uses. poor notes usually had weak written Select interview recordings were English, so we found other ways to When layered with the notes, these transcribed and translated to assess draw out and capture their findings. photos tell a rich story about the and validate researchers’ submitted respondent’s values, communication Be respectful. notes. habits, use of technology, technical capacity, and the environments that shape them. Always ask permission to take photos, and explain to respondents Program designers often return to how photos may be used. research photos several times over the course of a project to look for At the end of the session, show details that they did not know to look respondents their photos, and for the first time. How much do people give them the private space and top up in mobile credit each time? opportunity to delete any ones Does it say something about the they want. value placed on communications in a

84 Design Research for Media Development 85 Phase 3 // COLLECT // Ethnographic Photography Principles in Practice

Notice the objects the Taking Good Photographs respondent owns. The things people own or create can say a lot about them. This photo of an enter- This is an excerpt of a field guide prepared for our researchers in tainment magazine and an anthropology book, both lying on a coffee table, tell Pakistan. It features tips on how to use photography as a research us about our respondents’ interests. and communications tool.

Ease respondents into Take the ‘follow-up photo’. photography. As we probe with our questions, so For many respondents, a camera can can we probe with our cameras. After be intimidating. To put respondents at noticing the anthropology book pictured ease, introduce your camera only after above, we took several ‘follow-up rapport has been built and take ‘warm photos’—this one shows an inside page. up photos’ of team members or other Beyond revealing the book’s contents, innocuous subjects. this photo provides insight into the respondent’s interest in the topic by capturing her handwritten notes.

Capture the environment. A good photograph is not posed. As much as possible, try to document Staged photos obscure what is really your surroundings. This photo of a happening. Candid, in-the-moment respondent’s living room, for example, photos are much more revealing, and helps us learn about her values. Photos can help bring the team back to the of her neighborhood may tell us about moment and place later on. People are her economic status, culture, or the naturally animated, angry, frustrated, resources available in her day-to-day life. sad, engaged, or enthusiastic, so capture them as such.

Take photos from different angles. Take photos of the team in action. In this photo, we see the specific details The research process is very much a of what she chooses to feature. These part of the story, and researchers are artifacts, and this photo in particular, active participants. With permission, prompt further discussion about her the various phases of research and the family history and specific interests. team driving them should be captured.

86 Design Research for Media Development 87 Phase 4 // SYNTHESIZE

Discovering Patterns

Data has little inherent value refine and reframe research ques- unless placed in a larger context. tions and begin surfacing potential Categories, structure, and rela- hypotheses to be tested and vali- tionships all help turn pieces of dated through ongoing fieldwork. data into information. Design Researchers should not be asking the same questions on Day 3 of a synthesis is a process to exca- project as they are in Week 3. Weekly vate valuable insights from the synthesis sessions organize early mass, and to bridge information findings into frameworks for analysis. and inspiration. At the end of fieldwork, the process is used to identify opportunities for Design research can yield a massive intervention. amount of data, much of which seems random, counterintuitive, or All practitioners, no matter the project contradictory. To collate, sort, and or the field, are managing constant make sense of data—and the patterns and never-ending streams of data, and connections within—we use inputs, and feedback—across PHASE 4: a process called synthesis, which project phases, and from diverse occurs across several stages of a stakeholders. As a process for turning project. data into useful information, synthesis is a skill worth learning and In Phase 1, we called the Research mastering. Synthesize Framework a living document. Field- based synthesis allows the team to

88 89 Phase 4 // SYNTHESIZE // Synthesis Timing Principles in Practice Synthesis Timing

Synthesis is a continuous, dynamic Synthesis is heaviest during and process, and each session builds on immediately after field research, the ones before it. Detailed in this but it is valuable in later phases as chapter are the five key steps, and well, in considering how to integrate each step is executed at different new inputs and feedback into design, times and with varying frequency. implementation, and monitoring and evaluation.

Adapting Synthesis in FATA In any period of heavy data collection, it is useful to think about synthesis in three stages: Operational challenges in FATA necessitated a modified Daily (Steps 1 to 3) synthesis process. Below is a description of how the teams Gather every day—during field research, this might be in adjusted for the demands and limitations of the context. the evenings, back at the guesthouse—to share what was learned. Structured exercises help surface interesting Instead of in-person, nightly, group to focus the next days’ research on threads, challenge team members’ biases, and define synthesis, sessions were held over more targeted questions and topics, key questions that must be followed up on in the coming the phone (with each researcher drawn from the themes emerging days. and the Islamabad-based project across research teams. leads) every 1 to 3 nights, depending Weekly (Step 4) on mobile connectivity. These Weekly synthesis sessions, con- At the end of each week, the team comes together for conversations, ranging from 20 ducted twice, convened researchers a deeper dive, going beyond basic synthesis exercises minutes to 1.5 hours in length, saw in Peshawar and allowed them to with a focus on exploring possible solutions. The researchers relaying highlights develop a common understanding of weekly synthesis is critical for developing shared, wide from their recent research through each other’s observations, providing framework of insights for later design. narrative and working with the the team with a shared foundation project leads through analytical of insights to enable program design. End of Phase (Step 5) exercises. Group synthesis also allowed the At the end of any intensive project phase, it is valuable team to check and balance indi- to conduct a 2- or 3-day synthesis session to allow On the other end of the line, the proj- vidual observations, as group deeper reflection on the research and, depending on the ect leads drew connections between conversations would highlight makeup of the core and local team members, to begin the different research reports being individual biases and help discipline the program design process. received daily. At the end of each the team’s understanding of its session, researchers were directed findings.

90 Design Research for Media Development 91 Phase 4 // SYNTHESIZE // Synthesis Process

Synthesis Data Wall

Process Collaboratively capturing and sorting data is critical to synthesis. The Data Wall is a tool for doing so. Each synthesis session contributes to a Data Wall as a visual map of findings—often on a physical wall. If teams are geographically distributed, this can also be captured in digital format, but STEP 1: in-person interaction is ideal. The Data Wall represents a shared process to capture all research findings, make visual connections between ideas, and start generating Capture All Ideas possible designs.

Start by having each researcher Patterns are usually supported read their detailed notes from the by several Observations, and day, relaying what they heard and point to trends taking place saw, with some light commentary in the community or society on and analysis. The rest of the team a larger scale. The Mighty Post-It uses Post-It notes to record the Note: Assign each Pattern with a Ever wonder why designers seem to love Post-It notes so pieces that jump out at them, as letter of the alphabet; if there are relevant to the research themes. too many, start using double-letters. much? They are great tools for sharing and brainstorming, because they are: With one idea per Post-It, record Insights are revelations. Each Observations, Insights, and Patterns. is clear, concise larger truth in Democratic. Most people can write on a Post-It note. Assign a different colored Post-It per response to the research ques- Once an idea is captured on a Post-It, it is equal to all other category to enable easier organizations, abstracted from analyses documented ideas under consideration. (As Post-Its only tion and analysis later on. allow a finite length for each idea, so all must be equally of several Observations of succinct.) This prevents conversations from being dominated Patterns by louder, more articulate, or more confident team members. Observations can include a Note: Assign each Insight with single piece of data, based on a unique number. Collaborative. The format encourages collaboration. what a respondent said or what On its own, a Post-It note is useless, but by tangibly mixing, a researcher observed. There is no right or wrong idea, or matching, and building off other notes, team members are able to create new systems of meaning from disparate parts. Note: Assign each Observation with order in which they should be ex- a unique number, and tag it with pressed. At this stage, the key is to Infinitely reconfigurable. Moving ideas around on the the Location, Date, and Interview capture all that the team has wall inspires new connections and inspiration, and the semi- Number. learned in the above four categories, sticky Post-It allows for ultimate adjustability. When the team without judgment, and to put them finds a configuration it likes, it can take a photo, then move Quotes are verbatim quotes up on a data wall. them around again. that illuminate a crucial point Disposable. In the program design process, people can get Note: Assign each Quote with This process fills up many Post-Its, caught up in certain ideas early on. They speak for very long respondent identifiers (e.g. occupa- and the data wall, surprisingly quickly, about them, convince themselves of the idea, then try and tion or social role, age, gender, or turning a mess of ideas in individual advocate it to others. With Post-Its, as each idea is only a others) to help contextualize the researchers’ heads into a tangible small scrap of paper—or a collection of scraps—it is easy just quote during synthesis. map of shared understanding. It is a to toss out suboptimal ideas before someone grows to love physical brainstorm that allows the it too much. team to discuss, manipulate, and build upon its findings.

92 Design Research for Media Development 93 Phase 4 // SYNTHESIZE // Synthesis Process Principles in Practice

Capturing Our Data Wall in Pakistan had hundreds of colorful Post-It notes, capturing our team’s field Data observations. Here a snapshot of our Data Wall, minus the codes for ease of reading:

OBSERVATION QUOTE PATTERN INSIGHT

Student package for DSL: Young men send out “People come to me Hujras are becoming Given that mobility is Rs. 900 per month mass friend requests to not just for haircuts, fragmented, repre- tightly controlled and girls via Facebook; build- they come for informa- senting increasingly information sources are ing network of friends in tion. It’s part of my job, narrow viewpoints and limited in range and Moving image trusted b/c SE Asian countries. people expect it of me.” demographics. quality, people cross- “I can verify with my own (Barber, 34, M, rural) validate news across eyes”, not so with printed outlets and mediums, word (Pharmacist, 28 People “trust” their Rural areas: little and verify through their “The operations are all y/o, M, semi-urban) landlords b/c “seeing understanding and/or social networks. is believing”—visible we think, talk, dream awareness of elections. accumulation of capital about. Where was last hit, Urban areas: highly vis- Prayer marks can help signals wisdom? when, and was it meant ible campaigns. Constant/imminent build trust, particularly for them? And, if we have danger, lack of good when traveling to more the energy, how we can Brother in Saudi found information, and lack of conservative areas. stop it? But we never Despite fundamental- out about nearby attack individual agency have have good answers.” ists’ opposition to TV, before he did; called led to intense fixation (Teacher, 54 y/o, M, significant growth in home immediately. Glad on monitoring (scouring For women who can’t remote) satellite—even in conser- for such line for breaking for information about?) leave their homes, the vative areas—with dishes security situation. nomadic tribal women news, frustrated at lack hidden inside homes. that go door-to-door are of such info at home. “The green ceiling (sky) their source of info. that hovers over us is Discussions about sustained by God, not Backlash against politics often focus on Mullahs.” (Farmer, 25 mullahs’ political Farming tips/instruc- y/o, M, rural) engagements, belief tions on Pakistan India, Afghanistan, or the US, not Pakistan. they are trustworthy Tobacco promotional only in religious realm collateral. “People think it that (e.g. Eid-related news, Barbershops are social. if you talk about the timing fasts). Men go to watch TV, Taliban, they get alerted When long-distance read the newspaper (or to it. A red light goes off.” drivers enter teashops, have it read to them if (Shopkeeper, 52 y/o, M, people gather to ask illiterate), and discuss rural) about recent journeys— current affairs. things seen and heard. “I am not interested in politics, it has nothing to do with me.” (Labourer, 37 y/o, M, rural)

94 Design Research for Media Development 95 Phase 4 // SYNTHESIZE // Synthesis Process Principles in Practice

STEP 2: Identify Connections Identifying Connections Once a Data Wall has been populated When the team settles on a grouping by Observations, Insights, and Patterns, of ideas it likes, it should summarize The process of organizing, pruning, and making sense of the team begins a creative process what the particular arrangement our Data Wall was no easy feat. With 10 local researchers from to find connections or patterns reveals about the design challenge, between each disparate idea. document the arrangement through FATA and 2 international staff, the connections we saw—and photos, and move on—it is just one our interpretations of them—varied widely. We embraced our Related ideas are clustered together, way to think about the data, and differences in perspectives to generate some interesting insights. whether the relationships are obvi- there is value in trying others. ous or subtle. Team members take turns moving Post-Its and presenting Through conversation, new Insights Although we were constantly organizing and reorganizing our Post-Its into the different connections and or Patterns are surfaced and added, different clusters of ideas and relationships, here a few excerpts from our themes they see between them. interesting groupings are labeled, clusters, the themes they represented, and the new insights they inspired: and the team works to situate the Nothing is permanent. The Data Wall analysis within the larger research Cluster 1: is a tool for exploration, and ideas framework. RELIGION and insights are arranged and rear- ranged in different configurations. Prayer marks can help “The green ceiling (sky) Backlash against build trust, particularly that hovers over us is mullahs’ political when traveling to more sustained by God, not engagements, belief conservative areas. Mullahs.” (Farmer, 25 they are trustworthy y/o, M, rural) only in religious realm (e.g. Eid-related news, timing fasts). INSIGHT

Popular reverence for religion is waning, and the influence of religious leaders (mullahs) is increasingly tenuous and narrow. In spite of this, in contexts where social trust is low—for example, in cities or new relationships—religious exhibitionism (e.g. growing beards, playing Islamic music) helps encourage trust and smooth interactions.

Post-it Category Key

OBSERVATION QUOTE PATTERN INSIGHT

96 Design Research for Media Development 97 Phase 4 // SYNTHESIZE // Synthesis Process Principles in Practice

Cluster 2: Cluster 3: INFORMATION SOURCES TOPICS OF CONVERSATION

Brother in Saudi found “People come to me Hujras are becoming Constant/imminent “The operations are all out about nearby attack not just for haircuts, fragmented, repre- danger, lack of good we think, talk, dream before he did; called they come for informa- senting increasingly information, and lack of about. Where was last hit, home immediately. Glad tion. It’s part of my job, narrow viewpoints and individual agency have when, and was it meant for such line for breaking people expect it of me.” demographics. led to intense fixation for them? And, if we have news, frustrated at lack (Barber, 34, M, rural) on monitoring (scouring the energy, how we can of such info at home. for information about?) stop it? But we never security situation. have good answers.” INSIGHT (Teacher, 54 y/o, M, Barbershops are social. remote) Men go to watch TV, read Citizens lack reliable sources for the news they seek. the newspaper (or have Various reasons—lack of mandate, lack of access, it read to them if illiter- regulations—lead to suboptimal info from traditional INSIGHT ate), and discuss current sources (e.g. hujras, messengers) and available media affairs. (international can cover the issues, but does not have FATA communities are consumed by getting information the on-the-ground presence; local has the presence, but about the military operations. People seek information cannot cover the issues). primarily for survival (what can we do about it today), For women who can’t not development (what can we do to prevent it tomorrow). leave their homes, the Thus, emerging trusted sources are those that have nomadic tribal women access and mobility, both across physical distance and People lack understanding of why and how the security that go door-to-door are social classes. Drivers, for example, bring eyewitness situation is the way it is, or any sense of what they can their source of info. accounts of news from nearby towns, while barbers offer do to affect it. astute analyses of current events drawing from inputs from diverse customers/classes that pass through his Cluster 4: When long-distance chair daily. POLITICS drivers enter teashops, people gather to ask about recent journeys— PTI mobile campaigning “I am not interested in Rural areas: little things seen and heard. sparking popular dia- politics, it has nothing to understanding and/or logue about Imran Khan, do with me.” (Labourer, awareness of elections. who is then used as a 37 y/o, M, rural) Urban areas: highly vis- gateway for discussions ible campaigns. about the elections. INSIGHT

Discussions about The historical insignificance in the region of national politics often focus on politics in FATA means both little understanding of how India, Afghanistan, or Pakistan politics operate. But even just increasing Post-it Category Key the US, not Pakistan. exposure to information can often help define its OBSERVATION QUOTE PATTERN INSIGHT pertinence and shape interest.

98 Design Research for Media Development 99 Phase 4 // SYNTHESIZE // Synthesis Process Principles in Practice

STEP 3: Identify What’s Missing Identifying Missing Pieces Throughout the synthesis process, the team should be engaged in a With design research, answering one question often leads to parallel and complementary exer- a mushrooming of new questions, particularly as patterns start cise to identify the holes in collected to emerge and hunches for opportunities start to develop. data, and to collect the additional pieces of information needed to round out its analyses. 1: RELIGION 2: INFORMATION SOURCES

Categorize the data that is still needed into two categories:

3: 4: Open Questions, which can Action Items, are high-priority TOPICS OF CONVERSATION POLITICS be answered through further tasks that are critical to the field research, expert consulta- process. Sometimes, they are tions, or desk research. There the means through which Open should be no shortage of these, Questions will be answered. as checking one question off a Action Items might include list usually leads to several more. trying a specific service, talking to a certain stakeholder, or Based on the clusters shared previously, our team identified new prototyping a particular service topics and areas we needed to understand. These included: delivery model. Open Questions: Action Items: • What kind of information do people • Contact PTI group to understand In formulating the workplan and get from barbers? And from truck its strategy priorities for each day of field- drivers? Be fearless • Conduct observation at a work, review the Open Questions • How does the government barber shop about where and Action Items and integrate communicate at the community level? research is • Shadow a driver for a day them into the team’s schedule as coming up short. • How do people get information about • Assess advocacy CSO landscape appropriate. basic public services—e.g. school fees, in FATA It is better to be honest and vocal clinic vaccination schedules, etc? about gaps in data than to try to • Do people feel compelled to participate slide past them, as they will be sure in the national elections? Why or why to surface later. not?

Every few days, we sorted through the mass of Open Questions and Action Items that had been added to the master list and prioritized and assigned them to team members.

100 Design Research for Media Development 101 Phase 4 // SYNTHESIZE // Synthesis Process

STEP 4: Develop Conceptual Frameworks

The synthesis process yields many into conceptual frameworks to better Observations, Patterns, and Ideas. understand the relationships At regular intervals—and at least between disparate data points, once a week during fieldwork—the and to surface opportunities for team should work to organize them intervention.

There are countless ways of organizing information. Here are a few commonly used models:

Ecosystem Map Process Map 2 x 2 Matrix User Journey An ecosystem map represents the A process map illustrates the A 2x2 matrix visualizes discrete A user journey is a step-by-step relationships between actors in inputs, steps, and outputs of observations to help draw out account of how a user achieves a broader ecosystem—including a given process, as well as the their relationships. Each matrix a particular objective. Objectives end-users, information providers, variables that impact each: plots relevant observations may include calling a friend, service providers, governments, How does a news story make along two key axes—for example, finding a good radio program, and others. Each may represent it to air? How does a person time, risk, cost, or reach—on or getting through the day. one type of relationship or several, register to vote? By breaking a a quadrant grid. In examining Journeys show the instances such as the flow of information, process down into it component how each quadrant is populated, where users interact with other the exercise of influence, chains parts, this framework helps iden- relationships and opportunities actors (touchpoints) in pursuit of command, or networks of tify and visualized key leverage will emerge. of their goals, as well as the trust. Such maps illustrate the points and/or opportunities for challenges they along the way dynamics within an operational intervention. (painpoints). Painpoints present context, and how individual opportunities for intervention, actors may impact programming. and touchpoints represent channels by which interventions may be delivered.

102 Design Research for Media Development 103 Phase 4 // SYNTHESIZE // Synthesis Process Principles in Practice

Using a Truck Drivers 2 x 2 Matrix Barbers

Tribal Leaders As we were interested in understanding the relative trust Teachers people placed in information sources, and the access they Landlords had to various sources, we tried to map different actors along a 2 x 2 matrix. Political Agents Intnl. Media

In visualizing this, we saw that It is also interesting to note that Military sources that were trusted (e.g. tribal most of the upper-right quadrant is - +

leaders) were not always accessible. populated by ‘informal’, community- Trust Religious Leaders And those that were accessible (e.g. level sources—given the dynamics of local media) were not always trusted. the region, most institutions and their We see in the top-right of the upper- representatives are inaccessible. Client Services right quadrant an opportunity to This suggests that working through leverage broadly accessible and non-institutional actors may key in trusted actors (e.g. truck drivers). this region. NGOs Local Media Politicians

- Access

104 Design Research for Media Development 105 Phase 4 // SYNTHESIZE // Synthesis Process

STEP 5: Identify Opportunities for Intervention

Examine the Data Wall and its clusters of insights. What opportunities were Once several opportunities have been identified, focus the team’s attention surfaced through the team’s interaction with the findings? on those that have the highest potential to achieve program objectives. There are several ways to do so: Examine the frameworks developed. Look for themes. Vote. In mapping a particular process, is there one step whose inefficiency Brainstorm Cluster opportunities that may One way to agree on areas of causes outsized stress for users? with be alike in some way. Are there focus is ‘dot voting’: Each team Looking at an ecosystem map, is abandon. common themes? Do several member gets an equal number there an actor with great potential opportunities speak to a core of votes (or dots) to represent to impact others but may—for one Start by exploring opportunities challenge, and thus a larger the opportunities they believe reason or other—not yet be able to without regard to organizational, opportunity that is in line with have the greatest potential. do so? operational, or technological program goals? As always, try They can put all their dots next to constraints. To start, focus only on several different groupings to one idea, one vote beside each Opportunities are challenges re- what would solve the problem— see what stands out. idea they like, or any variation in framed, and are structured to spark not necessarily just what’s possible. between. Over time, constraints will be ideas. Analyze the frameworks, introduced. Consider the environment. discuss among the team, and brainstorm as many as possible. Capture Analyze the broader context, each one on a Post-It note. Choose and the actors and forces that a new color to signal Opportunities, shape it. What are other organi- and integrate them into the Data zations working on in the region, Non- Be a and would your organization’s Wall and/or frameworks, as app- opportunities pragmatic ropriate. entry into an area complement, are valuable, too. visionary. duplicate, or potentially undermine existing efforts? Are there The design research process is Innovative, blue-sky ideas are great, trends or events that may impact helpful for surfacing new areas but some point, we all need to the operational context—or to explore, but also abandoning implement with our feet firmly on community needs—in the near previous ideas and assumptions. the ground. Thus, it is best to be future? What are your organiza- Both are equally valuable for realistic about constraints upfront. tion’s strengths and limitations, making sure time, energy, and The most impactful solutions are of- and where can your energies be resources are well used. ten simple ideas well implemented. put to greatest use. Be honest.

106 Design Research for Media Development 107 Phase 4 // SYNTHESIZE // Synthesis Process Principles in Practice

Groupings of ideas—a mixture of observation, inference, and captured Identifying commentary—and the further insights they inspired helped our team identify Opportunities opportunities for our program, and areas we wanted to avoid (see pages 97-99 for a refresher on the clusters):

1: RELIGION Opportunity: How might we question com- 3: TOPICS OF Opportunity: There is great desire for objec- monly received wisdom (e.g. about religious CONVERSATION tive information on the forces that shape the influence) to prevent hunch-based program security situation in FATA, and ways in which design? ordinary citizens may be able to influence it. Since the late 1970s, when mujahedeen funding Currently, the news they get is simply reports poured in, mullahs have had considerable political of drone attacks or militant bombings—the influence and authority in FATA. Thus, many information is used to inform daily activities assume that their engagement is needed/ and decisions (e.g. areas to avoid). They want helpful for development activities. information for longer term planning, even if it is only currently aspirational. As we see that a) mullahs’ influence is contracting (and even openly rejected in the political sphere), and b) religion serves an increasingly utilitarian 4: POLITICS Opportunity: How can we increase access to for many citizens, we should reconsider any information about topics that will be of critical appeals to religion—whether through engaging importance to FATA communities, to help show them or through positioning/content in any its pertinence and spark interest? civically relevant programming. Not only does it While observers disagree on the impact the seem like it would not help, but it may actually upcoming national elections will have on the be detrimental. region in the immediate term, there is certainly the potential for leveraging the political interest in FATA over a longer time horizon. 2: INFORMATION Opportunity: Consider leveraging the human SOURCES networks/nodes through which people are increasingly dependent. For citizens to have a voice in the future of the region requires initiating interest today. PTI has Citizens are relying on a wider array of informa- been able to spark dialogue through creative tion channels, often those informal and social, and broad-reaching campaigns. How might we as more traditionally used information and learn from them? media channels are decreasing in relevance and perceived quality. (Channel for delivery)

108 Design Research for Media Development 109 Phase 5 // DESIGN

Design the Interventions

PHASE 5:

For many, the word “design” This chapter focuses on the applica- invokes an image of a lone, tion of design research to program inspired genius, tinkering away design and implementation. This Design entails working hand-in-hand with in a lab until Eureka! strikes. In reality, while there is a degree of implementation teams to operationalize designs and to translate alchemy, design is a collabora- proposed interventions into outputs tive and creative but ultimately such as project proposals, resource systematic process that anyone plans, and/or programs workplans. can master. This process is by turns generative, If you’ve made it to this stage, you creative, and technical. The purpose already have the building blocks of immersive research is to enable of a strong, contextually validated the ability to “think like a user”, the program. Research enabled the beauty of doing it as a development development of empathy for users; practitioner the technical expertise synthesis helped us understand that is brought to bear. Where user the dynamics of the present and needs, institutional capacities, and the opportunities for a better future; contextual opportunities intersect is now, design will help define realistic the sweet spot. There is not one right plans to realize those opportunities. answer, but trust in your research and the instincts of your implementation teams.

110 111 Phase 5 // DESIGN // Step 1: Define the Intervention Step 1: Define the Characteristics for Intervention Defining an Intervention Having identified opportunities activity schedules, logical frame- for intervention, it is now time to works, or indicator performance Need. articulate each option, drawing from tracking tables—field knowledge is What need or painpoint will the intervention address? the insights about context, capacity, critical to ensuring all designs are How will it do so? and culture surfaced through the clear, credible, and have strong design research process. potential for impact. The following Process. page outlines the aspects of an How can this intervention be delivered? What are its different While different institutions will intervention which benefit most components, and how do they fit together? Are the delivery mechanisms realistic given resource, time, and institutional require different frameworks for from hard-won field insights. constraints? presenting program plans—such as Actors. Who are the actors or stakeholders that need to be involved, and at what stages of the process? Does the design provide them with incentives to participate?

Network Effects. Is there capacity for the intervention to be amplified by the activities of others without further direct input? Choose Wear communities Risks. different hats. over funders. What are the associated challenges or risks, and how can they be mitigated?

Always examine each intervention At this stage, it is tempting to let Sustainability. from different perspectives, both donor preferences shape how Does this intervention serve the existing interests of enough of users and implementers. interventions are defined. It is only practical: If a program can not find actors to generate demand over time? How would the solution work for funders, it will not get implemented. Acceptance Criteria. a specific user profile? How should it be adapted to meet the needs of But donors are realistic, and most What are the metrics users will use to assess whether the diverse users? What features might recognize the limitations of their intervention has fulfilled their needs? (These may include be needed to gain the support of own processes for determining criteria around cost, time, accessibility, or other design partners, implementer, or the priorities. Have faith in your considerations.) government? Consider all factors, research, and use it to present Existing Capacity. then prioritize. new opportunity areas to funders. They will respect your commitment Does it draw from or build on existing behaviors, norms or to the communities you both seek systems within the context? to service, and be grateful for the contextually grounded insights.

112 Design Research for Media Development 113 Phase 5 // DESIGN // Step 2: Create Implementation Tools Step 2: Create Implementation Tools

After articulating the characteristics A few common tools include: of each intervention, it is time to prepare for operational planning and handoff with your implementation Research & teams. Design Guidelines User Personas Design Report To prepare for handoff, it is critical Design guidelines are User Personas are Research and Design to document the knowledge gleaned abstracted but action- amalgamations of Report helps anchor the from the design research process in able guidelines that clarify what prin- characteristics of a particular slice core rationale driving the program. useful, actionable formats. Whether ciples are essential for programming, of a population, assembled into one A comprehensive report should but that are independent of any one specific, recognizable character. They the design research team partici- include key findings and insights, intervention. focus attention on the most salient conceptual frameworks, and other pates in implementation, the team and relevant characteristics of a artifacts of the design research can offer a variety of tools to ensure Through implementation, programs typical user, as relevant to program process. It should also include details the hard-won field knowledge can will inevitably evolve in design. Feed- design. of the research investigation, includ- guide operational teams through back from stakeholders and pilots will ing locations, sample, methods, and implementation. lead to revised strategies or activities. Each persona includes a set of limitations. The implementation team will need to characteristics and constraints that Programs are living systems and address these changes, often under must be considered in program design All these aspects should be woven good ones will have a long lifespan. high pressure and short timelines. and iteration, as they are the user together with sufficient contextual needs against which an intervention Memories fade and knowledge is narrative to enable new program and Design guidelines will support their should be evaluated. They should be institutional stakeholders to quickly lost as new stakeholders and prac- decisions and ensure contextual created for all key stakeholders that understand how a program was titioners are brought into programs insights continue to inform a program an intervention will target or involve, conceived, structured, and designed. over time, so these tools are critical. as it evolves. including end-users, service delivery agents, and key implementing actors.

114 Design Research for Media Development 115 Phase 5 // DESIGN // Step 2: Create Implementation Tools Principles in Practice

Design Guidelines User Persona

Our work in FATA yielded a robust set of design guidelines As we wanted to understand what information people were that covered a range of topics and issues relevant to media getting, and how, we sought to highlight these elements in our development programming. Here are three examples: personas. This one is for a typical youth profile that a program in the region may target.

Design for non-networked mobile usage. Arif, Student Male. Age 19. Middle class. Lives in a village 1 hour form Daggar. Mobile penetration in the FATA is difficult to measure. Many people, Languages: Pashto, Urdu particularly women, who have mobile phones do not admit to owning University Education them—much less report them in surveys—for fear of reprisal. Even in areas with poor or no mobile connection, people use them at home Info Access Few Sources Many Sources to take photos or to use applications (e.g. calculator) that do not Internet Non-User Wizard require a signal. Mobile ownership is, despite cultural restrictions Mobile Non-User Heavy User and the lack of connectivity, still aspirational. Arif is a university student. He speaks and girlfriends; he now has several such friends Leverage trusted service providers. writes Pashto and Urdu, and although he from the Philippines, with whom he shares learned English in school, his verbal compre- photos (taken on his mobile) and updates Increasingly, citizens are relying on a wider array of information sources, hension is poor and he cannot speak it him- about his life. often informal and social, and challenging the relevance and quality self. He wants to be a doctor and plans to go of traditionally used news and media channels. Those with geographic to China to study next year. His father worked Arif says that he trusts mullahs “50/50”: “I mobility and/or access to a range of social classes are seen to have in Saudi Arabia for many years; now retired, trust what they say (religious topics but he supports Arif and his five siblings. not what they do (worldly activities)”. Many more accurate/informed information, and these include service of them, he says, are hypocritical and do providers such as truck drivers, barbers, and door-to-door saleswomen. Due to security in the area, Arif’s parents do not practice what they themselves preach. not allow him to leave the house without a Being a cricket fan, Imran Khan is his favorite good reason so he tries to keep busy at home. politician. Understand colloquial code and test your language. He spends up to seven hours a day watching People fear the repercussions that may come from talking about TV, and likes the talk shows on Geo TV. Capital Arif has a Nokia 3110 mobile phone, and is Talk is his favorite. If electricity were more an enthusiastic SMS user. He sends up to 60 sensitive topics, and in certain areas, suspect widespread surveillance reliable, he would watch even more TV. Being text messages a day, but makes only 5 calls, if of communications. Thus, many have developed a system of code a cricket fanatic, he is frustrated by the frequent that. Many are forwarded messages to friends words to overcome these challenges. While these systems are highly power fluctuations, which force him to follow from news and infotainment subscription localized and dynamic, and thus we should not try to map or imitate matches on the radio instead of on TV. He reads services, and related commentary. He sends the Daily Express each day, but only the sports news about bomb blasts and strikes, but also them, it is important to understand how they impact communications, section. jokes and inspirational messages. and to test our own language to ensure appropriateness. He often surfs the internet while watching TV. Arif also listens to music on his mobile. He His family got DSL two months ago, and he does not have mobile data, so downloads has a Yahoo email address and a Facebook songs on the family computer to transfer to his account. Although he has his real-life friends phone. He spends about PKR 300 a week on on Facebook, he mostly uses it to find mobile credit.

116 Design Research for Media Development 117 Phase 5 // DESIGN // Step 3: Hand Off to Implementation

Research & Design Report Hand Off Workshop Components

Facilitated by the design research team and using The Research and interactive exercises, the workshop should cover: Design Report from the Internews Pakistan Case Study will be Overview of Design Research Process. released in early 2013. To develop shared understanding of the research process undertaken, and to create broad ownership of the findings For more information, visit: and inputs that inform program design. www.design.internews.org Analysis of End-Users and Stakeholders. To ground conversations about program design and iteration based on the observed needs, capacities, and constraints of program beneficiaries and stakeholders along the program delivery chain.

Presentation of Conceptual Frameworks. To articulate how opportunities for interventions were Step 3: surfaced, and to show the painpoints within existing processes, systems, and experiences that interventions Hand Off will address.

to Implementation Discussion of Design Guidelines. To align stakeholders on why certain programmatic choices Once the guidance tools have been Ideally, the workshop should involve have been made, and key contextual factors that should prepared, the team should prepare all stakeholders with a role in imple- inform program iteration moving forward. a workshop to share their knowledge mentation, including donors, part- with the implementation team, ners, and consultants. Development of Operational Plans. assess each proposed intervention, To produce realistic workplans, timelines, and budgets and develop workplans to operation- based on the intervention opportunities. While the imple- alize the programs. mentation teams will lead this activity, the design research teams will be able to ensure all plans are informed by a strong understanding of context.

118 Design Research for Media Development 119 Phase 5 // DESIGN // Step 3: Hand Off to Implementation Principles in Practice

Learning from Design Research in FATA

The design research process yielded rich findings about the Media Culture: Contours of Influence: complexity of the information ecosystem in FATA. In particular, In FATA, we found highly skeptical Mullahs, traditional religious leaders, new insights about mobile trends, media culture and contours media consumers with sophisti- are known to be highly influential of influence in the region could not have been gleaned from survey cated methods for verifying news in FATA. Studies have historically data. These findings will be instrumental in designing contextually information. shown them to be among the most appropriate programs that improve access to information by trusted institutions or leaders for communities in this region. Many people perceived a hierar- residents. Our research, however, chy of media quality, based on found that there are limits to the how ‘manipulatable’ each med- faith people place in these leaders. Mobile Trends: ium was: The printed word, they Many areas in FATA are without (within a geographic range), people commonly thought, is easier to con- While many respect such leaders mobile signal—reasons range from use walkie-talkies. trol than moving pictures, making for their wisdom and knowledge in lack of infrastructure, to funda- television more trustworthy than religious matters, FATA residents mentalist restrictions on technol- Compared to survey data, examining newspapers. believe they have no place in politics. ogy, to government control of mobile the factors that impact mobile Separation of church and state, it networks to prevent militants’ usage yielded a more accurate view Yet, many factors, such as an outlet’s seems, is the common belief of communication. A point-in-time of mobile behaviors and near-term ownership and affiliation, the news our respondents, even though the snapshot of FATA today would opportunities. As restrictions on topic, the journalist putting forward nature of survey work may not allow suggest moderate levels mobile mobile access loosen, and infra- the coverage, impacted how people them to say as such. of ownership: 51 percent. Yet many structure investments grow, rea- evaluated a particular news story. families have long had mobile sonable technological competency For breaking news, they tried to Design research allowed us to probe phones, using them to take photos or among FATA populations and signifi- verify the story across several media the contours of trust in religious play games, and to make calls when cant potential for mobile services channels and personal networks leaders, a critical area of under- they travel to mobile-permissive can be expected. before accepting it as truth. standing for governance-related areas. To talk to family and friends programming.

120 Design Research for Media Development 121 // LOOKING AHEAD Looking Ahead

What makes an effective development This reference guide offers con- This covers both research tools and dictory observations. Design synthe- program? For all the conferences, structive alternatives to standard the practical aspects of your field- sis excavates valuable insights from knowledge management forums, approaches by detailing how to bring work: A field travel plan, risk man- the mass, and helps bridge informa- and long, long discussions, “effective contextual empathy and insights agement, and the composition of tion and inspiration. development” is surprisingly intuit- into program design. Our aim is to your research team. ive. help practitioners better understand Lastly, after research and synthesis the causes, relationships, and Collecting data through immersion— have helped understand the oppor- Most successful development pro- human dimensions of complex the foundation to any effective pro- tunities for a better future, design grams—not unlike all strong design— contexts, and then provide tools to gram design—follows next. Primary helps define realistic plans to realize are based on an understanding of incorporate this knowledge into the data collection is how the team those opportunities. This entails the institutional and environmental design of innovative and realistic identifies the human needs, aspira- working hand-in-hand with imple- context and empathy for the people interventions. tions, motivations, and constraints mentation teams to operationalize they serve. that programs must design for, and designs and to translate proposed This is a process that begins with the institutional capacities that will interventions into outputs such as These two concepts are the intuitive discarding assumptions to define a shape the program and enable its project proposals, resource plans, core of effective development. But challenge. Too often, projects define success. and/or programs workplans. Where institutional procedures, budget challenges by presupposing a solu- user needs, institutional capacities, constraints, policy or donor priorities, tion. Framing and defining a design Through the process, we build empa- and contextual opportunities inter- tight timelines, and the pressure of challenge, therefore, is key for pro- thy both for the communities we serve, sect is the sweet spot. solving urgent problems all conspire gram design, and for the design and for those that will deliver the to wear practitioners down and research that will inform it. programs and services. constrain their work. Sometimes, A companion report of findings from this means we settle for subpar The next step is planning the investi- As data is collected, regular synthesis the Internews Pakistan Case Study that solutions—leaving the communities gation, which should accommodate helps sort through what can oth- was featured throughout this guide will we serve to settle, too. for the unique context of a project. erwise appear a massive amount be released in early 2013. For details, of random, counterintuitive, or contra- please visit www.design.internews.org.

122 Design Research for Media Development 123 // about

www.theReboot.org www.innovation.internews.org

Reboot is working toward a 21st cen- and institutions sheds light on com- Internews is an international non- and shares innovative approaches tury social contract. We partner with plex human needs, behaviours, and profit organization whose mission is to communication through creative the world’s leading organizations perspectives. Through field immer- to empower local media worldwide to research and development worldwide. and the communities they serve to sion, often overlooked contextual and give people the news and information develop, design, and deploy platforms cultural factors that shape user expe- they need, the ability to connect and Founded in 2011, ICIL strives to for inclusive and participatory gover- riences are unearthed. We have par- the means to make their voices heard. balance local expertise and global nance. We deliver policies, programs, ticular expertise in navigating cultural, learning in support of our vision that and services that enable individuals demographic, and economic divides Internews provides communities the healthy information ecosystems are and institutions to engage one anoth- to understand user experiences, per- resources to produce local news and a root solution to furthering human er and work collectively to solve social ceptions, and aspirations with regards information with integrity and inde- progress. challenges. to livelihoods or other sectors. pendence. With global expertise and reach, Internews trains both media ICIL serves as an open knowledge hub Understanding people and the envi- Importantly, our research is translated professionals and citizen journalists, that develops and inspires collabora- ronments they inhabit is integral to into actionable initiatives, including introduces innovative media solutions, tive investigation and experimentation. our approach. Reboot’s methods for innovative program designs, ICT increases coverage of vital issues and Through a rigorous, iterative process data collection, analysis, synthesis, services, and policy frameworks for helps establish policies needed for of pilots and experimental research and presentation prioritize individual some of the world’s most influential open access to information. ICIL seeks to contribute information voices, integrating these within larger organizations including Nokia, and tools to better understand the programmatic and policy discussions. Concern Worldwide, Merck, and the The Internews Center changing world of communications. Our ethnographic approach to under- World Bank Group. for Innovation & Learning standing individuals, communities, This is far from a solo endeavor. Building upon the breadth and depth Internews hopes that ICIL activities of Internews’ activities and experience will engage and benefit both those accumulated over a 30 year history of working at the front lines of global promoting independent media in more development and the communities than 80 countries around the world, they serve. the Internews Center for Innovation & Learning (ICIL) supports, captures,

124 Design Research for Media Development 125 www.theReboot.org www.design.internews.org