Audience Research in Media Development Overview, Case Studies, and Lessons Learned with Nancial Support from The

Edition DW Akademie #2017

MEDIA DEVELOPMENT Audience research in Media Development Overview, case studies, and lessons learned With nancial support from the

Imprint AUTHORS EDITORS PUBLISHER Dennis Reineck Dennis Reineck Deutsche Welle Laura Schneider Laura Schneider Bonn Christoph Spurk Christoph Spurk Germany Esther Dorn-Fellermann Jan Lublinski Charles Nyambuga RESPONSIBLE Roland Schuerho PUBLISHED Christian Gramsch October

MEDIA DEVELOPMENT

Audience research in Media Development Overview, case studies, and lessons learned

Dennis Reineck, Laura Schneider and Christoph Spurk (eds.) 2 Audience research in Media Development

Contents

Executive Summary ...... 4

Introduction ...... 6

. Audience research in media development: an overview ...... 7 . Dening audience research in media development ...... 7 . Current audience research in media development ...... 10 .. CIBAR: International Broadcasters' Media Audience Research ...... 10 .. Studies by state bodies, NGOs, or private market research organizations ...... 11 .. Academic studies ...... 12 .. Media development organizations: diversity in quantity and quality ...... 13 . Conclusions ...... 15

. Case Study : Critical media and information literacy: testing a test in the Palestinian Territories ...... 16 . Introduction ...... 16 . Research methodology ...... 17 . Findings ...... 18 . Conclusions ...... 21

. Case Study : Impacts of listening to radio. The “Land Rights and Women” radio program at Radio Nam Lolwe in Kisumu, Kenya ...... 23 . Introduction ...... 23 . Research methodology ...... 24 . Findings ...... 25 . Conclusions ...... 29

. Case Study : Improving the social media use of Colombian community radios: a method triangulation ...... 31 . Introduction ...... 31 . Research methodology ...... 32 . Findings ...... 33 . Conclusions ...... 37

. Lessons learned and way forward ...... 40

Literature ...... 44

Edition DW Akademie 3 Executive summary utilizing dierent innovative methodologies, and located in dierent parts of the developing world. The results have both This publication takes a look at current developments in the implications for media development practice as well as for eld of audience research in media development and presents how audience research ought to be conducted. three case studies testing innovative methods that can be of use for research, monitoring, and evaluation. They are meant The rst case study dealt with the measurement of Media and In- as an orientation and inspirational source for future projects formation Literacy (MIL) of young media users aged to years in this domain. in the Palestinian Territories (West Bank). From the media devel- Based on the information assembled in this study we opment perspective, the following insights were gained: make the case that media development needs audience re- – The schools that had taken part in MIL courses (proj- search to improve its projects. Media development actors ect schools) fared slightly better than those that did not, need to know more about the impact on their nal benecia- though the dierences were smaller than expected. ries if they want their work to be truly successful. – Girls fared much better than boys. Project school girls also However, it does not always make sense for media de- received signicantly better test results than their non- velopment actors to conduct or commission expensive and project compatriots. broadly representative research. Budgets and project sizes – The project school participants fared worse than their non- are limited, and oen the results of market or academic re- project counterparts at only three skills (source transpar- search eorts are only of very general use to a particular me- ency, source diversity, and respect of privacy). dia development project in question. Therefore ways have to be found to conduct focused audience research in media de- Lessons learned from a methodological audience research per- velopment – in order to gain specic and relevant knowledge spective were: directly related to the interventions. – Measuring MIL in this age group requires hands-on testing. The three case studies we present in this publication are Abstract questionnaires do not seem to have the same va- directly related to ongoing media development projects. lidity as practical tests. They apply tailor-made approaches to a particular setting. On – Media examples and item language have to be adjusted to top of this, we have assembled more general knowledge from the levels of juvenile age groups. the literature and past studies in audience research that are – Absolute comparisons of test items are not possible because of relevance here. it is almost impossible to construct MIL questions that are The main ambition for this project and publication was exactly of the same di¢culty. Thus, relative comparisons for DW Akademie to gather in depth practice-related knowl- should be drawn between dierent groups of pupils. edge. We want to share our lessons in a transparent manner. Therefore, we do not only present convincing approaches The second case study focused on the impact of radio broad- and positive results. We openly discuss pitfalls, limitations, casts on women and land rights (station: Nam Lolwe) in Kenya. challenges, and ways forward. It was designed as a quasi-experiment, comparing listeners of the radio stations to potential listeners and non-listeners. The Denition and functions ndings in terms of media development practice included: Audience research in media development is dened here as – It is of utmost importance that media development orga- any rigorous empirical enquiry into the behavior, knowledge, nizations monitor the content and its quality. This is the and attitudes of persons in the developing world receiving, en- pre-condition to achieve impact. gaging with, and/or non-professionally contributing to media – Personal communication and radio were found to be the content, on the basis of social science methodology and/or main sources of information for all participants. technical measurement. Audience research in media develop- – Age has a signicant in¤uence on people’s attitudes to- ment can have functions for exploring potential target groups, wards land rights and women, elderly people tending to be monitoring the progress of media projects, measuring the out- more conservative and favoring paternalistic traditions. come of media projects, evaluating the impact of media proj- – No signicant dierences in levels of knowledge, opinions, ects, or contributing to the sustainability of media outlets. It and activation levels were found between listeners and is of special relevance to the media development practitioner, non-listeners of the radio station. An exception was get- wanting to gain meaningful insights about the beneciaries of ting involved in a social group that addresses land rights media development projects. questions, which was signicantly more oen the case for listeners of the programs. Results of three case studies The aim of this project was to gather experience in the eld of audience research. Three case studies were conducted for this purpose, seeking answers to dierent research questions,

4 Audience research in Media Development

For audience research purposes, the following lessons were strengthen partners’ capacities in monitoring and evaluat- gained amongst others: ing their activities or improving their activities to be more in – In quasi-experimental designs, special attention has to line with beneciaries’ needs. The Kenyan research design was be given to the selection of the treatment and the control more complex and proted from long-term capacity building group, making sure that the distribution of socio-demo- with partners in terms of methodological know-how (e.g., ran- graphic and other relevant variables is similar. dom route sampling). – The stimulus should be clear and focused and of considerable duration in order to have an eect. Overview of audience research in media development – When measuring the impact of media, it is imperative to Beyond the case studies, the publication also sketches out document the contents themselves, to match the ques- the current eld of audience research in media development: tionnaire items with the content, and to make sure that the A content analysis of studies in the eld identied ve content has been distributed at all. sources of import to people involved in media development: – Research conducted by members of the Conference of Inter- The third case study addressed the social media strategies of national Broadcasters‘ Audience Research Services (CIBAR) two community radio stations (Granada Stéreo and Contacto provides the most valid data available on media access, use, ) in Colombia. It used focus groups, a survey, and Digital ana- and attitudes in the developing world, though the detailed lytics methods. The main ndings useful to DW Akademie’s research results are only available at a nancial cost. media development practice were as follows: – Studies by state bodies and NGOs, especially prevalent in – While Granada Stéreo generated a great deal of interaction Latin America, can also deliver valuable information for with listeners on social media, Contacto used social me- the media development practitioner. dia primarily to distribute information. – Media and market research can be a cost eective way of – Readiness by (potential) users to engage on social media obtaining audience research results, though publicly fund- was found to be highest for topics with a local reference, ed research is not as readily available as it should be. but there was also considerable interest in the peace pro- – Academic studies are most useful, when they test research cess and the associated recent history. methods that might be used in a media development context. – Social media content was found by the participants to – Media development organizations themselves deliver re- be most engaging if it focused on a wide variety of top- search of varying quality and have very dierent approach- ics, both informative and entertaining. Photos and videos es in terms of the transparency of their research. were especially popular. Challenges Lessons learned in terms of audience research practice included: Audience research in media development presents research- – When drawing up a mixed methods design, the contribu- ers with specic challenges. The researcher should at least re- tion of every method should be re¤ected to ensure that the ¤ect on the following points when conducting a study: data complement each other to deliver in-depth insights – Cooperation with partners: To what extent can project not possible with stand-alone methods. partner organizations be involved in the planning, imple- – Provisions should be made to ensure that samples across mentation, and analysis of the research? Is it viable and sen- focus groups are comparable and all segments of the target sible to involve market research or academic institutions? group are represented in the sample, e.g., quota plans or – Monitoring of implementation and analysis: Have organi- random sampling. zations involved understood the goals and methods of the re- – If remote management is necessary, when the principal in- search correctly? At what intervals should this be monitored? vestigators cannot be on location all the time, monitoring – Preparing and adjusting the instruments for the target at key intervals in the research process should ensure that group: What specic features of the target group have be agreed upon research designs are implemented in accor- taken into account when developing research instruments dance with agreements laid down at the outset. and sampling strategies? When and how should a pretest be conducted to test the research instruments? In all cases, close cooperation between project partners and – Mixed methods approaches: Can dierent methods be applied media development organizations was imperative because the to gain insights in an e¢cient way that would not have been partners knew the beneciaries best and could tailor methods possible with a stand-alone method? Can the data from the to suit their interests and behaviors. It is important that com- methods be combined in such a way, that ndings are coherent? mon standards be dened together from the outset in order – Designing audience research: Have methods been tested for audience research to live up to the necessary requirements under similar conditions? Are they valid and viable? in terms of validity and reliability. – Eects of audience research: What (negative or positive) The Palestinian MIL test and the methods mix tested eects might the audience research have on the media de- in Colombia proved to be cost-eective and can be used to velopment environment in which it is to take place?

Edition DW Akademie 5 Introduction ents three case studies testing innovative methods that can be of use for research, monitoring and evaluation. They are It’s a warm September morning in the Palestinian village of meant as an orientation and inspirational source for future Jaba’. School is beginning. As children rush across the court- projects in this domain – as they yield a wealth of information yard to get to class on time, a group of children already sit, on what works, what does not, and why. listening attentively to an instructor in the computer room, Thus this publication is not meant as a handbook for audi- explaining to them what to do. They’re taking part in a Media ence research. The reader will nd other books more suited in and Information Literacy (MIL) Test, drawn up by DW Akad- that respect (e.g., Mytton, Diem, and van Dam, ). This is a emie and organized by project partners Pyalara. Just as the work-book for people interested in actively improving media youngsters lean over their computer keyboards, considering development projects through research and evaluation. what answer to type into the online questionnaire, the lights Chapter one denes audience research in media develop- go out. Power shortage. ment, outlining its functions and dening criteria for good While the children react calmly, as they are used to this hap- practice. It is argued that audience research today has to take pening, the researcher’s adrenaline levels rise as the room goes both active and passive roles of media audiences into account, dark. Months of preparation threaten to go down the drain: relying on traditional social science data collection methods Developing, translating, and pretesting the questionnaire, and digital metrics to obtain a valid description of the behav- achieving government and school consent for the MIL test, the ior, wants and needs of viewers, listeners and readers. The over- logistics of getting kids from six schools to Jaba’, organizing view chapter also looks at the current eld of audience research the technical equipment, etc. in media development. Based on a review of studies published Audience research is a challenge. It can be hampered by over the course of the past six years, the chapter identies all sorts of di¢culties. Telephone or computer-assisted sur- typical features of research and innovative methods. Research veys depend on technical infrastructure and equipment that from members of the Conference of International Broadcast- might not be available. Statistical data is oen non-existent ers’ Audience Research Services (CIBAR), the various media or- or unreliable, making it hard to judge the quality of audi- ganizations in the eld, national survey organizations, NGOs as ence samples. The degree of professionality amongst mar- well as academic studies are included in the overview. ket research companies varies from country to country and Chapters two to four present three innovative case stud- region to region. This is also true for the amount of experi- ies conducted by DW Akademie. The rst case study deals ence and the reliability of enumerators, interviewers, or in- with measuring the impact of Media and Information Literacy structors sent out to conduct interviews or administer other (MIL) courses in the Palestinian Territories with an MIL test. research methods. And results can either be predictable or The second one focuses on a local radio station in Kenya. A ¤awed in some way. quasi-experimental design was used to nd out how much If research conrms the assumptions of practitioners, it is impact its reporting has on people’s knowledge, attitudes and oen seen as super¤uous. What is the use of conducting re- behavior regarding women and land rights. Finally, the third search if one nds out what one thought to know already? On case study was aimed at researching social media activities the other hand, if the research does not conrm the assump- connected with community radio stations in Colombia. A tions of practitioners, the approaches and methods are ques- mixed methods design consisting of Digital analytics, a survey tioned. And indeed, if data are ¤awed by sampling mistakes, and focus groups was used to explore user activities related to social desirability biases, or faulty techniques of data analysis, two community radios. decisions will be based on invalid or unreliable information. It Chapter ve summarizes the ndings and lessons learned, giv- cannot be disputed that all of these things can and do happen. ing recommendations for future audience research in media de- Despite all these potential problems, there are certainly velopment and identifying aspects that deserve further attention. two things one can do: Realistically appraise the general situ- The authors wish to thank project partners Charles Nyambuga ation in the target country and choose research methods that (Kenya), everyone involved at Pyalara (Palestinian Territories) take restricting factors into account; and develop a Plan B. and at the University Antioquia (Colombia), as well as DW In the case of the Palestinian MIL test, paper and pencil Akademie country managers and associates Jutta vom Hofe questionnaires were Plan B. Though the test would have been (Kenya), Verena Wendisch / Mona Naggar (Palestinian Terri- easier using the online questionnaire, the data could also be tories) and Matthias Kopp (Colombia) for their great support. collected by means of ordinary questionnaires. So all was not Special thanks also to Kerstin Weisbach and Johanna Wergen lost on that morning in the West Bank. The test could proceed, from Deutsche Welles market and media research unit for levels of MIL could be measured, leading to results that en- their support involving training on digital analytics. Thanks abled DW Akademie to improve activities for the benet of the also go to Lois Aspinall from BBC Media Action for providing children taking part in the test. us with information on the media development charity’s in- This publication takes a look at current developments in novative audience research methods. the eld of audience research in media development and pres-

6 Audience research in Media Development

. Audience research in media International Telecommunications Union (), less than ve development: an overview percent of the population of eleven sub-Saharan African countries had access to the Internet in . Thus, digital analytics Dennis Reineck, Esther Dorn-Fellermann, Roland Schürho and metrics are relevant primarily in regions where digital access is wide-spread. Even if Internet access is common, as in . Dening audience research in media development many parts of the Middle East, Latin America, and parts of Asia and Africa, it should be borne in mind that people active on Audience research in its classic form is devoted to nding out the Internet are not identical with the population as a whole. more about who is listening, watching, or reading at the end And in the end, the aim of audience research in media devel- of the line. That line traditionally has been mass media com- opment is to go where the beneciaries are, and not vice versa. munication via channels such as television, radio, or printed As a consequence, digital media and the more active role newspapers/magazines. Today, with the advent of digitally me- audiences are playing should be seen as a complement to the diated communication, audiences can be dened as any group traditional media that still are in place and whose reach oen of people receiving, forwarding, and/or sharing entertainment exceeds digital modes of communication. The result is a com- or information content by technical means. The denition of plex new media landscape, in which the number of voices and media has changed, too. The monopoly of the incumbent me- opinions being circulated has increased exponentially, with dia has been broken, new players have entered the scene. Rele- traditional media existing alongside other modes of commu- vant information is provided by a myriad of sources nowadays. nication. Rather than viewing the exposure and the engage- However, despite several attempts at redening the media user ment models as mutually exclusive, they should therefore be as a “prosumer” (Anderson, ), “produser” (Bruns, °), seen as two perspectives of audiences that re¤ect the complex or “co-creator” (van Dijck, ±), some authors speaking of new media ecology in which audiences use both digital and “the former audience” (Gillmor, ) or “the people formerly traditional media, in which they can take on both passive lean- known as the audience” (Rosen, ), media audiences have back or active lean-forward roles, in which audiences remain not become extinct. Even in digital contexts, not everyone takes part of local communities and can nonetheless interact with on the role of a producer and there is a tendency to consume others across regions. Thus, audience research should draw media contents rather than co-produce them, whether it be via on both traditional data collection methods – surveys, focus video or streaming websites or via social networks. groups, guided interviews, observations – as well as digital Nonetheless, there have been changes in the media land- methods of data collection. The methods complement each scape, driven by phenomena such as social media and real- other and each has its advantages and its drawbacks. While time mediated interaction, and this has prompted Michelle J. social science type empirical research is resource-consuming Foster () to develop a new model of the role of audiences and oen based on self-report, digital methods are limited to in media development. In a study for the Washington-based certain forms of behavior and there is no way of nding out Center for International Media Assistance (CIMA), Foster ar- why people behave the way they do. gues that audience research should focus on an engagement Audience research in media development is rigorous em- model rather than on the traditional exposure model. The pirical enquiry into the behavior, knowledge, and attitudes exposure model, on the one hand, corresponds to the way of persons in developing countries receiving, engaging with, audience research is usually conducted, dening an area and/or non-professionally contributing to media content, within reach of certain media, determining a point in time on the basis of social science methodology and/or technical for the survey, and then drawing a random sample of target measurement. It can be conducted at a cross-national level, groups, asking them to indicate how they used media in the national, regional, local, or media house level. Oen in media past. The engagement model, on the other hand, focuses on development, research has to be pinpointed to small segments the changes in the media ecology brought about by digitali- of the population, since the beneciaries of projects are sel- zation. Therefore, Internet-based audience research focuses dom the national population of a country as a whole. on analytics and metrics (Cherubini and Nielsen, ), den- Audience research has increased in media development ing audiences, and measuring their media use on the basis of over the course of the past decade. This has to do with the gen- their actual real-time online behavior. eral aid eectiveness debate in development and a stronger The model is interesting because it draws attention to the focus on the impact and outcome of interventions. The Paris fact that audience-related data is continually produced on the Declaration on Aid Eectiveness states: “Managing for results Internet. This data is rarely used, even though it is, for the most means managing and implementing aid in a way that focuses part, readily available and the metrics give information on ac- on the desired results and uses information to improve deci- tual behavior rather than subjective estimates. It does seem, sion-making.” (OECD, : ) With media development in- however, that the diagnosis of a paradigm shi from the expo- creasingly focused on impact instead of output, the focus of sure to the engagement model is premature, especially consid- project monitoring and evaluation has invariably shied from ering the status quo in developing countries. According to the documenting the media projects themselves to nding out,

Edition DW Akademie 7 what (positive) eects the projects have. Without reliable and goes beyond impressionistic assessments. The representative e¢cient tools for measuring this, results will stay elusive and sample of - to -year-old media users in four provinces managing for results will remain a concept with little potential yielded interesting results, showing that television was being of being put into practice. watched more in the urban surrounds of Kigali than expected. To judge what (longer term) eects media development DW Akademie’s strategy in Uganda was developed based on projects have caused or contributed to, there is no alternative these ndings. Thus, the study allowed for informed decisions, but to take a look at what changes have taken place amongst rooted in knowledge of actual patterns of behavior, preferences the eventual beneciaries, ordinary citizens on the street. This and needs, rather than on assumptions based on a more or less is in line with the human rights based approach to media devel- thorough comprehension of what is going on. opment, which views citizens as holders of rights such as free As stated earlier, the strength of the argument relies on access to information and freedom of speech (Reineck and Lub- the quality of the data. Recent trends towards enabling non- linski, ). It is the degree to which these freedoms are upheld experts to conduct do-it-yourself research should be encoun- and furthered by media development projects that ultimately tered with caution. Bad quality data can lead to wrong deci- provides the scale for measuring their success. sions with costly consequences (Jerven, ). Good quality Two kinds of audience research in media development data, on the other hand, provide practitioners with a legitimate may be discerned: Intervention-oriented and media-outlet- base for setting goals and planning activities. Thus, ownership oriented research. These two research approaches can have in audience research presupposes a common understanding diverse functions. Depending on which function is prioritized, of standards of research, making sure that all partners comply they can be subdivided into ve distinct types of research. with quality standards throughout the research process.

Intervention-oriented research aims at – exploring potential audiences for media-based projects, Quality criteria for audience research – providing valid and reliable audience monitoring informa- Good quality audience research data depends on a number of tion to allow for informed project decisions, factors, the most important of which are: – measuring the total reach or circulation of media in con- – Dening the goals of the research in a clear manner, as nection with project activities as an outcome, or research questions or hypotheses; – measuring the impact of project activities for evaluation – Dening the target group or segment of the population to purposes. be researched comprehensively; – Choosing the appropriate research design to answer the Media outlet-oriented research, on the other hand, is primar- research questions or test the hypotheses; ily aimed at – Developing a data collection instrument (e.g., questionnaire, – enabling the nancial sustainability of local media outlets. interview guide, observation instructions) that produces valid and reliable data; In the latter case, measuring the reach or circulation of media – Choosing the appropriate method for drawing the sample is sometimes not directly related to project activities, but is a out of the population (or a subsection thereof); means for generally establishing a common currency in a de- – Administering the research instruments with the help of veloping advertising market. trained sta; A classic example of the rst type of exploratory research – Analyzing the data using appropriate methods of analysis to nd out more about potential audiences is a nationally rep- (e.g., statistics); resentative study by BBC Media Action, conducted to nd out – Visualizing and reporting the results in a transparent man- more about how young Cambodians’ access and use media ner, making it easy to understand how the research questions as well as their civic engagement (Harris and Gowland, ). or hypotheses were answered. Though the study showed some predictable outcomes, such as an urban-rural divide in terms of television and internet access, or a dependence of mobile phone and internet access on lev- The second type of research, oen involving smaller scale els of income and education, it also revealed that using mobile monitoring or midline studies, focuses on the audience in or- phones for listening to radio broadcasts was the second most der to improve the appeal, quality and/or eectiveness of proj- popular function of mobile phones for young people. ect related media. For instance, media and market researcher A study by German media development organization DW InterMedia conducted a survey and focus groups with inhabit- Akademie conducted in , designed to nd out about media ants of Papua New Guinea in , primarily to adapt National consumption patterns of young media users in Uganda (Fiedler Broadcasting Corporation’s radio program to the needs of the and Meyen, ), is a further example of this type of research, population. The station was being supported by InterMedia at showing how audience research is capable of providing media the time (Debeljak and Bonnell, ). They found stark dier- development practitioners with audience information that ences in reception depending on whether inhabitants lived

8 Audience research in Media Development

in the highlands or on the surrounding islands. This led to tronic – whose scope and in¤uence are unknown”, as Marie- recommendations on improving access to information and Soleil Frère points out (Frère, : ). Independent audience catering for the dierent segments of the population. research is a way of giving media markets and advertisers an One of the most common uses of audience research in information base to work from (Spurk and Dingerkus, ). media development is the third type, i.e., measuring the reach In , Gavin Anderson of development consultancy The or circulation of media products in connection with develop- Springeld Center and Paul Haupt of the Pan African Media ment projects or programs. In media for development projects, Research Organization (Pamro) presented their DFID-funded this method is frequently used to nd out how many persons eorts to introduce a common audience measurement sys- of a target group were reached by a specic media message. tem for the radio advertising market in Nepal at the Forum For example, the BBC World Service Trust, predecessor of BBC Media and Development (FoME) Conference in Germany. The Media Action, used a survey in ° to nd out whether pro- background was that percent of Nepalese radio stations grams addressing health issues such as HIV/AIDS, tuberculo- stated that it was likely or possible that they would close sis, and malaria had reached the Cambodian population in six down in the next years and more than half believed radio provinces. The ndings conrmed that indeed ° percent of was not a protable sector to be in. Advertising expenditure respondents had encountered HIV/AIDS content on television was very low compared to countries of a comparable popula- and percent had heard it on the radio over the course of a tion size such as Sri Lanka and Uganda and most of the mon- year (BBC World Service Trust, ±). ey was being spent on printed media, while radio had a con- Over and above basing media development activities on siderably greater reach than newspapers and magazines. The evidence rather than hearsay, audience research also has an hope was that a common currency of audience reach would important role to play in the assessment of these activities, lead to a redistribution of revenue that corresponded more which is the fourth, impact-oriented type of audience re- closely to the actual audience distribution in the media mar- search. Audience research is one of the most important elds ket (Anderson and Haupt, ). Such a common currency of action when it comes to evaluating and measuring the im- had been absent since Reuters had previously withdrawn its pact of media development projects. If the aim of media de- ratings system from Nepal. velopment projects is to have a broader in¤uence on society Despite audience research fullling important functions as a whole, then it is imperative to nd out more on how me- in media development, the extent to which such research is dia development projects improve audiences’ knowledge and conducted and the budgets available for such research are participation levels, and change their attitudes, opinions, and fairly limited, as research departments from media organiza- practice in a positive way. tions can attest to (compare: Cauhapé-Cazaux and Kalathil, An attempt in this direction was a study by US media de- ). Audience research in media development tends to suf- velopment organization Internews, focusing on the contribu- fer from the same tension that characterizes monitoring and tion of community radio stations to development in South evaluation (M&E) programs in general: the clash between the Sudan. On the basis of surveys with radio listeners each implementation and the compliance sides of work (Natsios, at ve community radio station sites in South Sudan, a total ). On the one hand, media development programs have of interviews, and focus groups. The survey consisted to be put into practice and this in itself bares many chal- of questions and was based on an indicative rather than a lenges. From this perspective, M&E represents an additional representative sample of listeners. The results, however, could burden, binding the attention and time of actors who some- show that the listeners themselves believed that the commu- times think they could be doing “more productive” work in nity radio stations had a positive in¤uence on voter participa- the eld, as well as binding resources that are thought to be tion, more girls attending school, less gender-based violence, better invested in capacity building or other measures on the reduced incidences of community con¤ict, or the election of a ground. It should, however, be emphasized that audience re- woman governor. It was, however, not possible to statistically search is essential for guiding those practical activities on the test these attributions by the interviewees. This would have ground. Many things can go wrong if activities are not based either required a random sample or a quasi-experimental de- on sound analysis and evidence. M&E activities do not only sign. Additionally, social desirability could have played a role serve purposes of accountability, but also of organizational in how respondents answered the questions. and project learning. However, audience research is not only a source of knowledge for media development practitioners but also a prerequi- site for sustainable media funding. This h type of audience research tries to overcome the lack of transparency of advertising markets as one of the major reasons why media companies in many countries nd it hard to generate revenues. Similar e orts are currently underway in South Sudan, conducted by US- “[P]otential advertisers are reluctant to invest much in buying based Forcier Consulting for Internews. In Uganda, a system for conducting space and airtime on media outlets – both printed and elec- regular radio listener surveys has been successfully installed.

Edition DW Akademie 9 On top of this, the most fundamental argument on behalf The starting point for drawing the sample was the resource of audience research in media development remains: Legiti- center at the Catholic Media Council website, which holds macy for media development ultimately depends on proving more than , publications on media development (Cam- what good projects have achieved for the bearers of human eco, ). The websites of organizations in the database were rights. And this evidence can only be achieved through good then visited to supplement the studies found at Cameco. The quality research that is adapted to a given media lanscapes sampling criterion was that each study had to be of an empiri- and also oriented towards the questions that are of relevance cal nature and would have to deal with media in at least one to development interventions. developing or transition country, dened roughly as a country in Latin America, Africa, Asia, Oceania, or Eastern Europe.

. Current audience research in media development .. CIBAR: International Broadcasters' Media Audience Audience research has always had an economic and a politi- Research cal side. In the beginning, audience research was an exercise carried out by media houses themselves, driven by the need CIBAR’s (Conference of International Broadcasters’ Audience to comprehend their audiences and to attract advertising rev- Research Services) broad purpose is to establish and main- enue through convincing descriptions of who was receiving tain technical and audience research standards for interna- their media messages. This type of economically motivated tional broadcasters. The data produced on behalf of CIBAR are research then led to the establishment of national ratings sys- amongst the most rigorous and reliable data available on me- tems, measuring the circulation of newspapers and magazines dia use in developing countries, though they are not collected as well as ratings and reach statistics for television, radio, and with a media development purpose in mind. The data sets are internet-based media. available for purchase and results are only rarely published. On the other hand, audience research can also be connect- Due to the fact that radio and television of state-funded inter- ed with political interests. Some actors want to nd out more national organizations like Voice of America or Deutsche Welle about the eectiveness of political messages. This research are broadcast abroad, they inevitably gain deep insights into was usually funded by state or other international institutions the media landscapes of their target countries. While the focus interested in knowing to what extent and with what eect po- initially was on radio, now all electronic media are covered, in- litical messages were being received by whom. cluding listener- and viewership on mobile phones. Both directions of audience research are still predominant, CIBAR research focuses on Subsaharan Africa, Middle East though audience research in media development has devel- North Africa, the Former Soviet Union, as well as South East Asia oped as a small but growing additional eld in recent years. and South Asia. The main data source within CIBAR’s Interna- Of course, economic and political agendas also in¤uence and tional Audience and Research Program (IARP) is Gallup’s World shape research in this eld. But a common denominator of ef- Poll. This survey series is currently the largest source of world- forts in media development is that audience research is aimed wide, nationally representative data on media use in countries at the betterment of the situation of the beneciaries them- of the Global South. The current IARP is based on data from ° selves. So audiences are more than a research object to be ex- country surveys (Diego-Rossel, ). country reports, show- ploited economically or to be in¤uenced politically – they are ing the main ndings, are available on BBG’s homepage. holders of human rights, to be strengthened and to be served CIBAR’s preferred method is quantitative surveys based on by the means of media development. The aim of this chapter national representative samples. Regular sample size is about is to provide an overview of the eld and to identify shortcom- , respondents. In the case of smaller countries, samples ings that can be addressed by the case studies conducted by of around inhabitants (e.g. Haiti, Suriname, North Cyprus) DW Akademie and presented in chapters two to four. are possible; representative surveys in larger countries with The following overview is based on a sample of studies heterogeneous populations like Russia, Nigeria, Indonesia and published between and , publically available by April Pakistan require larger sample sizes. For instance, the sur- . The sources include the Conference of International vey in India had some , respondents. Broadcasters‘ Audience Research Services (CIBAR), media de- Respondents years and older are usually interviewed. velopment organizations, research by state bodies, and mar- Interviewing method is the face-to-face paper-and-pencil or ket researchers as well as academic studies. face-to-face oral interview aided by computers. Topics in the Because of the indicative nature of the sample, few per- questionnaire range from media usage habits, classic media centage points but rather trends in the data are reported: radio and TV and – increasingly important – internet access, studies were of CIBAR providence, studies stemmed from mobile phone usage and social media, to attitudes towards academic sources, from state bodies and market research- news and topics of interest like politics, health, economy, en- ers, and studies were from diverse media development vironment, or sports and music. organizations.

10 Audience research in Media Development

The strength of CIBAR’s audience research stems from Worldwide media and market researchers conduct surveys the large amount of timely representative country surveys on a variety of topics. Oen, such national surveys pertaining based on standardized questionnaires. This methodological to the media are aimed at establishing or maintaining a com- approach enables the big international broadcasters to have mon advertising market currency. The Pan-African Media Re- exact descriptions of long-term trends concerning media us- search Organisation (PAMRO) is engaged in setting up nation- age, access and attitudinal or behavioral change, both in the al surveys in various countries in Africa to establish a common total population as well as in subgroups. CIBAR also publish- currency for the advertising market (Haupt, ) and market es comparative data in coordination with Gallup, such as the researcher Ipsos runs an All Media & Products Survey (AMPS) study “Women & Media: Africa in Focus” of , based on in Kenya, Uganda, Tanzania, Mozambique, Zambia, and Ghana seven surveys (Gallup, ). at regular intervals (Ipsos, ). The results of these surveys, Media development practitioners will nd the published CI- once established, are rarely published, since they are expen- BAR data useful to contextualize and legitimize their work. The sive and for exclusive use of business customers (media out- data is as representative as possible and can thus be used as a lets, advertising industry). An example of short-term audience sound basis for decision making. The drawback is that the num- research aimed at helping TV and radio stations generate ad- bers are only published in an aggregated form, reporting media vertising revenue is a survey done by the US-based Interna- access, usage and attitudinal data for world regions or countries. tional Research & Exchanges Board (IREX, ) in Kosovo, This means that only broad statements can be made, without nding out what market shares the stations have. being able to break the data down to parts of the country or seg- The ICT surveys by the International Telecommunica- ments of the population. The original data sets are only available tion Union (ITU), collecting data on the use of telecommu- for purchase and quite expensive. Nevertheless, if resources are nications and information technology in over countries available and long-term projects with a broad focus are planned, and regions, are an example of internationally comparative it can make sense to acquire and use CIBAR data sets for explor- studies (ITU, ). Closer to classic mass media audience atory or baseline purposes. Also, CIBAR surveys may exist for measurement, mobile survey provider GeoPoll distributes regions such as Iraq or Afghanistan, where own research is not SMS based surveys to nd out about the reach of media mes- viable (see the overview of data sets at: http://www.gallup.com/ sages. The company advertises being “the largest provider of services/±/country- data-set-details.aspx). overnight media ratings in Africa” (GeoPoll, ), though sampling issues could draw the representativity of the nd - ings into question. .. Studies by state bodies, NGOs, or private market Both national and international studies conducted by research organizations media and market researchers are aimed at generalizing the results to the national population. This requires represen- In several countries, notably in Latin America, state funded tative samples, which is why the studies oen employ ran- agencies and bodies are responsible for nationwide audience dom sampling and aim at large samples of more than , research. For instance, the Peruvian Consejo Consultivo de Ra- respondents. Unfortunately, country methodologies are sel- dio y Televisión conducts surveys on television and radio use dom reported in detail for internationally comparative stud- every couple of years (Concortv, ) and several Latin Ameri- ies, so the reliability of the data in most cases is impossible to can countries have Observatorios de Medios – oen funded by ascertain. On the other hand, the trustworthiness of national local NGOs – which also collect data on media use in Bolivia, surveys meant to provide advertising markets with a com- Nicaragua, Colombia, or Mexico amongst others (see the Boliv- mon currency depends precisely on reliable data and trans- ian example: ONADEM, ). This might explain why media parent methodologies. Though the data are seldom provided organizations are less active in terms of their own audience to the public, it may be assumed that they are based on con- research in Latin America, where national audience research siderable methodological rigor. is oen run by state organizations, universities, or local NGOs. Most of this type of audience research is done using large scale representative surveys, conducted at intervals of every two to four years. The samples oen reach sizes of more than Members of CIBAR include the British Broadcasting Corporation (BBC) , respondents, with a bias slanted towards urban dwellers World Service, Broadcasting Board of Governors (BBG) with its (regional) which in most cases is reduced by weighting the respondents ac- U.S. civilian international media (Voice of America, Radio Free Europe/Ra- cording to census data. In Chile, for example, all respondents to dio Liberty, RadioFreeAsia etc.) and Radio France International (RFI), Ra- a survey on television use were chosen from the biggest city in dio Nederland Wêreldomroep (RNW), Radio Canada International (RCI) or each province (CNTV, ). Generally, there is a strong focus on Deutsche Welle (DW). television in Latin America, though internet consumption has The countries currently on the website (www.bbg.gov/bbgresearch): received more attention in recent years. There is also consider- Afghanistan, Burma, Cambodia, Egypt, Indonesia, Iran, Mali, Nigeria, able interest in the media consumption of younger age groups. Pakistan, Somalia, Russia, Tibet, Turkey, Ukraine, Vietnam, Zimbabwe.

Edition DW Akademie 11 The sources of audience research data – state funded, ). In the following, the ndings are sorted according to the NGO driven or conducted by media and market researchers geographical regions where the universities which carried out – can serve dierent purposes in media development. The the studies are located. national data generated by state funded bodies or NGOs in Most studies in the sample stemmed from British and Latin America can be used for exploratory or baseline pur- American universities and research institutes. The main top- poses. While disaggregated CIBAR data are only available at ic was social media, with a special focus on information and a cost, it is conceivable that media development organiza- communications technologies for development (ICTD). Inno- tions could obtain detailed data sets of national surveys if vative studies included interviews of radio station audiences the government in question is willing to oblige. Media and via SMS on social topics from the Center for Governance and market research, on the other hand, is the most eective way Human Rights at Cambridge University, and a study on citi- of conducting audience research, if the research organization zen journalism gaining insights into the perceptions and uses has the experience and professional know-how in the eld. It of voice-based tools in rural and remote parts of India (Preeti, comes at a cost and thus most data sets are not readily avail- Donner and Thies, ). able. Data published are only at the consent of the funder. Numerous studies also came from universities located in Since many media development organizations are publicly the Arab region. Especially the Mohammed Bin Rashid School funded, there is no reason why they should not make their of Government in Qatar but also the United Arab Emirates data readily available to the public. Also, media development University, the Northwestern University in Qatar, and the organizations can themselves engage in audience research Dubai School of Government were interested in big compara- capacity building. In its most advanced form, this can entail tive studies in six or more countries in the region. The elds installing a professional audience measurement and ratings of research were predominantly access to and use of the inter- system able to make media sustainable by giving the adver- net in Arabic countries, the increasing connectivity between tising market a common currency. Here, media development government and citizens through social media and the role of practitioners go from being audience research data recipi- social media in the transformation process. The most popular ents to enablers of data production. method of data collection was quantitative online questionnaires. The number of people interviewed varied from to ,. Qualitative methods were seldom utilized. Method- .. Academic studies ological innovations were rare. An example might be the tool developed to measure digital literacy for Arab Internet users While international broadcasters and media development orga- (Dun and Mutassem, ). nizations pursue their own agendas, academic institutions, at Numerous studies have been conducted by Latin American least in theory, generate knowledge beyond strategic interests. academic institutions. An interesting overview of audience The most extensive research interest is directed towards nding research in Latin America is provided by Jacks, Marroquin, out about access to and/or usage of digital media, especially so- Villaroel, and Ferrante (). The predominant mode of data cial media since , sparked by the so-called Arab Spring. This collection was quantitative surveys. The main focus was on is in contrast to the media focused on by CIBAR and media de- media use in general or on internet access and use. Other top- velopment organizations, indicating that universities are more ics, oen related to poverty were reporting on elections, media interested in media innovations and communication ¤ows be- credibility, reading behavior and prejudices, and the responsi- yond the classic media channels. Topics related to social issues bility of media. Most of these studies looked at traditional und such as health, education, or environment, however, receive digital media. The target groups were mostly years or older. considerably less attention in academic research projects. There Where it was recorded, the number of participants diered was little focus on impact, since academic studies are rarely from to ,. The few studies in the sample that used a done for evaluation purposes. The most active universities, ac- mixed methods design with a strong qualitative focus were on cording to our ndings, came from the Arab region, perhaps be- digital media and especially aspects of the digital divide (Pit- cause these are developing countries with the highest levels of taluga and Rivoir, ) and ICTD. education and the best funding for universities. Far less studies in our sample were published by Asian and A range of methods from focus groups, in-depth interviews African universities. The Asian studies focused on the usage of and online questionnaires to experiments or observations mobile phones (Media Asia special issue, ) and there was no were used and there was no clear preference for quantitative preference for qualitative or quantitative methodologies. The or qualitative methodologies. The studies dealt with diverse African studies were qualitative studies, addressing issues such target groups. The minimum sample included participants as health communication in Ethiopia (Worku, ) and the use at a women-led farming cooperative in Lesotho who were of mobile phones for agricultural purposes in Lesotho. provided with mobile phones (Vincent and Cull, ), while For media development practicioners, academic studies the maximum number of respondents included representa- can be a source of information as well as an inspiration for in- tive samples from countries (Salem, Mourtada and Alshaer, novative research techniques. Our review found that universi-

12 Audience research in Media Development

ties tend to pick up on new trends of media usage faster than BBC Media Action not only conducts more audience research media development organizations, perhaps because they are than any other media development organization, relatively purely interested in research and the research is rarely asso- speaking, but also regularly tests innovative methods that ciated with media development interventions. Depending on go beyond the standard methods repertoire in the context of the country and academic tradition (especially in terms of em- media development related audience research. Examples of pirical research), academic institutions can be a good partner the past (and present) include, but are not limited to: for media development practitioners. This ensures that the – Most signiﬁcant change technique: Form of participa- academic research done is useful for the project at hand. tory monitoring and evaluation, involving participants Of course, not every academic journal is of interest to in data analysis and the selection of sorts of change to be people involved in media development. Relevant articles are recorded in order to document (unanticipated) impacts rare in top academic journals such as the Journal of Com- (employed, for example, in Myanmar); munication, Public Opinion Quarterly, or New Media and – Participatory video technique: Documentation of evalu- Society. Journals focusing on media development or global ation, empowering participants by letting them tell their communication (e.g., Media Development Journal, Journal stories in more personal ways by making their own videos of Development and Communication Studies, Information (employed, for example, in Myanmar); Technologies & International Development, Global Media – Community observation (transect) walks: Guided walks Journal) are quite small but can contain topics of interest through rural or urban dwelling by an expert (such as a to the practitioner. Finally, journals that focus on certain village elder), to help understand the social and physical regions can contain context specic information useful for context of audiences targeted and documented in a tran- justication of media development decisions (e.g., Journal script of the conversation during the walk, the researcher’s of African Media Studies, Asian Journal of Communication, assessment and observations, including a sketch map of Asia Pacic Media Educator, Comunicación Y Medios, Central the vicinity (employed, for example, in Afghanistan); European Journal of Communication). – Propensity score matching: Statistical matching of viewers and non-viewers in program evaluations, to ensure that dierences in knowledge or actions are not due to .. Media development organizations: diversity in self-selection or other interverning factors but due to the quantity and quality program itself (employed, for example, in Kenya); – Randomized controlled trials: Experimental designs, Media development organizations have a specic geographic comparing a treatment and a control group to nd and/or topical focus (e.g., journalism training, media educa- out how eective programs are at conveying messages tion, government consultancy) that requires matching audi- and in¤uencing attitudes (employed, for example, in ence research methods. Oen, the resources available for au- Bangladesh); dience research are limited, which means that the scale and – Structural equation modelling: Measuring the reciprocal scope of audience research are smaller on average as com- eect of factors of in¤uence in a predened setting on pared, for instance, to CIBAR research. This, however, not only each other – mostly employed to measure the reciprocal has to do with limits in funding but also with the fact that eects of media message and audience properties on the more detailed information is sought on specic target audi- eectiveness of media messages (employed, for example, ences in specic locations or regions, information that surveys in Bangladesh). at the national level are not always able to provide. A notable exception is BBC Media Action, whose research division regularly conducts studies that rival CIBAR research in the sheer scale of the research. An example is the Climate Asia Project, focusing on environmental/climate issues and communication and based on a sample of over , interviews conducted in seven countries (BBC Media Action, ). Of course, doing research does not automatically mean it gets published. For instance, Geneva-based Fondation Hirondelle is quite active in evaluating projects by means of audience research (Lechien, ) but the results rarely are made known to the wider public.

The surveys were conducted in Bangladesh, China, India, Indonesia, Nepal, Pakistan and Vietnam.

Edition DW Akademie 13 An important aspect of audience research by media develop- Sample sizes of quantitative surveys vary according to ment organizations is monitoring and evaluation. Audience purpose. Nationally representative studies are oen based on research frequently yields baseline and endline data for such more than a , respondents, while project-specic surveys projects, sometimes also mid-term data. The underlying ra- of smaller target groups in certain locations can have samples tionale generally is that changes in knowledge, attitudes, and of less than , or even less than respondents. The Au- behavior can be attributed to the media development inter- dienceScapes samples, for example, range from , to , ventions. But organizations go about nding proof of out- respondents. In contrast, two baseline surveys conducted by come and impact in various ways, audience research being Search for Common Ground in Pakistan (Search for Common only one of them. Ground, a) and Nepal (Search for Common Ground, b) An evaluation study under the auspices of Washington- in had sample sizes of and ° people respectively. based NGO Search for Common Ground, for example, mea- Since telephone based samples are seldom an option, most sured the eects of the radio and television program “The surveys are conducted as face-to-face interviews. The Afroba- Team” by asking respondents whether they had the impression rometer surveys, currently conducted in African countries, that their conduct with people of other ethnic backgrounds including various political and development indicators (see had changed (Abdalla, ). There are a couple of problems for example question on media use in the Ugandan survey with this way of going about measuring impact. Besides the Afrobarometer, ), are generally set at , respondents. fact that apparently no baseline was conducted to measure Sample sizes for guided interviews oen depend on the level and quality of interaction between people of dier- whether a mixed methods approach is chosen or whether the ent backgrounds before they had listened to or seen the show, qualitative interviews are meant as a stand-alone method. The there is the problem of social desirability (compare Carlson, same applies to focus groups. In combination with quantita- ). Ethnic bias and "tribalism" are o¢cially frowned upon tive surveys, samples of less than in-depth interview part- in Kenya, so respondents were bound to answer accordingly. ners and less than ten focus groups are quite common. In this In comparison, a study funded by American development or- case, the qualitative methods are mostly used to illuminate ganization USAid and conducted by Internews in Kenya used the reasons behind the distribution of media usage or evalu- a baseline and an endline to document that the proportion ation patterns found in larger scale surveys. For instance, East of the population that had understood the constitutional re- African media development organization Panos Grands Lacs forms had increased by percent (McCallum, ). used focus groups and in-depth interviews to complement a It is, however, interesting to note that we did not come large scale survey on media use in Burundi (Frère, ). As across many experimental or quasi-experimental designs, a stand-alone method, in-depth interviews can reach maxi- or randomized treatment and control designs in our sample, mum sample sizes of and more while the number of focus although these are considered the most eective means for groups were found to range from to . The purpose of stand- measuring impact (Leeuw/Vaessen, ±). Exceptions include alone qualitative methods is to nd out about the context of a large scale cluster-randomized design, realized by BBC Media media use and information needs, linking both aspects to the Action in Burkina Faso (Head et al., ). daily lives of the beneciaries of specic media development Most media development organizations employ quantita- projects. A good example is a project done by a consortium of tive methods for audience research more oen than qualita- development organizations headed by the United Nations Of- tive ones. These are predominantly quantitative surveys (± ce for the Coordination of Humanitarian Aairs (UNOCHA), percent), followed by focus groups (± percent), and qualitative in which focus groups were conducted in three cities in Iraq interviews ( percent). An example of large scale quantitative to gain in-depth insights into the information and communi- studies was Intermedia's AudienceScapes research program. cation needs of refugees (Quintanilla et al., ). Stand-alone The program included publicly accessible data on media use qualitative methods are oen employed in regions with secu- from four African countries (Ghana, Kenya, Tanzania, and Zam- rity issues such as Syria or Somalia. bia), before being taken o¼ine in because the studies had Unfortunately, many studies do not report openly what been discontinued. It is interesting to note that BBC Media Ac- precise sampling techniques are used. Representative stud- tion, though famous for large-scale quantitative research, in ies require random samples and most large scale quantitative fact utilizes qualitative methods more oen than quantitative samples try to get as close to random samples as they can. It ones. A content analysis, conducted by us, of research sum- is no secret that in many countries, geographical and security maries published until April revealed that ° percent were constraints prevent every citizen from having the same like- based on qualitative, percent were based on quantitative lihood of getting into the actual sample. The most common methods and percent were based on a mixed-methods-de- sampling technique is dividing countries or regions up into sign. This emphasizes that qualitative methods not only have quadrants or provinces (stratication – stratied sample), se- their merit but can produce thorough insights into audience lecting a random number of villages, towns, or cities within behavior and motivation. each quadrant or province (clusters – cluster sample) and then randomly selecting individual participants from those

14 Audience research in Media Development

selected clusters (randomization – random sample). However, . Conclusions convenience samples ignoring the principles of random sampling, are quite common. For instance, a survey under the aus- Considerable eorts have been invested into audience research pices of Internews in the three largest cities in Chad saw inter- over the course of the past six years. Most studies are aimed viewers requesting every third person they met on the streets at assessing the needs of beneciaries or measuring the reach of the cities ( respondents per city) to take part (Samuel and impact of media development activities. A great deal of re- and Bakirdjian, ). This amounts to a convenience sample search follows the same routines, asking media users to detail with no possibility of making any inference to the population the way they use media, testing their knowledge, requesting as a whole. If random sampling is not an option, quota sam- their topic interests and information needs, and asking how pling is oen seen as a sensible alternative, but many authors they judge their local media. Innovative approaches are rare – doubt the results of quota sampling studies are generalizable. with the exception of BBC Media Action (see text box on page Over the past couple of years two trends in audience re- ). Specically, most audience research is based on the expo- search amongst media development organizations can be dis- sure rather than the engagement model of audience research. cerned, one concerning the media focused on and the other Digital data available from various internet sources for analy- concerning what target audiences are studied. While Internet sis are seldom used. Furthermore, samples tend to be biased use has received considerable attention over the past decade, toward adult audiences, while children and adolescents do recent studies also focus on mobile and smartphone usage. not feature very prominently in research, though they make This trend is illustrated by a survey on feature phone use up a signicant proportion of the population in developing in Ethiopia, Ghana, Kenya, Nigeria, and South Africa, conduct- countries. Mixed methods, oen professed in theory, are sel- ed by the telecommunication consultants at Balancing Act dom found in practice. Tests and quasi-experimental designs, (). Nevertheless, the focus of most of the studies done by quite common in social scientic research, are almost absent media development organizations is still on radio and televi- in audience research in developing countries. While measur- sion use, with newspapers, magazines, and digital media playing impact is oen professed in word, many studies do not go ing only a marginal role. the extra mile to draw the methodological consequences for Regarding the second trend pertaining to target audiences, achieving it. Simply measuring the reach of an intervention or refugees and their information needs have increasingly been asking people whether a program was good or had an impact, focused on, no doubt as a consequence of the growing inten- is not enough. sity of con¤icts, in Syria, South Sudan, and other regions. US Some of these decits will be addressed by the following consultancy Forcier Consulting has, for instance, done sur- three studies, conducted by DW Akademie during the course veys in cooperation with Internews in several refugee camps of and designed to further our knowledge and repertoire in South Sudan, nding out what information needs refugees of innovative methods as well as enabling intervention-based have and which sources of information they use (e.g., Forcier, insights to be used for DW Akademie’s activities in the coun- ). And the Danish media development organization Inter- tries they were conducted in. Thus the regional and thematic national Media Support () published a study on the hu- specics were not only determined by gaps identied in cur- manitarian information needs of Syrian refugees. rent research, but also by the shape and the needs of specic Geographically, there is a denite focus of media develop- media development projects currently underway. The research ment organizations' audience research on sub-Saharan Africa. was meant to supplement country specic work in the eld, More than half of the BBC Media Action research summaries while making sure that there was a certain regional and the- as of April referred to a sub-Saharan African country. matic diversity. The following three chapters present the three Studies from this region also accounted for over half of media case studies conducted. development organizations' studies in our sample. Because of recent developments such as the Arab Spring revolutions and the Syrian con¤ict, the Middle East and North African region has also been in the limelight of audience research studies in recent years. Other regions, including Latin America, Asia, Oceania, or Eastern Europe, are focused on considerably less.

Edition DW Akademie 15 . Case Study : Critical media and knowing how media work, and demanding good quality, inde- information literacy: testing a test in the pendent media (Reineck and Lublinski, : ). In this sense, Palestinian Territories MIL is the set of skills and attitudes citizens need to make informed decisions in their everyday lives and to take advantage of their rights to access of information and freedom of expres- Dennis Reineck sion. Furthering MIL in developing countries thus has repercussions not only for the awareness of these rights but also for Case study fact sheet the degree to which these rights can be put into practice. Goals of the case study: To test the viability of a Media and In- One of the challenges of media education in general and formation Literacy (MIL) Test and to evaluate MIL related work MIL oriented work in particular is evaluating its outcome and in a development project context in the Palestinian Territories. impact. As opposed to common literacy (reading and writing), MIL requires a more complex measurement tool because it is Research questions: a composite of very dierent competencies. For instance, an – Do the project schools obtain better results than the non- MIL study commissioned by the European Commission in- project schools? cluded ± indicators measuring various aspects of MIL (EAVI – Does gender have an in¤uence on the results obtained in and DTI, : ). This kind of complexity is oen not viable the MIL test? and can produce too much data, preventing unitary interpre- – Does the test for measuring Critical Media and Information tation of the results (Bulger, ). Rather than trying to har- Literacy work? ness the overall complexity of MIL in a single tool, monitoring – What is the relationship between a MIL skills test and ask- and evaluation of media development projects should focus ing participants direct MIL skills questions? on the aspects of MIL specically relevant to the case at hand. This pragmatic approach requires identifying which elds of Method: Test Survey MIL are relevant for a specic target group in a specic regional, national, and/or cultural context and aiming to measure Sample: ° participants aged to years of age from six MIL prociency in these elds. schools in the Palestinian West Bank The following is a case study involving an MIL measurement tool tested in the Palestinian Territories. Media literacy Innovative aspect/approach: Using a skill and competency education in the Arab region “remains in its infancy strug- test rather than a conventional abstract questionnaire based gling to a¢rm its importance” (Melki, : ). The Palestin- survey ian Territories (PT) are a special case, since the Israeli-Pales- tinian con¤ict has had and continues to have specic eects Challenges: Finding Arabic media examples and developing a on how media and other information sources report on cur- questionnaire suited to the age group rent aairs. From ± to ±±, Israeli authorities exercised tight censorship over Palestinian media in the West Bank and Main results: Gaza Strip (Bishara, : ). Aer the Oslo Agreement, for – The validity of the MIL test is higher than that of conven- the rst time Palestinian media companies were allowed to tional MIL surveys. operate in the PT. However, the Palestinian Authority and the – Project schools obtained slightly better results than non- dominant Fatah and Hamas political factions have had con- project schools. siderable in¤uence on the media (Berger, : ). Therefore – Gender has a strong in¤uence on test performance, with it is not surprising that a BBC Media Action survey in girls faring much better than boys. found that the Palestinian participants “expressed a strong desire for trustworthy programming covering local issues” (BBC Media Action, ). Beyond the national media, the young generation is espe- . Introduction cially keen on international Arab satellite television and social media. Though a great amount of information spread via so- Media and Information Literacy (MIL) is a key concept in me- cial media is unveried and the web has been a preferred ve- dia development. According to UNESCO, “the cultivation of a hicle for spreading hate speech and propaganda (for example media- and information-literate population is essential for the by IS in Iraq and Syria), a lot of Palestinian youngsters consider sustainable development of society, requiring the individual the social media to be trustworthy (Studio Social, ), which person, community, and nation at large to obtain a diverse provides a further argument for the necessity of MIL eorts range of competencies” (UNESCO, : ). These compe- in the PT. It is the critical, evaluative dimension of MIL that is tencies include abilities to access information, to use, evalu- relevant here, since media users need to be able to gauge and ate and create media and information, participate in media, judge the quality of information provided. This enables them

16 Audience research in Media Development

to evaluate the media according to certain criteria (e.g., source questions. It is only when tests and abstract questions correlate diversity, independence, transparency) and demand media to highly, that replacing the test by abstract questions makes sense. conform to these standards. Local NGO Pyalara has been working in the eld of MIL in the Palestinian Territories since ±± and DW Akademie . Research methodology launched long-term MIL projects in cooperation with Pyalara in at four boys’ and four girls’ project schools in the West To measure MIL realistically, a test was drawn up, based on pre- Bank. It is in the context of these activities that the proposed vious research on the MIL of children and adolescents (Sowka tool for measuring MIL to be presented here was developed. et al., ; Arke and Primack, ±). The test consisted of ten assignments rather than abstract questions and each MIL skill Research questions of this study, which features a test for Criti- was tested using a two-step-procedure. Oen, MIL tests directly cal Media and Information Literacy, are as follows: ask participants to evaluate certain properties of media (e.g., Arke and Primack, ±). The test applied here rst seeks to nd RQ: Do the project schools obtain better results than the non- out whether the participants recognize the media properties project schools? at all and only then requests them to evaluate the properties. To illustrate how the test is analyzed and produces results, the This prevents the test from producing non-opinions, i.e., opin- research question at the heart of the MIL test in the Palestinian ions that are voiced without the participants understanding Territories is addressed, namely whether the schools involved what they are being asked to judge. A further innovation was in DW Akademie’s MIL project fare better than their non-proj- introduced by comparing the skills of participants who had ect counterparts. Thus, research question is concerned with taken part in MIL classes with ones who had not. Such quasi- the results of the MIL measurement. experimental methods are seldom found, though these sorts of designs are of particular use in measuring the eectiveness of RQ: Does gender have an inuence on the results obtained in development projects (Leeuw and Vaessens, ±: ). the MIL test? Thus, the rationale of the test was to rst nd out whether The second research question deals with the gender of the partic- the participants actually recognize what is wrong (or right) ipants in the MIL test. While it is possible that male participants with a news item and then to nd out if they evaluate it ac- had preferential access to technical equipment such as smart- cordingly. An example: A news item only makes reference to phones, project partners PYALARA reported girls doing better one source. The rst question would be: Do the participants and being more disciplined at MIL activities. A clear hypothesis notice that only one source is referenced? And the second one: could thus not be derived and it remains to be seen whether gen- If they notice this, do they evaluate it negatively, showing that der plays any role at all in the distribution of levels of MIL. they are aware that source diversity is a good thing, or don’t they see the problem? Whether the youth recognize and evalu- RQ: Does the test for measuring Critical Media and Informa- ate media correctly was tested for the following aspects of me- tion Literacy work? dia and reporting: The third aim of the study is to test the tool. If it is found to be of use, the test will most probably be transferable to dier- – Product placement: Do participants recognize product ent contexts, i.e., dierent countries and dierent media users. placement and evaluate it negatively? Two criteria for a good test are applied here: objectivity and – Balanced reporting: Do participants recognize balanced validity. Reliability could not be measured here since methods reporting and evaluate it positively? such as test-re-test – testing the same skills with the same par- – Independent reporting: Do participants recognize political ticipants again aer a short interval – were not a viable option dependency in reporting and evaluate it negatively? due to time and organizational constraints. – Transparent communication: Do participants recognize insu¢cient source transparency and evaluate it negatively? RQ: What is the relationship between the skills test and direct – Relevance of reporting: Do participants recognize topics of skill questions? low importance to society and evaluate them negatively? The test to be described below requires participants to use and judge media in much the same way they do on a daily basis. The more straightforward, easier alternative would be simply to directly ask the participants about these properties. If the test ¡ The competencies were derived from previous tests (esp. Sowka et al., ¢ ¡) items and the abstract questions were to correlate highly, this and literature on the quality of journalism (e. g. Arnold, ¢¢£). Thus the would mean that the test would be super¤uous because the skills tested certainly are strongly oriented towards Western standards of abstract question would be just as capable of measuring an MIL journalism. It would be worthwhile to research to what extent di erent cul- skill as the test. As Eszter Hargittai () has pointed out, the tural values can lead to di erent quality criteria and which quality stan- test of actual MIL skills always has a greater validity than abstract dards can be regarded as universal.

Edition DW Akademie 17 – Topic diversity: Do participants recognize topic diversity in The sample of the study consisted of ° youngsters aged reporting and evaluate it positively? to years from six schools in the Palestinian West Bank. – Opinion diversity: Do participants recognize lack of ± were recruited from schools that were involved in extra- opinion diversity in reporting and evaluate it negatively? curricular MIL classes administered by partner organization – Source diversity: Do participants recognize lack of source PYALARA under the auspices of DW Akademie and were diversity in reporting and evaluate it negatively? from non-project schools. There were girls and boys. The – Respect of privacy: Do participants recognize reporting pretest in June of found that the test was too “grown-up” that invades people’s privacy and evaluate it negatively? for the pupils to understand (Reineck, ). Several media ex- – Protecting sources: Do participants recognize reporters’ amples were exchanged for more age conform ones, the lan- right to withhold information on sources and evaluate this guage was simplied and comic gures and a coherent story positively? line were introduced to lead the pupils through the questionnaire. Aer these alterations, the actual test took place from ± to September at a school in the West Bank village of Jaba’.

. Findings

RQ: Do the project schools obtain better results than the non- project schools? The rst question compares the results of the project and non- project schools with each other. The MIL test was developed precisely to enable this kind of comparison between the participants who hadn’t taken part in extracurricular MIL activities and the ± participants who had. A direct comparison between the average scores of pupils from project schools and non-project schools would suggest that project school participants have a higher critical MIL prociency than non-project scholars taking part in the test. The project school pupils Figure : Excerpt of the Critical MIL Test questionnaire scored an average of . points, while the non-project school participants obtained .° points on average. The best schol- ar, receiving out of points, was from a project school. So In order to test to what extent the participants actually rec- were the top seven participants. On the other hand, the worst ognize and evaluate these aspects, as they should according participant was from a project school as well. The dierences to MIL theory, ten examples of television, radio, or online re- were not statistically signicant (p > .). This means that the porting, Facebook posts, YouTube videos and scenarios were mean scores of the project and non-project scholars are not collected and/or produced (see questionnaire in annex). The su¢ciently dierent to rule out they came about by chance. procedure then was to let the participants view an example There is a dierence between the scores, but it is quite small. and consequently let them answer the corresponding items in A closer look at the ten skills shows that the project school the questionnaire before going on to the next example. The participants fared better at seven of the ten skills. Two skills, sum of the responses was then added up to a Critical Media however, were practiced slightly better (recognizing and con- and Information Literacy (CMIL) index score, since the skills doning source transparency, recognizing and sanctioning vio- tested were specically chosen to measure the critical, evalua- lations of the private sphere) and one was practiced consider- tive aspects of MIL. The maximum score was , the minimum ably better (recognizing and sanctioning source bias) by the points. It is important to stress that the test did not measure non-project pupils (see Fig. ). MIL as a whole, but one aspect of MIL, namely critical analy- The results seem to imply that indeed the MIL training for sis and evaluation of media and information sources. Critical the pupils has produced small comparative benets in select- analysis and evaluation of media and information sources is ed areas. The project schools fared slightly better on the whole especially relevant in the Palestinian Territories because of the compared to their non-project counterparts. One possible specic slant of the media in connection with the Israeli-Pales- reason is that MIL courses can only achieve so much improve- tinian con¤ict and the strong a¢liation of the media to certain ment and that there are limits to the degree to which MIL activ- political and religious factions. Also, the test did not measure ities can bolster pupils’ achievements. Education systems and a unitary scale, i.e., it was not assumed, that all skills measured general levels of education, media systems and distribution of the same underlying construct. Rather, each skill measured a media technologies all have an in¤uence on MIL performance dierent aspect of CMIL. and these factors transcend what pupils are taught in MIL

18 Audience research in Media Development

Figure : MIL Test Scores of Project and Non-Project Participants in Comparison

2,0 Project School

1,7 Non-Project School 1,6 1,5

1,6 1,5 1,4 1,3 1,3 1,2 1,2 1,1 1,2 1,2 1 1,0 1 0,9 1,0 0,9

0,8 0,7 0,7 0,6

0,4

Project Opinion Source Balanced Relevance Privacy Diversity Diversity Respect of ProtectingSources Placement Reporting Transparency Independence Topic Diversity

Note: Maximum score per skill of , minimum score of

courses. This explains why the scores for the skills are quite the groups were highly signicant (p<.). The girls obtained similar for both project and non-project candidates. Also, the better results than the boys on all ten skills, with big dier- participants come from di¢cult socio-political backgrounds, ences being registered for journalists’ respect of privacy, the stemming from underprivileged and politically volatile areas relevance of topics, and opinion diversity (see g.). of the West Bank. Therefore, there are also contextually deter- Considerably more boys did not see anything wrong in pa- mined limits to the scores students would be able to achieve parazzi taking photos of the children of famous people, found under these circumstances. Additionally, the MIL classes were trivial news to be relevant, and did not recognize a biased re- mainly concerned with producing media products such as port that only cited opinions in line with the journalist’s own wallpaper magazines or intercom reports. There was less of a views. These might be departure points for future MIL courses. focus on critical media analysis. It cannot, however, be ruled out that the dierences be- RQ: Does the test for measuring Media and Information Lit- tween the project and non-project schools already existed be- eracy work? fore they started learning MIL, or that they can be attributed to The third research question addressed the viability of the tool general levels of education rather than the specic MIL cours- for measuring MIL. Power shortages and safety issues due to es. To rule this out, pupils obtaining MIL tuition would have unrest at the time the test was conducted in Jaba’ hampered had to be chosen at random from all schools. This was not done the execution of the main survey. However, objectivity was for logistical and organizational reasons. But the limitation has ensured by reverting to paper and pencil instead of the origi- to be borne in mind, so as not to overinterpret the results. nal online questionnaires for all participants. Furthermore, all tests were conducted at the same venue with participants RQ: Does gender have an inuence on the results obtained in being transported to and from the school by shuttle busses. the MIL test? Research question addresses gender dierences in the MIL ¤ However, the results were indeed signicant for female participants (see test results. female and male participants took part in the footnote ¦). test. The girls obtained considerably better results than the ¦ Interestingly, while the girls that took part in the MIL courses had signi- boys. The average score of the girls was ., as opposed to the cantly higher results than their non-project counterparts ( .¤¦ vs. .£¢), boys’ average of ±.°. The best score of out of points was the same is not true for the project school boys compared to the non-project achieved by a female participant, while the lowest score of boys. The project boys in fact fared slightly worse than boys who hadn’t re- out of points was reached by a boy. The dierences between ceived MIL education (£.¨ vs. £.¨¤).

Edition DW Akademie 19 Figure : MIL Test Scores of Female and Male Participants in Comparison

2,0 1,8 1,8 Boy Girls 1,6 1,5

1,6 1,3 1,5 1,3 1,1 1,2 1,2 1,2 1 1 1,1 1 1 0,9 0,8 0,8

0,8 0,7 0,4 0,4

Project Opinion Source Balanced Relevance Privacy Diversity Diversity Respect of ProtectingSources Placement Reporting Transparency Independence Topic Diversity

Note: Maximum score per skill of , minimum score of

Table : Correlations between skill scores in the test and skill scores with direct questions The media examples were shown on laptops and read aloud to all participants to ensure that everyone knew the examples before being asked to judge them. The instructors had detailed Product Placement ¿.ÀÁ written manuscripts and were told not to assist the participants in any way (except for technical assistance), to prevent Balanced Reporting ¿.ÂÃ them from prompting the correct answers. The data was analyzed in a unitary manner throughout. All these measures Dependence ¿.À¿ contributed to the objectivity of the results. Validity and viability of the MIL test was ensured by making sure that experienced project partners PYALARA judged Intransparency -¿.¿Ä the media examples and questionnaire items as suitable for the age group (face validity), that the skills were independent Irrelevance ¿.¿Å of each other, which was tested with Cronbach’s Alpha and intercorrelations between the dierent skill items (construct Topic Diversity ¿.ÅÂ validity), that the variance of test results across participants was big enough for each item, and that the distribution of dif- culty across the items was even (viability). Only recognizing Opinion Diversity ¿.¿¿ and evaluating product placement turned out to have quite low variance, and there were slightly easier than di¢cult ques- Source Diversity ¿.¿Á tions in the questionnaire. The detailed results can be found in the annex. Overall, however, the results suggest that the test Respect of Privacy ¿.¿Á for measuring CMIL is useful.

RQ: What is the relationship between the skills test and Source Protection ¿.¿Â direct skill questions?

If test assignment items and abstract survey questions were to Note: Bivariate Correlations between items of skills test and direct skill correlate perfectly, this would mean that they measure the ex- questions (Pearson’s r); = perfect connection; = no connection; - = act same thing. In this case, the test could be discarded because perfect inverse connection

20 Audience research in Media Development

it would be much easier to simply ask the participants directly and including more di¢cult ones to obtain an even distribu- about their skills/attitudes. If, on the other hand, the correla- tion of di¢culty. Also, the reliability of the MIL test should tion is low (less than .), the ability of the direct questions to be put to the test, for example by running the test more than measure corresponding MIL skills/attitudes would also be low. once with the same participants. In this case, the test would be an absolute necessity since how a question about balanced reporting, for instance, is answered Recommendations for MIL practitioners in the Palestinian would have nothing to do with whether balanced reporting is Territories detected and condoned in practice. – Future MIL eorts might want to focus on issues of source As it turns out, balanced reporting is the only test skill transparency, source diversity, and respect of privacy, issues that is predicted well by the responses to the direct questions which project school pupils fared worst at. This is especially in the questionnaire. All other direct questions are not ca- true for conveying the importance of having dierent sourc- pable of predicting whether the participants are able to iden- es in news reports. tify and evaluate the corresponding examples of bad or good – Since the scores also varied from school to school, a closer journalism. This result shows that asking young persons look at deciencies at the school level should give additional to years of age questions about good or bad journalism insights into which areas should be addressed. ° instead of letting them watch/read/hear journalism “hands – One consequence of the gender gap in MIL levels should on” and then letting them judge it is, for the most part, not a be to intensify MIL eorts at boys’ schools. The strongest viable option. Young media users require methods that are deciencies were found pertaining to the relevance of news suited to their capabilities and these are practical, not ab- and the respect of privacy. This might be the focus of fur- stract empirical methods. ther MIL courses. – MIL courses at the project schools have been focused mainly on producing own media like wallpaper magazines or in- . Conclusions tercom formats. The results suggest that more attention should be paid to critical analysis of media with methods The Critical Media and Information Literacy (CMIL) Test found such as comparisons of the perspectives of dierent media that participants who had had some form of MIL training on the same event. fared slightly better than participants that had not. This was to be expected and can be regarded as a success for the MIL Recommendations for researchers project of DW Akademie, at least for the female participants, – The comparison between direct questions and the MIL test who fared signicantly better than their non-project compa- shows conducting tests is worth the eort. Children and triates. The male participants from project schools were only teens apparently need support to understand what they are on a par with the boys from non-project schools. In this sense, being asked to do and the best way of giving them support is the results should be used to probe dierent methods of con- by constructing a test that connects with and draws on their veying MIL, focusing on gender-specic requirements. Males everyday media consumption. fared considerably worse than female pupils in every category, – The test requires media examples that are suitable for the though this could also be attributed to general educational age group and that provide valid examples for testing the and developmental dierences between girls and boys at this appropriate skills. At the same time, the skills have to be easy age. The only skills boys were almost as good at as girls was to measure. This implies simplifying the skills (and the an- spotting and evaluating product placement and recognizing swer possibilities) involved to get comparable results. and evaluating the transparency of Facebook postings. Proj- – The pretest should be used to make sure that the di¢cul- ect partners PYALARA informed the researchers that female ty of the items is distributed evenly. Testing ten skills was pupils generally were obtaining better results at school than found to be compatible with the concentration capacity of boys, suggesting that MIL performance might be highly cor- young participants of an average age of . With older tar- related with general school performance. It was also reported get groups, more – and more sophisticated – skills could be that the results were in line with their impression that girls’ tested, including for instance issues of online privacy and MIL knowledge and skills proved to be more extensive than data security. boys’ in the MIL courses themselves. Two research questions addressed the viability and ap- propriateness of the MIL test which consisted of twenty items paired together to measure ten CMIL skills. The test was found to be viable for measuring MIL. Skills were found to be independent of each other and almost all skills in the test could discriminate high MIL scorers from low ones quite well. The ¨ Results were also analyzed at the school level, but are not included in this test could be optimized by reducing the number of easy skills chapter for reasons of brevity and relevance.

Edition DW Akademie 21 – Tests should restrict themselves to the aspects of MIL relevant to the intervention context. The MIL test presented here focused on Critical MIL. MIL consists of a number of other aspects that could be included in other contexts depending on target groups and media landscapes. – MIL tests should be used to draw relative comparisons, not direct comparisons across MIL skills. Only comparing the results of MIL trained participants with those of people without MIL training can reveal something about how well each group fared in the MIL test.

In summary, the MIL test has proved useful for studying Media and Information Literacy competencies. The results should be taken as an opportunity for improving and rening the methods for conveying MIL, especially at boys’ schools. Future research will have to focus on testing the reliability of the tool, applying it to dierent settings, populations, and testing other sorts of MIL related knowledge and skills. For media development organizations, the results hold two important insights. On the positive side, Media and Infor- mation Literacy is measurable. If relevant competencies are identied, the research design presented here can be adapted to the context of other cultural and national settings and the quantitative results can serve as reliable and valid indicator measurements. On the challenging side, a test is more di¢cult to construct than a standard survey questionnaire. Close cooperation with project partners is essential in choosing good media examples and making sure the technical equipment and logistics are in place. The budget for the study did not exceed , Euros, making it a relatively cost eective way of measuring, for instance, baselines for Media and Information Literacy projects.

22 Audience research in Media Development

. Case Study : Impacts of listening to constitutional land inheritance requirements. The program to radio. The “Land Rights and Women” seeks to inform, educate, and mobilize them to change their radio program at Radio Nam Lolwe behavior on land issues. Therefore, radio may be playing an important role in the acquisition of land knowledge. in Kisumu, Kenya The question of media eects is highly relevant for media development organizations. Both to know and then improve Christoph Spurk, Charles Nyambuga the way they work and to justify their work to donors and the general public. One potential impact of media support is that Case study fact sheet the target groups of specic media start to gain in knowledge, Goal of the case study: To measure the impact of a Kenyan to change their opinions or attitudes and to nally start taking radio program on land rights and women on the knowledge, action in various ways, as inspired by the improved reporting opinion, and actions of listeners. or media coverage. It is evident that media coverage cannot be the only factor triggering those changes, but it is important Research question: to get to know the contribution by media in concert with all – What are the dierences between the treatment group other in¤uences. These intervening factors include but are not (Radio Nam Lolwe listeners to the LRP series) and the con- limited to the education sector, provincial administration, and trol group (non-listeners) regarding knowledge, opinion and the religious sectors. activation levels? In the literature there are several indications that radio has an in¤uence in Africa. Tsegyu and Asemah () assert that Method: Quasi-experimental design with quantitative survey community radio broadcasting can be used to bring about positive attitudinal change in rural areas. Its programming Sample: (potential) listeners of Nam Lolwe radio station in re¤ects local interests and contributes towards the develop- Kisumu and Migori districts ment of rural women by educating them about banking, child health, and government policies among other issues. Innovative aspect/approach: Quasi-experimental design com- There are also authors that come to the conclusion that paring (potential) listeners and non-listeners exposure to the media can in¤uence citizens’ involvement in various forms of electoral participation. Women’s expo- Challenges: Identifying and sampling comparable groups; re- sure to the media has the potential to enable them to acquire mote management of audience research information on land rights which might lead to more interpersonal discussion and ultimately to practice in accordance Main results: with the new land inheritance regulations. Prior studies have – It is of utmost importance that media development organi- established that exposure to the media promotes individuals’ zations monitor the content and its quality. This is the pre- engagement in interpersonal discussions (Norris, ; Tian, condition to achieve impact. ). These discussions facilitate participation by providing – Personal communication and radio were the main sources land information and triggering interest among listeners (Eve- of information on land rights and women. land and Scheufele, ). Further, Isaksson, Kotsadam and – Age has a signicant in¤uence on people’s attitudes towards Nerman () indicate that past studies in Africa have estab- land rights and women. lished that access to information which is based on radio own- – No signicant dierences in levels of knowledge, opinions, ership is positively related to behavior. and activation levels were found due to listening to the Land Radio is the most popular mass medium used by Kenyans Rights Programme. (Schmidt, ). Their usage of radio has transcended age, education level, or rural/urban context. Local radio stations have specically mushroomed in the last ten years, oen broadcasting in vernacular languages and thus attracting new audiences . Introduction at the local level. The number of households owning radio sets in Kenya has increased at a fast rate with the calculated This study seeks to analyze the role of Kenyan radio in in¤u- percentages showing an increase from percent owning ra- encing listeners to change their perspectives on an emotion- dio sets compared to percent owning TV sets in to ± ally charged topic: the land question and the related rights of women. The aim is to analyze what eect radio messages may have on attitude, knowledge, and behavior. £ As stated in the overview in chapter , quasi-experimental designs are The research is based on the premise that Nam Lolwe radio, rare in media development. BBC Media Action has recently employed full- a radio station based in the West Kenyan town of Kisumu, has edged experimental designs in Burkina Faso (see Head et al., ¢ ¡) and broadcast special land programs that seek to expose listeners Bangladesh.

Edition DW Akademie 23 percent of households owning radios sets against percent . Research methodology owning TV sets in (Kenya National Bureau of Statistics, ). More than any other mass communication medium, ra- The method used in this research was not completely new. It dio oen transmits its programs in the language and accent of was a questionnaire which comprised of closed and open-end- its community (Odhiambo, ). ed questions regarding: These stations have the potential of triggering debates on – socio-economic characteristics of interviewees political and social issues, enhancing participation of ordinary – media and especially radio use people, and thus fostering democratic space. This way they can – sources of information regarding the question of land rights work as an instrument for strengthening the human rights – practice of land rights and inheritance based approach to development, providing rights holders with – knowledge of land rights relevant information and thus empowering them to bring – attitudes on land rights about societal change. However, the level of professionalism – activities regarding land rights. in these radios is still limited, and since DW Akademie In order to nd out about the impact of the radio program on supports ten local radio stations in strengthening both profes- land rights the project applied a comparison between a treat- sionalism and economic sustainability. Professional decien- ment and a control group. Radio Nam Lolwe is one of the three cy has been particularly seen in the quality of their programs most listened to radio stations in (former) Nyanza province, and also in the presentation of the same. the area close to Lake Victoria in Western Kenya (Spurk, ). DW Akademie was interested in knowing whether the im- The project took advantage of the fact that Radio Nam Lolwe pact of a radio station’s reporting can be demonstrated by im- has an excellent technical receivership in almost all areas plementing concrete measures. Therefore, exploratory research kilometers around Kisumu, Kenya’s third largest city, where was undertaken with a view of determining those potential me- Nam Lolwe is situated, but cannot be received in a specic dia eects. It used one specic radio program/series, broadcast area (a depression where the signal cannot be transmitted to) by one radio station in the Western part of Kenya, namely the close to the small town of Migori. This area was expected to “Land Rights and Women” Program (LRP) of Radio Nam Lolwe. be very similar to the Kisumu area in terms of poverty levels, The question of land rights is of utmost importance in Ke- language, and cultural patterns. It is populated by people of nya, as access to land was one of the major causes of violent the same ethnicity as in the Kisumu area, where the same land con¤ict in the country as was witnessed in °. In the inheritance customary laws apply. Kenyan constitution was changed, giving equal rights to wom- Therefore, comparing the dierent parameters of interest en in many areas, but also in land rights. This is of practical rel- (see research questions) in the two areas of Kisumu and Migori evance, as many women still don’t know their rights and lose could yield the contribution the radio broadcasts have had re- access to land when their husbands pass away as – according to garding knowledge, attitude and practice on land rights and Luo customary law – the land then can be claimed by the male women, thus making a contribution to solving the attribution family members of the spouse (Whitehead and Tsikata, ). problem in evaluation. The dierences between the average of listeners and non-listeners in Kisumu and Migori (where there The project was mainly interested in the following research are only non-listeners) is the contribution of Nam Lolwe to the questions: knowledge, attitude, and action levels.± . What are the dierences between the treatment group (Ra- dio Nam Lolwe listeners to the LRP series) and the control To apply this approach, the researchers had to distinguish group (various areas with dierent non-listeners) regarding three groups in the sampling: – Knowledge level on the topic of land rights and women . Listeners of Nam Lolwe’s Land Rights Program (LRP) in the – Opinions and attitudes regarding land rights and women Kisumu area (with good reception) – Activation level regarding land rights (stronger participa- . Non-Listeners of Nam Lolwe’s Land Rights Program (LRP) in tion by listeners?) the Kisumu area . All these items should be dierentiated by gender . Interviewees in Migori area (all non-listeners, but including The underlying assumption of the project was that well- potential listeners) structured and well-presented information on land rights and One might ask why non-listeners in Kisumu are not the “appro- women will signicantly increase the knowledge about land priate” control group. The reason is that listening to a radio sta- rights of those who listen to the program, might change their tion is a matter of choice. Therefore, non-listeners might have opinions on various issues and ultimately lead to some action, other characteristics than the Nam Lolwe listeners that strong- for example in terms of debating with other people, solving ly in¤uence knowledge, attitude, and activity regarding land contentious cases of land inheritance or supporting laws to rights, for example political orientation, traditional attitude, etc. protect land rights of women. So, we cannot say whether a dierence between those groups is only caused by listening to the radio or by other factors. Never- theless, the comparison between those groups can be relevant to

24 Audience research in Media Development

observe the magnitude of dierences and also to compare with for the interview. When the decision to temporarily suspend the dierences between the treatment and control group. the research exercise was taken, the local team in Kenya had al- In both areas the respondents were selected by a combina- ready been in the eld to do interviews. These interviews were tion of a cluster approach and strict randomization within the later used as a pretest and helped tremendously to streamline cluster. In Kisumu, listeners and non-listeners of Nam the questionnaire. Ten questions were deleted that were per- Lolwe and in Migori non-listeners were recruited using the ceived as repetitions and had confused the respondents. random walk and Kish grid sampling techniques (see Annex for The original plan was to conduct the eld phase with in- details). While the inhabitants in both cluster areas were similar, terviews in November at the latest, right aer a period of the Kisumu cluster was closer to a big town while the Migori clus- six weeks of continuous broadcasting about land rights and ter was more rural. The nal sample included respondents. women. Due to administrative di¢culties the research team The questionnaire was developed by Kenyan and Swiss re- in Kenya could only go into the eld in January , which searchers using local knowledge on land rights. Nevertheless, can be seen as a little late. Interviewees might have forgotten the interview questions with regard to knowledge and attitude some information they would have remembered when asked needed to be very specic in order to show that the information earlier. On the other hand this timing ensured that not only in the radio was related to the knowledge of that respondent. short-term media eects were identied. Monitoring of the content produced revealed that the messages being sent out were at rst somewhat confusing and contentious, which is why it was decided to do additional train- . Findings ing before proceeding with data collection. The data already collected were used as a pretest for the main survey. There was In total the sample is comprised of respondents, distrib- no monitoring of the subsequent programs, raising doubts uted as required in the three dierent clusters, and also well about the exact quantity, quality, and content of reporting af- distributed regarding gender. The only mismatch to be noted ter the second training phase. Administrative problems also is that one area in Kisumu (Nyahera) had Nam Lolwe listen- postponed data collection, so that more than six months lay ers (only were required). But the more important number between the rst broadcasts (May ) and the survey (Janu- of listeners to the Land Rights Program (LRP) shows a good dis- ary ). This may have had an adverse eect on the extent to tribution and a good relation between what was planned and which listeners could remember the information. what was realized (see Annex for details). The research team had received recordings of six Nam Lol- In the entire sample, respondents had listened to LRP in we program on land rights, broadcast in Luo in May and June Kisumu while there were in Kisumu who had not listened and produced transcripts in English. It was discovered to the Land Rights Program. In Migori the entire respon- that the information in those broadcasts was rather confus- dents had not listened to the LRP (see table /page ª). ing and contentious, missing a clear and basic description of The importance of personal communication and infor- facts regarding land rights and other laws. Also, experts in talk mation by radio is conrmed in all areas. Family (° percent), shows contradicted each other without the presenter inter- the local chief, and public meetings ( percent) are men- vening or clarifying the issues. The supervision team recom- tioned, but with an almost equal number radio (° percent) is mended stopping the exercise because the relation between mentioned as an important source for information about land the information provided by the radio and potential knowl- and land rights. Migori is a bit dierent with higher shares of edge level of listeners was very weak or non-existent. family and social groups, and less use of radio. It was then decided to conduct an additional training for Nam Lolwe’s radio sta before further data was collected. The objective was to rst and foremost improve the clarity and cor- rectness of information, and only then conduct the eld phase of research, because only with the right information is there a chance of discovering the associations between listening and knowledge or attitude. The additional training was conducted in September , and a new series of programs was planned to be broadcast. The research team only got information about the radio produc- tions already broadcast during the training, but not about any further programs, despite repeated, but in the end, futile eorts ¢ If the information in the radio is rather general the chance is high that the to receive them. This raised some doubts about how many (if knowledge of the person comes from any other source, not from that radio any) programs were nally aired, during which period, and program. what the exact content was. This made it additionally di¢cult For a more detailed description of the problems associated with the study, to develop su¢cient specic knowledge and attitude questions compare the digital annex.

Edition DW Akademie 25 Table : Sample description with regard to listening to LRP

Listener LRP Non-Listener LRP Non-Listener Kisumu Kisumu Migori

Treatment Self-Selected Control

Women Men Women Men Women Men Total Planned Á¿ Á¿ Á¿ Á¿ Á¿ Á¿ Ã¿¿

Realized ÃÁ ÃÅ Ä¿ ÈÂ ÁÀ ÂÉ Ã¿À

To check the structural dierences between the dierent We see that for some questions the results show small dif- areas socio-economic data and basic knowledge data were col- ferences between listeners and non-listeners in the expected lected. These data point to the fact that the dierences between direction (for example on the constitution question (Q )). Kisumu and Migori are rather large: Respondents in Migori But the dierences between listeners and non-listeners are are less formally educated, they are signicantly younger, very small ( percentage points) and they are not signicant. they are denitely poorer, but they have the same general On top of this there is contrary evidence. Sometimes re- political knowledge as people in Kisumu. Table also shows spondents in Migori achieve the highest levels (Q ° and Q ±) that women are less formally schooled than men in both areas. or non-listeners know facts better (for example Q about We also see a dierence in the practice of land ownership knowing the right institutions where one can get free legal between the areas. Migori hardly knows the ownership of land advice and Q about writing a will to protect women land by women, whereas in Kisumu it is already practiced, at least rights) than the listeners to LRP. a little. This might in¤uence the impact of listening to radio. This shows that the results of the eects of the LRP pro- The same is valid for the fact that in Migori many respondents gram on knowledge of listeners are very small at best, but they are ne with the opinion that it is ok for men owning all the are not consistent regarding various knowledge questions. So, land in a family. Migori has more of a traditional rural setting we would rather say that there is no eect that can be proven. as compared to Kisumu. The dierences between listeners and non-listeners of LRP in Kisumu are much less accentuated. Listeners are slightly better o economically and they are signicantly older: percent of the listeners to the LRP belong to the elderly (+) whereas the non-listeners are younger with only percent belonging to this age group (please see table ). In the following we will rst describe the correlations between the outcome variables (knowledge, opinion, practice) and the predictor variable (listening to LRP), before we check for other potential explaining variables by simply looking at dierent bivariate correlations. At the end of the ndings section we will then use multiple regressions to look at all potential predictor variables to identify the contribution of listening to radio.

Gaining knowledge through listening to LRP? If we compare only Kisumu and Migori on some knowledge That can be explained by the distance tot he nearest urban agglomeration, questions we see large dierences. But as already said the dif- young people in Kisumu having more chances to migrate to town than their ferences in terms of age, education, and poverty levels look too Migori counterparts, who had a tendency to stay in their home region. big to attribute those dierences to listening to the LRP. These Computes three political knowledge questions (president, senator, former doubts are strongly enhanced by the fact that the dierences president) in one gure. There are no di erences between regions, but be- between listeners and non-listeners within Kisumu are rather tween men and women. small, so the contribution can hardly be as big as suggested. Although these questions were explicitly addressed in one of the broadcasts Table summarizes the results of some knowledge questions. produced during Nam Lolwe training in September ¢ ¡.

26 Audience research in Media Development

Table : Dierences between Kisumu and Migori

Factor Operationalization Area

Kisumu Migori

Low education level ÁÂ% ÈÄ% Education (none + primary up to Ê) (Female: ÈÄ%) (Female: ÊÅ%)

Medium level (secondary) ÃÉ% ÅÃ%

Age groups Age Å¿% vs. ÃÈ% ÃÀ% vs. ÅÁ% (share young (up to Ã¿) vs. elderly (ÁÀ+) Poverty level Socio-economic level ÂÄ% ÊÂ% (share of very poor + poor) ÈÉ% ÈÊ% Share of people having all three Political knowledge in general (ÊÄ% men, ÁÅ% ÊÅ% men, ÁÃ% questions right women) women)

Practice of ownership of land Land owned by men only Ê¿% ÉÂ%

It is proper for men to own all ÃÈ% ÄÃ% Attitudes land belonging to a family (women: ÅÊ%) (women: ÄÅ%) (strongly agree + agree)

Table : Results on knowledge questions

Listener LRP Non-Listener Question Migori Kisumu LRP Kisumu

Q New constitution allows equal rights for men and women ÉÀ% ÊÈ% ÁÁ% Share of right answer YES Q Are women allowed to own assets in home? ÄÊ% ÄÉ% ÊÈ% Share of right answer YES Q Spouse is entitled to sell land without consent of other ÊÁ% ÉÃ% ÉÉ% Share of right answer NO

Q ¡ Contradiction between constitution and customary law ÊÊ% ÊÂ% ÊÉ% Share of right answer TRUE

Q Contradiction between constitution and ÄÃ% Ä¿% ÉÀ% practice regarding gender equality – Share of strongly agree + agree ALL

Men ÄÊ% ÊÂ% ÉÅ%

Women ÈÉ% ÁÄ% É¿%

Q ¢ Where can people go for free legal advice? ÃÄ% ÂÅ% ÀÊ% Share of right answers (Chief and Fida)

Q What can be done a£er death of spouse to protect women rights? ÄÊ% ÊÂ% É¿% Share of right answer: Writing a will

Edition DW Akademie 27 Changing attitudes and opinions through listening to LRP? Further computations revealed that the attitude towards A few questions tested the attitude of respondents to impor- women’s rights on land depend a lot on age. Elderly people are tant questions regarding women’s land rights. Table summa- in general more conservative, i.e., less in favor of women rights rizes the results, dierentiated also for gender. and the listener group in Kisumu is older than the non-listener The overall result is similar to the one on knowledge. group (see above). So age might be the reason for the dier- We see rather small dierences between listeners and non- ences seen here, not listening to LRP. listeners. For some questions (Q , and Q ) it looks as if listeners are a bit more encouraged towards a pro-equal rights attitude, but it is the other way round in another question: Whether it is proper for men to own all land, we have the ¡ The elderly people in Kisumu are much more in agreement with men having lowest gures (= being more positive towards gender rights) all property (¡¢ percent +) whereas other age groups think a bit more mod- from non-listeners. So the non-listeners show a more posi- ern (only percent with young and ¤ percent in middle age group think it tive attitude than the listeners to LRP, which is a bit disturb- is ok with men owning all land) . So, it is mainly age that inuences thinking ing as LRP was championing women’s rights. on women’s ownership of property.

Table : Attitude questions

Non-Listener Listener Migori Kisumu

Q Men and women should be treated equally with regard ÈÉ% ÈÂ% È¿ % to land inheritance – Share of strongly agree + agree ALL

Men ÈÃ% ÈÂ% ÁÉ%

Women ÄÂ% ÈÂ% ÈÀ%

Q ¥ Politicians should positively champion women rights ÄÃ% ÈÈ% ÂÉ% Share of strongly agree+ agree ALL

Men ÈÉ% ÈÅ% ÃÁ%

Women ÄÄ% Ä¿% ÈÃ%

Q ¦ Proper for men to own all property – Strongly agree + agree ALL ÃÉ% ÃÁ% ÄÃ%

Men ÂÄ% ÂÂ% ÄÁ%

Women ÃÀ% ÅÈ% ÄÀ%

Table : Recent activities with regard to land rights

Non-Listener Listener Migori Kisumu

Q § Having visited an o©ce to seek legal advice? – Share of YES ÅÊ% ÅÅ% ÀÂ%

Q Recently talked and acted – Share of YES ÃÄ% ÅÄ% Ã¿%

Q ¥¡ Engaged in social group – Share of YES (signicant: p = ¿.¿ÂÄ) ÀÀ% Ã% Ã%

Action was inspired by Nam Lolwe? ÅÄ% À% ¿%

28 Audience research in Media Development

Triggering activities through listening to LRP? This has various reasons: Various questions were asked about activities respondents did a) One reason is that the originally planned design – having with regard to land rights (talking, looking for help, becoming Migori as the area of the true control group – was not possible. It active in a group). turned out that the structural dierences regarding education, Table (see page ®) shows that the listeners of LRP per- age, and poverty were too large to treat the areas as similar. It is form a bit better on these activities, but the dierences are a decisive dierence that the area in the Kisumu County is more small and not signicant. Only the question whether respon- urban or peri-urban than the remote area of rural Migori. dents got engaged in a social group that addresses land rights So we had to refer to the dierences between listeners and questions, shows a big dierence, and it is signicant. Surely, non-listeners in one area (with rather small dierences be- those who acted and were LRP listeners said they were inspired tween those groups), but by doing it this way it is very chal- by listening to Nam Lolwe to do so. At least the results on activ- lenging to nd out whether the dierences can be attributed ities are much more consistent among each other than those to listening, because we do not know the starting level of on knowledge and attitudes. knowledge, attitude, and practice. This is especially relevant, There were also open answers that were recoded to assess when the dierences are small. A baseline study identifying them. They show that the topic of land rights is relevant in those issues before the start of LRP and a subsequent “before- all areas: Land disputes are mostly mentioned as a topic ( aer” design would have been an alternative. percent), followed by the topic of “how to divide the land” ( b) Another main problem in this study was that we were percent), and the issue of “title deeds” ( percent). There are unable to get hold of the content of the broadcasts on LRP in hardly any dierences in those topics in the dierent areas. September . Therefore, the main reason that signicant On the question of what can be done to protect women’s dierences between listeners and non-listeners were not dis- property, the answers of LRP listeners favored equal treatment covered was that we were not able to include very specic ( percent compared to percent with non-listeners and questions on the LRP. We could only create general knowledge thus supporting the law) and were less negative (nothing can questions (that are in¤uenced by other factors than radio only) be done: only percent compared to percent with non-lis- and not concrete issues, which could only be known through teners). Respondents in Migori favored the solution of giving listening to that LRP. women title deeds, and seeking protection through new laws, Not knowing the content had to do with “di¢cult” coop- maybe not being aware that the laws already exist. eration with Radio Nam Lolwe despite a very good start. Possi- In order to see what has contributed signicantly to the bly the radio managers did not have enough ownership of the outcomes in knowledge, attitude and practice and what the research project. They might not have had enough awareness relative contributions of those factors were, we also computed of the importance of the content of their program. several multiple regressions using SPSS (see Annex for details). All three outcome levels showed similar results in the Recommendations for media development practitioners sense that listening to LRP did not play a role at all, but instead – Our study points to the fact that good audience research gender, education, and sometimes the location (Kisumu or starts with content. In case the content is not appropriate, Migori) were the most important determinants. Therefore, the audience research hardly makes sense. And more oen multiple regressions conrmed our ndings from looking at than not content is less developed in terms of quality, quan- simple correlations: “Listening to LRP” was not a signicant tity, and regularity than radio managers and development contributor to better knowledge, attitudes or practice. cooperation agents think or are convinced of. It seems essential that development agents working on capacity building for journalists and media managers monitor the . Conclusions achievements of trainings and other interventions stricter and more regularly. Otherwise larger eects of these eorts The study could not attribute the levels of knowledge, atti- can hardly be expected. tudes, or practice in a consistent way to the listening of the LRP – As a rst step this requires that project managers check reg- program. The dierences seen between listeners and non-lis- ularly what the media output is by just listening to some teners were rather small and mostly not signicant and could of the emissions or reading some rough transcripts. This sometimes be better explained by other factors like education, is challenging, especially in an environment where radio gender, or the rural-urban divide. In other words, the poten- broadcasting is done in vernacular languages that are usu- tial beneciaries of the land rights program did not receive the ally not understood by foreign experts. As a second step, help they need from the broadcasts to better put their rights project managers could commission some very basic con- into practice. tent analysis providing quantitative information on essential elements of that very content and its quality.

Edition DW Akademie 29 That might lead to radical changes to the approach on how ca- Although this research project has not yielded the results we pacity building is delivered. were aiming for, it has greatly inspired us to try again. Radio programming can have tremendous impacts on radio listen- – The cooperation between external audience researchers ers and the entire society. We should use the opportunity of and development cooperation project managers needs to strengthening radio stations by conducting audience research be closer. Without extra eorts by the project team these that supports them in making their programs more useful to kinds of research eorts can hardly be successful. These their audiences and for the betterment of their societies. extra eorts were not factored in this project, but need Regarding the practicability of the research, quasi-exper- to be in the future. Clear expectations and requirements imental designs are quite demanding. The project proted should guide the process of dening mutual obligations from previous capacity building activities in Africa, establish- and achieving an agreement. ing a network of African researchers capable of conducting advanced research in the eld. Samples using clustering, random Lessons for audience research route, and Kish grid methods cannot be taken for granted, so – Future studies need to make sure that the radio programs capacity building has to take place ahead of time to ensure that have been aired, recorded, and handed over to the research- the research can take place with project partners instead of the ers. Its content has to be clear, consistent, and comprehen- usual market research companies. With respect to the radio sible before researchers go on with the audience surveys training, expert trainers have to be on hand to assist journal- investigating media eects. ists at the stations in arranging their programs to maximize – Achieving impact requires a sound stimulus being repeat- eectiveness in conveying good content. ed regularly and addressing virulent topics from dierent angles. The facts should be checked by experts ahead of broadcasting and tests should be done with listeners, before a format is launched.

In our view all three of those elements (duration and consisten- cy of stimulus, test) were missing in our study. The stimulus was not su¢ciently long (at least according to our limited information on the number and duration of LRP broadcasts over time). We also suspect that the content was not consistent as such, but we cannot be sure. We only assume, based on the former broadcasts (between May and July ) which were rather confusing.

– Audience researchers need to work very closely with all involved stakeholders: radio editors and managers, development project managers, donors, as they touch on sensitive issues and are looking very closely at the performance of those actors. – Cooperation between audience researchers and practitioners entails two things: (a) it is rst of all making research understood by practitioners, and if possible making it useful to them, and (b) building trust between researchers and practitioners. Practitioners have to deliver a lot of services to the research that comes in addition to their daily work load, like recording programs, retrieving programs from ar- chives, and elaborating time schedules of programs – without immediate benet to them. – Regarding the selection of treatment and control group, we have learned that it is rather di¢cult to select a true con- ¤ Compare, for example Farm Radio for an impact approach or “theory-based trol group that is perfectly similar in all aspects to the treat- evaluation” (see Spurk, ¢ ¡). Facing non-availability of baseline studies ment group. Approaches measuring traits in a baseline this approach uses proven ways of achieving impact ( from other research) study, and then realizing a before-aer approach, and/or a and compares the requirements for impact with the project under inves- dierence-in-dierences approach, could be an alternative. tigation and draws conclusion based on this “plausibility of achieving im- Panel designs seem to be most promising. pact”. Information about e ectiveness of Farming Programs on Radio: see African Farm Radio Research Initiative,¢ .

30 Audience research in Media Development

. Case Study : Improving the social information are the results. While the peace talks between the media use of Colombian community government and FARC rebels have given cause for hope, main- radios: a method triangulation stream media and with it public debate are characterized by polarization and distrust. Laura Schneider, Esther Dorn-Fellermann, Roland Schürho Especially in contested environments like this, community radios can be important alternative sources of informa- Case study fact sheet tion, providing rural areas and small communities with local Goals of the Case Study: To test a combination of social sci- content, and giving a voice to those who are oen ignored ence and digital analytics tools to measure social media en- or otherwise discriminated against. In Colombia, there are gagement and to help Colombian community radio stations around community radios broadcasting predominantly to improve their reach and community building capacities in municipalities smaller than , inhabitants (Kopp and on social media. Ziemsen, ). They constitute a signicant counterpart to the country’s two big radio companies RCN and Caracol, the Research questions: commercial stations, the two national state radio broad- – Is the method mix of triangulation of two social science casters as well as the more than radio stations of “public methods and digital analytics methods viable? interest” run by the police, military, local governments, and – How can the two community radios reach more people on universities (ibid.). social media and how can the engagement of their social According to Colombian law, community radios have the media users be increased? mission to create participatory spaces for communication, information, expression, and education of the community Methods: Focus Groups, Quantitative Survey, Digital Analytics (MINTIC, ). Listeners can be enabled to participate and in- Measurement teract with each other and community radio stations with the help of new digital channels. Almost percent of all Colom- Sample: ° (potential) listeners of two community radio sta- bians have access to the Internet (Internet World Stats, ). tions Contacto and Granada Stéreo Social media can play a particularly important role as they allow for a constant exchange with community members about Innovative aspects/approaches: Using insights from digital program topics and the audience’s interests and needs. Face- analytics for media development purposes; mixed methods book and Twitter are the social networks that are used most design by Colombians, especially by younger people (MINTIC, ). Based on these facts, it is crucial for community radios Challenges: Bringing data from digital analytics and social sci- to make the most of social media. Community radio sta - entic methods of data collection together; remote manage- tions oen have not developed a systematic social media ment of audience research strategy, owing to lack of resources and expertise. Media development organizations, on the other hand, seldom Main results: focus on social media reach and engagement, but tend to – While Granada Stéreo generated a great deal of interaction restrict their assistance to classic media or journalism pro - with listeners on social media, Contacto used social media duction/reception. primarily to distribute information. The aim of this project is to help Colombian citizens by – Readiness by (potential) listeners to engage on social media developing a method for nding out what information and was found to be highest for topics with a local reference, but social media interaction needs they have. This entails mea- there was also considerable interest in topics dealing with suring social media reach and engagement with limited the peace process and the recent history associated with it. resources, enabling local partner organizations to conduct – Social media content was found by the participants to be the research themselves and consulting the radio stations, most engaging if it focused on a variety of topics, both in- whose activities are the subject of the research, on what to do formative and entertaining. Photos and videos were espe- to obtain better results. cially popular. This leads us to the rst research question:

. Introduction RQ: Is the method mix out of social science methods (focus groups, survey) and digital analytics methods a viable option Decades of violence between the state and dierent guerilla for analyzing the social media activities and attitudes of media groups have made Colombia one of Latin America’s most cri- outlets’ audiences? sis-prone and con¤ict-aected countries. Massive restrictions This project thus applies a triangulation of three dierent re- of freedom of expression and limited access to independent search methods (focus groups, survey, digital analytics) to evalu-

Edition DW Akademie 31 ate the social media use of two Colombian community radio sta- Figure : Method Triangulation tions, and how it can be improved. The specic question is: RQ: How can the two community radios reach more people on social media and how can the engagement of their social media users be increased? Two community radios with dierent characteristics were chosen for this project: Contacto and Granada Stéreo. Survey Contacto is an Internet community radio based in Focus District (Comuna ) in Medellín, Colombia’s second big- groups Digital gest city. The radio is run by Sumapax, a collective aiming to Analytics promote the development of individuals and the community through local communication projects. The foundation provides young people with training on topics such as politics, democracy, recording and editing of radio and video, and Findings and Recommen- website design. The station was founded in , not least as dations a platform for the young community members. Apart from music, the station’s programming addresses a variety of issues related to state projects and employment, as well as announcements relevant to the community. Granada Stéreo was founded in ±±° by the Association for the Cultural Development of Granada (Asociación para el De- sarrollo Cultural de Granada). Granada is a small, con¤ict-af- fected municipality in Colombia's northwest Antioquia prov- The questionnaire, focusing on topics of interest, media ince. The goal was to establish a radio station which promotes use and socio-demographic variables, garnered basic informa- Granada's identity and culture and provides information on tion about the composition of the focus groups. local events and issues. Granada Stéreo broadcasts music as The aim of the focus groups was to discuss these the- well as six programs dealing with current aairs, culture, iden- matic interests in detail, and to assess the participants’ will- tity, higher education, sports, and entertainment issues. ingness to interact and respond to dierent topics on social Both stations make active use of social media, especially media. They helped to ascertain which kinds of issues were Facebook and Twitter. However, as both the basic characteris- generally relevant to the participants by showing them a tics of the two radios and their social media activities vary sig- broad range of potential topics (in the form of articles and so- nicantly, no direct comparison is intended. In fact, in order to cial media postings) and getting them to select those which include dierent conditions and scenarios, it was deliberately most sparked their interest. Participants were presented with opted for an Internet radio based in a big city (Medellín) and a four news items at a time, and then debated which topics traditional radio in the rural area of Granada. they were most interested in, and in what way they would be willing to interact (in the form of likes, comments, shares, retweets etc.) with such posts. . Research methodology The digital analytics, in turn, provided a precise overview of the social media communication of the two radio stations A triangulation of three empirical research methods was used: (amount and type of posts/tweets) on the one hand, and the focus groups, a questionnaire-based survey (with focus group users’ interactions (likes, shares, retweets etc.) with the radios members), and digital analytics of the social media commu- on the other. For this analysis, data from the radio stations’ nication of the two stations. While the survey and the focus Facebook and Twitter sites were collected, these two being the groups were conducted by the University of Antioquia in Me- most popular social media channels in Colombia. dellín, the digital analytics as well as the data analysis of the For Facebook, audience growth was measured by the num- survey results were managed by DW Akademie in Germany. A ber and increase of fans during the period of examination, en- multi-method approach was chosen to get not only a descrip- gagement was measured using data for the number of interac- tive overview of the social media activities and the listeners’ tions (shares, likes, comments) per post as well as post type, behavior, but also more detailed insights, to enable the com- and a list of the most popular posts was drawn up. For Twitter, munity radios to cater more e¢ciently for their interests and audience growth was measured by the number and increase needs. The dierent methods serve dierent purposes and of followers, engagement involved the number of retweets of thus complement each other. the station’s own tweets, as well as the number of favorites and replies to the station’s tweets, and here too, a list of the most popular tweets was assembled.

32 Audience research in Media Development

Overall, in August and September nine focus groups Survey results were conducted at the community radios’ premises with lis- Overall, the participants of the focus groups who completed the teners and potential listeners of the radio stations: ve re- questionnaire used the Internet frequently. ° percent used it lated to Contacto , and four related to Granada Stéreo. Con- on a weekly basis. Facebook was the most popular social media venience and snowball samples were drawn with the help of channel used. percent of respondents from Contacto had the radio stations, grouping the participants together in age used Facebook the previous day and ° percent had used the so- homogeneous groups. While the focus groups of Contacto cial network during the course of the last seven days, compared had a total of participants, people took part in the discus- to percent of Granada Stéreo participants using Facebook sions regarding Granada Stéreo. So, the Granada Stéreo focus on a daily, and percent doing so at least on a weekly basis. groups were rather large, which generally makes it more di¢- Twitter was used far less. percent of Contacto listeners had cult to generate an in-depth discussion among all participants. used it the previous day and ± percent had used it over the last Sample details are featured in table . seven days, while only percent of Granada Stéreo participants While the survey and the focus group discussions had a to- used Twitter on a daily, and percent used it at least on a weekly tal sample of (potential) listeners and users, the digital ana- basis. Younger users had a strong preference for Facebook, while lytics data is based on all users that frequented the social media the older ones also used the predominantly text-based Twitter. sites of the community radios during the period of the study. This dierent composition of target groups on Twitter and Face- book should be taken into account when developing social media communication strategies. . Findings The questionnaire also revealed the groups of participants were interested in dierent topics (see table ). The rankings The ndings of the three parts of the examination are present- must be seen under the point of view that the respondents from ed according to the respective method. Granada Stéreo consisted mostly of young people up to the age of ° years compared to Contacto with older listeners on average.

Table : Samples from Granada Stéreo and Contacto

Age Gender

≤ ÀÊ ÀÉ – ÃÂ ÃÁ – ÁÉ È¿+ female male

Granada Stéreo (n = ÈÉ) ÉÂ% ¿% Ã% Ã% ÈÁ% ÃÁ%

Contacto À¿ (n = ÀÊ) ÅÅ% ¿% ÁÈ% ÅÅ% ÂÂ% ÁÈ%

Note: Not all focus group participants lled in a questionnaire due to logistical problems.

Table : Topics of interest related to the community radio stations

Ranking Contacto 10 (N = 16) Granada Stéreo (N = 67)

À. Social topics (ÈÃ%) (Local) sports (ÂÉ%)

Å. Municipal issues (Á¿%) Church issues (ÃÈ%)

Ã. (Local) sports (ÃÀ%) Municipal issues (ÅÊ%)

Â. Music (ÃÀ%) Radio broadcasts themselves (ÅÊ%)

Á. Art and culture issues (ÅÁ%) School, university issues (ÅÁ%)

Edition DW Akademie 33 The respondents from Granada Stéreo were very interested happened in my neighborhood, the good things as well as the in local topics concerning the community such as local sports, bad things.” For younger people, these topics were related to church, and municipal issues. The participants from Contacto schools, local transport, and sports, for older participants the mentioned social topics most oen, followed by local top- focus was on health and society related issues. ics concerning the municipality and sports. Thus, topics with a Furthermore, the results show that one topic is of particu- clear local focus would seem to be the way to go for the radio sta- lar importance to (potential) listeners, irrespective of their tions on social media. In a separate item, the respondents from age: the ongoing peace process in Colombia. Although this is a Granada Stéreo demanded more sports, music topics, and jokes national topic, participants identied the question of peace as in the program, whereas Contacto respondents felt social and relevant to everyone. municipal issues should be dealt with more oen. Some participants expressed a wish for more variety of topics: “It is ne to broadcast information about the activities and Focus groups main subjects of the radio station, but it is important to talk Overall, the analysis of the focus group discussions revealed about other issues as well. As a media outlet, the radio shouldn’t that people are interested in the social media channels of the repeatedly address the same topics only, as this becomes boring.” two community radio stations. Although not all focus group Regarding the (potential) listeners’ engagement on social members were active listeners of the radio stations or engaged media platforms, the focus groups revealed that, independent with their Facebook and Twitter accounts regularly, there was a of their age, photos and videos as well as ¤ashy titles attract consensus that social media are important tools for spreading their attention and they are more likely to ‘like’ (Facebook) and information and interacting with the community. ‘favorite’ (Twitter) or share and retweet them. Moreover, people commented that the texts of the radio station’s Facebook Granada Stéreo posts should be shorter and more precise. Focus group participants of all age groups said they are most Especially the younger discussants said that they are most interested in local topics that are linked to their immediate sur- likely to engage on social media when their peer groups are roundings and everyday life. “The closer the content is to our di- included in the posts, such as adolescents from the commu- rect living environment the better,” said a participant. While they nity or district or their friends. The older participants without consider dierent issues relevant – especially culture, sports, and social media experience said they would be interested in train- education – the geographical closeness is crucial. That means ing courses to enable them to use Facebook or Twitter. that national topics should also be adapted to the local context. Furthermore, the members of the Contacto focus While younger focus group participants had a special interest in groups suggested that the use of Twitter hashtags should be music, the older discussants emphasized the signicance of so- improved by making them unambiguous, more precise, and cial and religious issues as well as educational programs. by using already existing hashtags. This would enable users to Moreover, the results show that the process of dealing with nd specic topics of interest and start a dialogue about them. the con¤ict-ridden past and violence as well as the rehabilita- tion of democratic rights play a signicant role independent Digital analytics of the discussants’ age groups. Due to the history of Granada, Digital analytics were utilized to look at audience size and in- which suered severely from violence, this issue aects almost teraction for each radio station. all community members. Especially adolescents claimed that they use Facebook Granada Stéreo mainly for communicating with their friends as well as enter- Granada Stéreo had , Facebook fans on November , . tainment through music and videos. Accordingly, they sug- There was a two percent rise in Facebook fans during the pe- gest that information should be conveyed on social media in a riod from August to November . Also there was a relatively more entertaining form. high number of interactions of per post. The radio station The discussants said that they are more likely to engage sent out posts. The most used post type was photo, fol- with photos – and even more so with photos in which they can lowed by videos, links, and status messages. Table shows the recognize places from their home town. Furthermore, they stat- post type distribution. ed that they would interact more with videos than with pure text. Overall, they suggested that Granada Stéreo’s posts should Table : Post type distribution – Granada Stéreo generally have less text, and that posted texts should be shorter. Total Contacto « amount Photos Videos Links Status Contacto focus group participants also stated that they are of Posts most interested in topics that have a strong connection to their community and living environment. As one participant Á¿Â ÀÊÄ ÀÅÉ ÀÀÅ ÄÈ commented, “I want to know what is happening and what has

34 Audience research in Media Development

Figure a: Most interactions: Laura Noreña from Granada at Figure b: Most comments: year commemoration of singing contest The Voice Kids Granada killings

The post type with the highest interaction rate (likes + com- Table : Tweet type distribution – Granada Stéreo ments + shares) was also photo material, followed by links, videos, and status messages. The response rate of Granada Steréo Station Others Station’s is percent, which means that the radio station answers all Station’s retweet- retweeting responses user posts. The station responds to Facebook user posts within tweets ing others’ station’s to user an average of ° hours. tweets tweets questions The digital analytics data underscored the validity of the survey and focus group ndings in that local stories about people from the community, commemoration and public hol- ÂÄÊ ÀÄÈ ÅÂÁ ÀÅ idays proved to be most popular. The ten most popular posts were again photos. Granada Stéreo’s most popular Facebook post, i.e., the post Table : Post type distribution – Contacto with the highest number of likes, shares and comments (° in sum), focused on the success of a local singer and her success at a casting show (see Fig. a), while the post that generat- Total ed the most comments () had to do with the region’s history amount Photos Videos Links Status of violence and bloodshed (see Figure b). of Posts Granada Stéreo had , Twitter followers at the end of November , up .° percent from the beginning of August. The number of retweets also rose from in August to in Å,ÈÃÅ Å,¿ÊÄ ÅÊÃ ÀÂÂ ÀÀÄ

Edition DW Akademie 35 Figure : Most popular tweet: “Granada switches on the Figure : Example of a typical Contacto Facebook post Christmas lights.” with a lot of text

November. During the period of examination, the sta at the station tweeted tweets a day on average ( in total). In- teractions of all types were also on the rise (see table ). The response rate was lower than on Facebook but still quite high at percent. As with the Facebook page, community related topics were particularly interesting for the followers of Granada Stéreo’s Twitter pro le. The screenshot in Figure shows Granada Stéreo’s tweet with the highest number of retweets and favorites ( in sum). The switching on of the Christmas lights is of special importance to the local cultural identity of the inhabitants of Granada. Facebook identi es each post containing a photo as post Contacto « type “photo”, even if the picture is embedded in a long text. Contacto had fewer Facebook fans (°) at the end of Novem- The latter is o en the case at Contacto ’s Facebook page and ber than Granada Steréo but the fan base had increased by ±. this might be an explanation why the participants in the focus percent as compared to the beginning of August . At the groups indicated that they would prefer less text on the page. same time, the station was very active, posting , messages during the period under investigation. The most used post type on this page is photo material, followed by video, links, and status messages. Table shows the post type distribution.

36 Audience research in Media Development

Status messages received the highest interaction (. Figure a: Most interactions: Gra¢ ti: “Peace is not silencing interactions per message), followed by photos, videos, and the guns, it is overcoming social inequality” links. On average, less than one interaction was sparked by a post (.±). One of the reasons might be that the response rate of Contacto was zero, i.e., user reactions were never acknowledged by the station. The posts from Contacto which generated the most likes, comments and/or shares were event announcements and reports on events, but also reports about people who were somehow connected with the community. Contacto ’s most popular Facebook post obtained ± likes, shares, and comments (see Figure ±b). The station’s most commented post received four user inputs, indicating that interaction levels were low. Contacto was being followed by Twitter users on November , , a rise of percent compared to the beginning of August of that year. The output was high, though. Con- tacto was tweeting own messages per day, a total of ,° tweets during the time period. Most of these tweets were links to the own Facebook pages. No tweets from other pro les were retweeted, and only ± own tweets were retweeted by other users. The response rate was percent, indicating that there were no e orts to engage with other users. The tweet which generated the most retweets on the Figure b: Most comments: Contacto wins peace journal- Twitter pro le of Contacto was a report on the criminal ism prize prosecution of a national politician, followed by event announcements and historical issues (see Figure ª/page ®). This implies that the followers were not only interested in community speci c topics.

. Conclusions

The digital analytics showed both radio stations used Face- book and Twitter, but in di erent ways. Listeners and fans of Granada Stéreo could get in touch with the radio station via their Facebook page, leading to a high number of fans and interactions. The same applied to Twitter. Focus group participants emphasized their interest in news with reference to local events and the conlict-ridden past. Younger focus group participants were more likely to interact via social media, demanding social media contents should also entertain and contain photos and videos, the latter seeming to spark more readiness to interact. Less possibilities for interaction were o ered by Contacto Table : Tweet type distribution – Contacto on Facebook. The page was more as a showcase, posting mainly event announcements that did not generate much interaction. In addition, the radio station did not answer the Responses Retweeted Retweets of fans’ requests. The same applied to Twitter. Most of the tweets Own tweets to user tweets own tweets were links to their Facebook page only. Listeners and poten- question tial listeners in the focus groups were also interested in local topics and the peace process. Photos and videos prompted more willingness to engage regardless of age. Older audienc- ÅÈÊÃ ¿ ÅÉ ¿ es were interested in engaging with radio stations via social media but lacked the know-how to do so.

Edition DW Akademie 37 Recommendations for community radio stations phase, close collaboration and constant information ex- – Local, local, local! The (potential) listeners are most interest- change is necessary. ed in events in their immediate surroundings. Content must – Focus groups should ideally consist of between and ° par- be adapted to the local context and broken down, so that its ticipants. immediate relevance for local residents becomes apparent. – It is better to have fewer but balanced and well-composed fo- – The peace process and dealing with the past are of crucial sig- cus groups than many groups with participants who may not ni cance for all community members independent of their be able to respond to the questions. age. This is part of a common heritage and community and – In the run-up to the focus groups, the moderators should be engagement building should take it as a starting point. trained accordingly. – At the same time, a variety of topics as well as both informative and entertaining issues should be covered on social media. Topics should not be too repetitive. – Users are interested in more photos and videos with a direct General recommendations for an improved use of social link to the local context, less in long text. media "by community radio stations". – A dialogue about topics of interest should be started and – It is important to adapt social media contents to the maintained with the social media community, especially speci c features of the respective platforms: Facebook through direct questions, invitations to comment, and re- should be used to build a community and directly plies to user comments, etc. The topics of interest should be communicate and interact with this community regularly identi ed through digital analytics as well as com- through posting comments and questions and answer- munication with the users. ing users’ interventions. The interaction with listeners – Users and listeners are also active on Instagram and particu- on Facebook also helps to identify their interests and larly WhatsApp, enabling community radios further possi- needs, which in turn should be used to improve the bilities to interact with them. radio programming. – Older community members unfamiliar with new technolo- – Twitter should be used for establishing a broader, less gies could be trained, if they are interested. This way they personal and more professional network. Through the could become part of the social media communities. use of hashtags it is possible to take part in speci c discussions and thereby ensure a more prominent Recommendations for researchers representation of the own radio station. Twitter is – The time budget of the present study was quite tight. Es- normally used to publish more news-related content pecially the recruitment of participants takes time and re- and link back to the own website. However, also on searchers should plan at least a month for nding suitable Twitter interaction with other users should take place, focus group participants, preferably using random sampling e.g., through retweets, commenting other tweets, and or a quota plan rather than snowball sampling. answering questions. – Clear agreements with local partners should be made and – Quantity is not everything. It is better to publish posts the partners should take ownership and be part of the meth- and tweets that are of direct interest for the commu- odology development from the beginning. During the eld nity, that generate response and interaction with users and listeners, and motivate them to actively participate – rather than posting something for the pure sake of it. Figure : Most popular tweet: “Political persecution by the Less is sometimes more! attorney general of Congressman Iván Cepeda Castro” – Con- – Facebook pages and Twitter pro les should be ad- tacto vertised in the main radio program and their special features should be highlighted and vice versa. That means that Facebook and Twitter pages should also refer to the radio program. The radio program could be complemented by the social media platforms, and not by simply publishing the radio content : on Facebook or Twitter. – Watch what you are doing. One’s own social media activity as well as users’ interaction and response should be monitored constantly. This can be done through Facebook Insights and Twitter Analytics. These tools are provided to each pro le owner for free, directly within the Facebook and Twitter accounts.

38 Audience research in Media Development

– Researchers should decide whether to use costly soware that enables them to gather and analyze data e¢ciently, or whether the data from media house’s own accounts is suf- cient for the purposes of research. The last recommendation points to the fact that DW Akademie used the comparatively expensive services of Quintly.com to collect and analyze the digital analytics data. For community radios, it will almost always be more cost-eective to use the free data provided by Facebook Insights and Twitter Analyt- ics. Competitors can be added under “Insights” and “Pages to watch.” In this way it is possible to compare the performance of the own page and posts with other similar Facebook pages by observing the development of the fans, the number of posts and total engagement of the current week. For community radios, short surveys in combination with focus groups are a comparatively cheap and easy way to obtain information from target audiences. Especially the focus groups give listeners and users the opportunity to give direct and eective feedback on whether the stations are living up to their mission of providing participatory spaces for communication, information, expression, and education of the community. On the other hand, focus groups can be time consuming in analysis and it can be a challenge to get a real discussion going. Guided interviews can be an alternative, though less people can participate because conducting interviews is more time consuming. As stated above, free digital analytics data is readily available if community radios use their own accounts, and the validity of this data is higher than self-reported behavior in questionnaires. But digital analytics cannot replace information gathered with the other methods since it only measures behavior but tells us nothing about people’s attitudes and the reasons for their behavior. If measuring the impact of community building eorts is the goal, it is equally important to see how people are acting as it is to nd out whether they identify themselves with a community radio station and whether the radio station is indeed successful in providing its community with participatory spaces. This complementary role of dierent methods is the advantage of method triangulations of the sort presented here.

Edition DW Akademie 39 . Lessons learned and way forward Involving partners in such a way requires a common understanding of the standards of audience research. Clear and Dennis Reineck concise research questions have to be developed, questionnaires and other methods of data collection should adhere to The case studies presented here were ambitious in that they certain standards, and sta has to be trained so as to ensure chose innovative approaches to address gaps in previous me- that all respondents give their input under comparable condi- dia development related audience research. Their results are tions. It is important to develop a common denominator on primarily important from a methodological perspective. They these fundamentals before going into the eld, since devia- were aimed at gathering experience in audience research and tions can jeopardize the validity and reliability of the ndings. at the same time at testing new and innovative ways of going A pretest is a good method for making sure that all partners about it. So even though not every method yielded the results have a clear understanding of these common principles. expected and things went wrong from time to time, these ex- Sampling is another aspect of research that can be chal- periences should be viewed as a ladder, to be climbed in order lenging. While working segments of the population were un- for future DW Akademie research to gain in quality and scope, derrepresented in the Colombian sample, the control group increasingly enabling us to measure media development im- of the Palestinian sample was recruited from two instead of pact and learn to improve our eorts for the benet of the ben- four schools. Project partners sometimes recruit participants eciaries under examination. at their own convenience. In large samples where statisti- Reviewing the results of the study, the most valuable les- cally representative data exist, this can be compensated by sons we’ve learned, leading rung by rung up the ladder of in- weighting the data. When drawing smaller samples or when sights, are as follows: no representative data exist, quota plans based on estimates of gender and age distributions could help prevent stark im- Cooperation with partners balances as compared, for instance, to snowball sampling A rst important insight from our studies is that close cooper- techniques. However, randomized samples should be drawn ation with project partners in possession of national, regional, whenever possible, since they enable drawing general conclu- and/or local know-how is essential. Not only media develop- sions from data. In any case, do-it-yourself research without ment activities require ownership. When it comes to develop- xed standards of recruitment all too oen leads to corrupt- ing instruments such as questionnaires or focus group guides, ed or biased data samples that do not provide a suitable basis partners can provide important knowledge of the local media for informed decisions. landscape, as well as impressions of usage patterns and information needs. Audience research directed at measuring the Monitoring of implementation and analysis outcome and impact of such activities should be entrusted to Quite closely related to the previous point is the question of partners as far as possible, in order for partners to build up ca- how best to monitor how research instruments are imple- pacities in the eld. Partners’ ideas and agendas should shape mented and how data are analyzed. Monitoring implementa- the research from the outset, though it is important to create a tion and analysis too closely results in high workload especial- synthesis between what the media development organization ly for the development organization, having to check research wants and what partners need. details in short intervals. However, leaving too much time Ownership and involvement of partners can be maxi- between monitoring activities bears the risk of discovering mized in intervention-based settings, i.e., where audience misdirected data collection and analysis too late. Reporting research provides the basis for media development action should thus take place at every key interval of data collection: (such as in the Colombian case study, see chapter ). Caution Before the pretest, aer the main test, aer the rst day of the is advised in evaluation contexts, where the work of partners main test, aer completion of the main test. The same applies is taken under scrutiny (such as in the Palestinian case study, to data analysis: Before analysis, aer the rst day of analysis, see chapter ). While project partners can play an important aer completion of analysis. While most of this reporting can part in conceptualizing and organizing the research in the be of an informal nature, e.g., a telephone call exchanging info latter case, presenting and interpreting results should in this on the latest developments, aer the pretest is a good point in case be le to others to guarantee the impartiality of the nd- time for a more detailed exchange on the shape and quality of ings. In both settings, research should openly be directed to- the research. This not only holds true for cooperation with proj- wards identifying the role media development projects can ect partners, but also for work with media and market research- play for the betterment of the beneciaries of the interven- ers. Of course, no one likes presenting results before the overall tions, i.e., towards achieving and measuring outcomes and process is nished. But even a short exchange can shed light on impact that relate to them. how methods are being applied and data are being analyzed.

40 Audience research in Media Development

Monitoring audience research helps prevent data from and quantitative (survey) method based on the identical spiraling away from the original research questions, project sample. In this case, data linkage is unproblematic since the partners imposing their own agendas on the research process data stem from the same source. (unless this was agreed upon earlier) and choosing method- Combining the aforementioned data with digital ana- ological approaches or sampling techniques that do not com- lytics data, on the other hand, does raise a few questions. If ply with standards of empirical research. For this purpose, digital analytics data re¤ect user behavior as a whole, in what project partners have to provide ample documentation of sense can the same be said of the focus group and survey re - implementation and analysis to allow for quality checks. The sults. In fact, the latter methods only deliver a selective pic- Kenyan study showed that an assessment of the content of ra- ture of reported behavior and attitudes. So this data should dio broadcasts is important to be able to judge the quality of be combined with caution, as was the case in the chapter on reporting on women and land rights, before the genuine audi- the Colombian study. Dierent sets of data do not always tell ence research started (see chapter ). the same story. Due to time constraints, it was not possible to use the digital analytics data to identify user groups that Preparing and adjusting the instruments for the target group were interesting to the researchers and which would have al- Project partners oen have quite a good impression of the tar- lowed to link their de facto behavior in social media to their get group of audience research. Knowing the beneciaries of statements during the focus groups and their answers in the media development projects helps adjust instruments for data survey. This would have been a good alternative for triangu- collection, be they questionnaires, interview or focus group lation but time constraints did not allow for sequential phas- guides, to suit their levels of language prociency and comes of data collection and analysis. Another option would have prehension. The challenge in the Kenyan project was to break been to spread an online questionnaire via Facebook and down the legal jargon of land rights legislation into a language Twitter to nd out more about the users and their preferenc- ordinary Kenyans could understand. The Palestinian question- es. This, too, would have made data linkage easier, though at naire was adapted to children’s language and included comic the cost of surrendering the in-depth motivational insights gures to be more motivating to the participants. Knowing gained from the focus groups. the participants also helps choosing media alternatives that Mixed methods designs are misunderstood, if methods are realistic. For example, the media examples in the Pales- are simply combined with each other additively. Triangula- tinian MIL test were taken from actual Arab media or social tion entails merging the data to create a whole that is greater media messages were simulated to come as close to the com- than the sum of its parts. Each method should be designed munication they encounter in their everyday lives as possible. to shed light on the data provided by the other methods, to Pretests are a good way of nding out whether a research allow for insights that go beyond what any single method instrument is suited to the people taking part in the research. can produce. It makes sense to always pose the question, Both the Kenyan and the Palestinian studies used extensive what additional insights another method could deliver and pretesting to adapt the methods to their participants. The Co- whether these insights are worth the extra eort. For in- lombian study did without a pretest, since it made use mainly stance, the Kenyan case study could have used a combina- of the qualitative focus group method. In retrospect it might tion of content analysis of radio broadcasts and audience have been sensible to conduct a pretest to make sure there was survey, in order for the content to better inform the ques- a common understanding of the research questions and the tionnaire development. exact methodology. Designing audience research Mixed methods approaches The other two studies conducted also yielded important in- Mixed methods approaches are oen proposed for audience sights into how best to design audience research. Results from research in media development but seldom put into prac- the Palestinian study indicate that straightforward surveys for tice (see chapter .). One reason is because mixed methods measuring Media and Information Literacy (MIL) will not do approaches require more eort on the part of the research- when trying to measure the MIL levels of children and ado- ers who have to be qualied in each method employed. This lescents. Rather, hands on testing with participants judging makes it more likely for stand-alone classic methods like realistic examples of mass and social media proved to be su- large-scale surveys or focus groups to be used, than com- perior to abstract surveys. Comparing methods for validity is binations of classic with more innovative methods of data something that media organizations rarely have time to do. collection and/or data analysis. Another reason is that data This could be a productive eld for universities and institutes collected with dierent methods oen stems from dier- of applied research interested in audience research in the me- ent sources, raising the question of how the data can best be dia development context. combined to gain an accurate picture of audience behavior and attitudes. The Colombian example delivers one possible solution to this problem by using a qualitative (focus group)

Edition DW Akademie 41 In the Kenyan case it was found that the strength and clear- Eects of audience research ness of the stimulus was not great enough to have measur- A nal lesson from our study has to do with the intentional able eects. In¤uencing attitudes and behavior apparently and unintentional eects of audience research. Audience re- requires considerably more communicative eort. Journal- search should not be misconstrued as an act of measurement istic reporting is not as clear cut as targeted communica- comparable to the measurement of temperature or time. Un- tion like advertising or public relations messages. The huge der normal circumstances, physical measurement has scarcely challenge is how to demonstrate media eects for news jour- measurable repercussions for the object of measurement. The nalism in the same convincing way as for communication day does not become markedly warmer or colder because it campaigns, without changing the independent and much is being measured by a thermometer, just as it does not be- less “message-oriented” character of journalism. This hav- come noticably longer or shorter because its duration is being ing been said, one could imagine individual radio stations measured by a clock. Because audience research most oen is devoting a whole week of programming (or several weeks of concerned with living people, reactivity plays a role. Research continuous regular programs, aired once or twice per week) has the potential to change people participating in it and it to a certain topic or getting several media partners to coor- has the potential to change project partners and media associ- dinate activities devoted to informing on specic topics. The ated with the research. Eects research, most oen used for results, however, could also be interpreted as a warning to purposes of evaluation, is precisely intended to eventually ef- not expect too much from media message diusion, if this is fect changes in its objects of measurement, i.e., the benecia- not ¤anked by interpersonal communication activities and ries and/or project partners. This can be institutionalized for by instruction at the local level. Nonetheless, impact research example in the form of change workshops. For instance, the remains worthwhile and has the potential to help establish Colombian study was used by the radio stations involved to new standards in media production and reporting, if eects improve their social media strategies. can be proven to be robust. But research can also have unintentional eects. It was certainly not a purpose of the Kenyan study to make the participants aware of gender dierences in how land rights were Criteria for audience research in media development being practiced. Nevertheless, the participants will have Audience research in media development presents researchers gone out of the interviews more conscious of the topic than with specic challenges. The researcher should at least re¤ect on before. The same is true for the participants in the Colombian the following points when conducting a study: focus groups, who will have become more conscious of what – Cooperation with partners: How can project partner the community radio programs in their home towns have on organizations be involved in the planning, implemen- oer. And the ° Palestinian pupils, having judged ten media tation and analysis of the research? Is it viable and examples, will have been more aware of the mechanisms of sensible to involve market research or academic institu- media reporting aer the test. This highlights the fact that au- tions? dience research can contribute to a greater awareness of top- – Monitoring of implementation and analysis: Have orga- ics and lead participants to re¤ect their own behavior. In some nizations involved understood the goals and methods of cases it might even be sensible to have a short discussion with the research correctly? At what intervals should this be the participants aer data collection. And in some cases such a monitored? research eort can become a small intervention in itself. – Preparing and adjusting the instruments for the target On the other hand, it is conceivable that research can group: What specic features of the target group have be have unintended negative eects. This might be the case if in- taken into account when developing research instru- sensitive questioning lets people’s willingness to participate ments and sampling strategies? When and how should a in research in the future dwindle or if participants are made pretest be conducted to test the research instruments? to feel uncomfortable with the situation of data collection in – Mixed methods approaches: Can dierent methods be any other way. applied to gain insights in an e¢cient way that would not have been possible with a stand-alone method? Can Next steps the data from the methods be combined in such a way, What should be the next steps on the audience research agen- that ndings are coherent? da? Depending on available resources, media development – Designing audience research: Have methods been tested organizations should be encouraged to test and develop a under similar conditions? Are they valid and viable? standard repertoire of methods for audience research. Build- – Eects of audience research: What (negative or positive) ing expertise in the eld not only enables organizations to eects might the audience research have on the media do their own research, which is oen necessary because large development environment in which it is to take place? representative surveys do not address relevant questions or do not lend themselves to disaggregated data analysis, but also enables organizations to judge the quality of third party

42 Audience research in Media Development

research such as the work of media and market research in- though one should be careful of using this as an excuse, not stitutes, a¢liated universities, or project partners. This will to have to go to the eort of engaging with the audience it- make it possible for organizations to prot from the expertise self. The digital world is full of traces of audience behavior, of partners. waiting to be analyzed. This, however, raises some ethical Audience research in a media development context will al- questions. Is it okay for media development organizations ways require a certain degree of creativity. This applies to how to ght for data protection and at the same time prot from standard methods are adapted to the context, but it also ap- the data collection activities of multinationals such as Face- plies to using innovative methods. Once an organization has book, Twitter, and Google? How openly can one talk about a standard repertoire, it should not take this as the end point what is going on in social media, if the government of the of its endeavors. Rather, the dynamic development of existing country in question actively combats freedom of expres- media and information ecologies will always challenge organi- sion? What data security measures have to be adhered to, zations to optimize existing tools and develop new ones more to protect people whose behavior shows up in digital data? suited to what it is they want to nd out. Audience research in the digital age holds several ethical The methods tested in this volume should be pursued and questions that cannot su¢ciently be dealt with here. They adapted to other contexts themselves to nd out whether they should, however, in future be thoroughly re¤ected in order indeed are replicable and generalizable. Digital analytics pres- for audience research to do no harm in the context of me - ents a host of methods and metrics that can be harnessed for dia development as a whole. the good of media development. The Colombian study can be seen as a starting point. Quasi-experimental or experimental designs should increasingly become standard practice in audience research aimed at measuring impact. The Kenyan study is not the rst to do so, but it presents an innovative design, de- serving imitation and adaptation. Tests using media examples seem to be a good way of testing Media and Information Liter- acy of children and teens, as the Palestinian study has shown. This could also be tested with adults or with other media related skills and attitudes. But audience research is not about the methods themselves. These methods are only a means to an end, that end being measurement of the outcome and impact of media development activities for its beneciaries, documenting medium- and long-term eects of activities, collecting proof that positive changes have happened, and identifying weakpoints of the activities. So the case studies uncovered several points of departure for improving DW Akademie’s activities. In Co- lombia, the ndings revealed that community radio stations need more support in social media based community building and encouraging listeners and users to engage with the stations. The Kenyan study showed that journalism training has to be long-term and system-oriented and thus sustainable. Results need to be monitored. One-o training sessions do not seem to be enough to ensure good quality reporting that can have an impact on listeners. And the Palestinian case study found that media production is not enough to foster critical Media and Information Literacy. Analysis and comparison of professional and social media is essential for enabling pupils to eectively decode media messages. Future work and research could take a closer look at the relationship between media use and content. Combining content analysis with audience research would be one way of nding out more about what content has what eects. On the other hand, content analyses of social media communication might be a valid method of doing audience research without having to engage with the audience at all,

Edition DW Akademie 43 Literature

Abdalla, Amr. ². The Team: Kenya. Final Evaluation Report. Bruns, Axel ²²®. Blogs, Wikipedia, Second Life, and Beyond: Search for Common Ground. https://www.sfcg.org/wp-content/ From Production to Produsage. New York: Peter Lang. uploads/²/²®/KEN_MT_Mar-The-Team-Kenya-Final-Eval- Bulger, M. ². “Measuring Media Literacy in a National Con- uation-Report.pdf text: Challenges of Denition, Method and Implementation.” African Farm Radio Research Initiative. ². Participatory Radio Medijske Studije / Media Studies (ª):®-². Campaigns and Food Security – How Radio can Help Farmers to Cameco. ²ª. Resource Centre. http://www.cameco.org/eng- Make Informed Decisions http://farmradio.wpengine.netdna- lish/resources/resource-centre cdn.com/wp-content/uploads/farmradio-prcreport².pdf Cauhapé-Cazaux, Eduardo and Shanthi Kalathil. ²±. O¶cial Afrobarometer. ²±. Summary of Results: Afrobarometer Round Development Assistance for Media: Figures and Findings. A Re- ª. Survey in Uganda. http://afrobarometer.org/sites/default/les/ port by CIMA and the OECD. Washington, D.C.: CIMA. https:// publications/Summary%²of%²results/uga_rª_sor_en.pdf www.oecd.org/dac/governance-peace/docs/CIMA.pdf Anderson, Daniel. ²². “Prosumer Approaches to New Media Cherubini, F. and R.K. Nielsen. ²ª. Editorial Analytics: How Composition: Consumption and Production in Continuum.” News Media are developing and using Audience Data and Met- Kairos ® (). rics. Oxford: Reuters Institute for the Study of Journalism. Anderson, Gavin. ². “Making media markets work in Nepal. Consejo Nacional de Televisión. ². VIII Encuesta Nacional Private sector audience research and its role in building media de Televisión ². http://www.cntv.cl/prontus_cntv/site/ar- markets.” Presentation at FoME Conference ². http://fome. tic/²/asocle/²²±²±ª/®_entv.pdf info/wp-content/uploads/²//FoME-²-Anderson-Pre- sentation.docx Consejo Consultivo de Radio y Televisión (Concortv) ². Estudio cuantitativo sobre: Consumo televisivo y radial en Arke, E. & B. Primack. ²²¼. “Quantifying media literacy: Devel- ninos, ninas y adolescents. Peru. http://www.concortv.gob.pe/ opment, reliability, and validity of a new measure.” Educational ºile/²/investigaciones/²-resumen-ejecutivo-estudio- Media International ª ():±–ª±. ninos-adolescentes-rtv.pdf Arnold, K. ²²¼. Qualitätsjournalismus. Die Zeitung und ihr Debeljak, Klara and Joe Bonnell. ². Citizen Access to Informa- Publikum. Konstanz: UVK. tion in Papua New Guinea. Washington: InterMedia. Balancing Act. ². . Feature Phone User Survey: Ethiopia, De Masi, Sonya ². When does Information Change Lives? An Ghana, Kenya, Nigeria and South Africa. http://www.balanc- Evaluation of Community Radio Development in South Sudan ingact-africa.com/sites/balancingact-africa.com/ºiles/prod- and the Three Areas. London: Internews ucts/.%²Feature%²Phone%²Report%²FV.pdf Dun, S. A. and D. Mutassem. ². Digital Literacy in Arabic Speak- BBC Media Action. ². Research Summary: Exploring gover- ers: The Role of Bilingualism in E ective Use of Web Resources. In nance in the Palestinian Territories. http://downloads.bbc.co.uk/ D. Moser & S.A. Dun (Eds.), Digital Janus: Looking Forward, Look- mediaaction/pdf/research_summaries/BBCMA_RS_PT_au- ing Back. Oxford, UK: Inter-Disciplinary Press, -±. gust_².pdf (.®.²) European Association for Viewers Interests & Danish Techno- BBC Media Action. ². Climate Asia. http://www.bbc.co.uk/me- logical Institute. ². Testing and Rening Criteria to Assess diaaction/climateasiadataportal Media Literacy Levels in Europe. Final Report commissioned by BBC World Service Trust ²²¼. Cambodia sentinel survey ²²®: the European Commission, Directorate-General for Information media consumption (radio, television, internet and mobile Society and Media, Media Literacy Unit. Brussels. phone) and HIV and AIDS information in the media EUROPEAN BROADCASTING UNION. ¼¼ª. Harmonized Audi- Berger, M. ². Palestine’s Occupied Fourth Estate: An Inside ence Measurement for International Radio, nd Edition, Geneva. Look at the Work Lives of Palestinian Print Journalists http:// Eveland, William P. and Dietram A. Scheufele. ²²². “Connect- www.arabmediasociety.com/?article=®² (.ª.²) ing News Media use With Gaps in Knowledge and Participation. Bishara, A. A. ². Back Stories: U.S. News Production and Pales- Political Communication ½ ():±-½. tinian Politics. Stanford: Stanford University Press. Farm Radio International. ². Agricultural Radio That Works. Bortz, J. and N. Döring. ²²ª. Forschungsmethoden und Evalua- http://farmradio.wpengine.netdna-cdn.com/wp-content/up- tion für Human- und SozialwissenschaÄler. Fourth Edition. Wi- loads/Farm-Radio-Agriculture-Radio-That-Works.pdf esbaden: Springer.

44 Fiedler, Anke. ². “Jaimerais bien être er de l’ORTM.” Enquête International Media Support. ². Kurdish Region of North- sur l’utilisation et l’image de la chaîne de télévision nationale au ern Iraq. Humanitarian Information Needs of Syrian Refugees. sein de la population de Bamako. Friedrich Ebert StiÄung / Me- https://www.mediasupport.org/wp-content/uploads/²/²®/ dia in Cooperation and Transition. http://www.mict-internation- humanitarianinformation-syriarefugees-ims².pdf al.org/wp-content/uploads/²±/²/MICT_etude_ORTM.pdf International Telecommunication Union. ²±. ICT Facts & Fig- Fiedler, Anke and Michael Meyen. ²ª. Information – Educa- ures: The World in ²±. http://www.itu.int/en/ITU-D/Statistics/ tion – Participation. Media Use Among Youth in Uganda. Bonn: Documents/facts/ICTFactsFigures²±.pdf DW Akademie. http://www.dw.com/popups/pdf/±®²²²¼®¼/in- International Telecommunication Union. ²ª. Metadata for formation-education-participation-media-use-among-youth- Percentage of Individuals Using the Internet. http://www.itu.int/ in-uganda-pdf.pdf en/ITU-D/Statistics/Documents/tatistics/²ª/Individuals_In- Forcier, Nathalie I. ². South Sudan National Audience Sur- ternet_²²²-²±.xls vey: A Nationally Representative Assessment on Radio Listen- Intermedia. ²±. AudienceScapes. http://www.intermedia.org/ ing Habits with Key Findings in Five Booster Areas for Internews research-ndings/audiencescapes/ Stations. Forcier Consulting. https://internews.org/sites/default/ºiles/resources/InternewsSouthSudanAudienceSurvey_ Internet World Stats ²±. Latin American Internet Users - No- web²-²-.pdf vember ²±. Accessed January , ²±. http://www.internet- worldstats.com/stats².htm Foster, Michelle J. ². Measuring the Audience: Why It Matters to Independent News Media and How It Can Contribute to Me- Ipsos. ². AMPS. Africa’s Media and Product Survey. http://www. dia Development. Washington: CIMA. ipsos.com/mediact/sites/ipsos.com.mediact/les/AMPS.pdf

Frère, Marie-Soleil. ². News and New Media in Central Africa. IREX. ². Audience Research Kosovo. URL: https://www.irex. Challenges and Opportunities. Open Society Foundation Refer- org/sites/default/les/SURVEY%²FINAL%²Sept%²²%² ence Series No. ¼. http://publicmediaalliance.org/wp-content/ ENG%²%®%¼.pptx uploads/²±/²/Mapping-Digital-Media-Central-Africa.pdf Isaksson, Ann-Soe, Andreas Kotsadam, and Mans Nerman. Gallup ². BBG Research Series Women and Media: Africa in ². “The Gender Gap in African Political Participation: Testing Focus. Presentation of Results in Washington. https://www.you- Theories of Individual and Contextual Determinants.” Retrieved tube.com/watch?v=s±²P®fr²M® from http://andreaskotsadam.les.wordpress.com/²²/²ª/ gender-gap-in-participation.pdf GeoPoll. ²ª. Kantar-GeoPoll Media Measurement. https:// research.geopoll.com/case-studies/case-study-media-measure- Jacks, Nilda, Amparo Marroquin, Mónica Villarroel, and Natália ment-roi.html Ferrante. ². Análisis de recepción en América Latina. Un re- cuento histórico con perspectivas al future. Quito: Ciespal. Gillmor, Dan ²². We the Media. Grassroots Journalism by the People, for the People. Sebastopol: O’Reilly Media. Jerven, Morten ². Poor Numbers: How we are Misled by Afri- can Development Statistics and What to do About it. Ithaca et Government of Kenya. ²²½. Kenya Vision ²². A Globally al: Cornell University Press. Competitive and Prosperous Kenya, Government Printing Press Kopp, M. and R. García-Ziemsen. ²±. Medien- und Akteurs- Hargittai, E. ²²±. “Survey Measures of Web-oriented Digital Lit- landschaÄ Kolumbien eracy.” Social Science Computer Review ():½-½¼. Lambino II, Antonio, Alexandra Tebay, and Sarah Buzby. ². “A Haupt, Paul. ². “Media Audience Research in Africa: Where Monitoring & Evaluation Toolkit for Media Development: What are we, and Where Should we go?” Presentation held at FoME do Available Indicators and Integrative Approaches Have to Of- Symposium ² (Audience Research in Media Develop- fer?” Global Forum for Media Development / Harvey, Mark (ed.): ment). http://fome.info/wp-content/uploads/²//FoME- Perspectives on Advancing Governance & Development from ²-Haupt-Final.pptx the Global Forum for Media Development Lechien, Olivier ². Head, Roy, Joanna Murray, Sophie Sarrassat, Will Snell, Nicolas In-house vs. External Audience Research: Sharing ² Years of Ex- Meda, Moctar Ouedraogo, Laurent Deboise, and Simon Cousens. perience With Foundation Hirondelle. Summary of the Presen- ²±. “Can Mass Media Interventions Reduce Child Mortality?” tation at the FoME Conference ². http://fome.info/events/ The Lancet ®ª:¼½-²². http://www.developmentmedia.net/ symposium-²/abstracts (November ª, ²) sites/developmentmedia.net/les/dmilancetarticle²±.pdf

Edition DW Akademie 45 Leeuw, F, & J. Vaessen. ²²¼. Impact Evaluations and Develop- OECD (²²±): The Paris Declaration on Aid E ectiveness. http:// ment. NONIE Guidance on Impact Evaluation. Washington: Net- www.oecd.org/dac/e ectiveness/®±.pdf work of Networks on Impact Evaluation. Observatorio de Medios (ONADEM) Bolivia (²): Tu palabra McCallum, Judith. ². RiÄ Valley Local Empowerment for sobre las noticias y el DIC. Informe nal. http://www.cedib.org/ Peace II (LEAP II). Final Evaluation Report. http://www.dmefor- wp-content/uploads/²/²±/Informe-Consulta-Ciudadana- peace.org/sites/default/les/USAID%²Kenya_²®²².pdf ²-Final-®-may.pdf

Media Asia. ²². Special Issue: Innovations in Mobile Use. Ed- Parta, R. Eugene. ². “Audience Research in Extremis: Cold ited by Sundeep R. Muppidi. Volume ½, Number . http://www. War Broadcasting to the USSR.” Participations. Journal of Audi- yumpu.com/en/document/view/½½®½/volume-½-number-- ence & Reception Studies. ® (). http://www.participations.org/ ²²-idl-bnc-idrc Volume%²®/Issue%²/special/parta.htm

Melki, J. ². “Sowing the Seeds of Digital and Media Literacy Pittaluga, Lucía & Ana Rivoir. ². “One Laptop per Child and in Lebanon and the Arab World. The Importance of Locally Bridging the Digital Divide: The Case of Plan CEIBAL in Uru- Grown and Sustainable Curriculum.” In Belinha, S. de Abreu guay.” Information Technologies & International Development and Paul Mihailidis (eds.), Media Literacy Education in Action. ® ():±–±¼. Theoretical and Pedagogical Perspectives, New York/London: Preeti Mudliar; Jonathan Donner; William Thies. ². “Emer- Routledge, ½½-®ª. gent Practices Around CGNet Swara: A Voice Forum for Citizen Melki, J. ²². Media Habits of MENA Youth: A Three-Country Journalism in Rural India.” Information Technologies & Inter- Survey. Issam Fares Institute for Public Policy and International national Development. ¼ () ICTD² Special Issue: ª±–½¼. A airs American, University of Beirut. Quintanilla, Jacobo, Alexandra Sicotte-Levesque, & Madara Melki, J. ²±. Media Literacy O ers the Arab World a Way For- Hettiarachchi. ². Understanding the Information and Com- ward. http://www.al-fanarmedia.org/²±/²½/media-literacy- munication Needs among IDPs in Northern Iraq. Internews. o ers-the-arab-world-a-way-forward/ (.½.²±) https://www.internews.org/sites/default/les/resources/Iraq_ IA_CwC_Report_²-²®_web.pdf MINTIC – Ministerio de Tecnologías de la Informacion y las Comunicaciones (²²). Resolución Número ²²± (Capítulo Reineck, Dennis. ²±. Palestine: Making media surveys kid II, Art. ®). Accessed January , ²±. http://www.mintic.gov.co/ friendly. dw.com / #mediadev. http://www.dw.com/en/pales- portal/ª²/articles-½¼½_documento.pdf tine-making-media-surveys-kid-friendly/a-®ª½¼½®

MINTIC – Ministerio de Tecnologías de la Informacion y las Reineck, Dennis and Jan Lublinski. ²±. Media and Informa- Comunicaciones (²±). Ocina Internacional. Cifras. Accessed tion Literacy: A Human January , ²±. http://www.mintic.gov.co/portal/ª²/w- Rights-based Approach in Developing Countries. Bonn: DW article-±.html Akademie. Myers, Mary and Pascal Chirhalwirwa. ². Media: A Voice Rossell, Pablo D. ². Media Access and Usage Around the for All – Final Evaluation Report. SFCG ‘Media: a Voice for World. Presentation at the FoME Conference ². http://fome. All’. Evaluation by iMedia. https://www.sfcg.org/wp-content/ info/wp-content/uploads/²//Gallup-Media-Interest-and- uploads/²±/²/Final-Evaluation-Media-a-Voice-for-All_iMe- Usage-For-publication.pdf (November ª, ²) dia_.².².pdf Rosen, Jay. ²²ª. The People Formerly Known as the Audi- Mytton, G. ²²±. “CIBAR’s ²th Annual Conference: Looking ence. http://journalism.nyu.edu/pubzone/weblogs/press- Back and Looking Forward.” CIBAR proceedings, Bonn, Vol. think/²²ª/²ª/½/ppl frmr.html :±-. Mytton, Graham, Peter Diem, and Piet Hein van Dam (²ª): Media Audience Research. A Guide for Professionals. Salem, Fadi; Mourtada, Racha; Alshaer, Sara. ². The Arab Third Edition. New Delhi et al: Sage. World Online ²: Trends in Internet Usage in the Arab Region. Dubai: Dubai School of Government (DSG). http://www.mbrsg. Natsios, Andrew ²². The Clash of the Counter-bureaucracy ae/getattachment/ ½²cc±-²fce-²±d-bf-¼c¼®dca/The- and Development. Washington: Center for Global Development. Arab-World-Online-²-Trends-in-Internet-and.aspx Norris, Pippa. ²²². A Virtuous Circle: Political Communica- Samuel, Atchenemou Azoudoum and Maud Bakirdjian. ². tions in Post-industrial Societies. Cambridge: Cambridge Renforcement des capacités des Médias pour promouvoir le University Press. développement collaboratif au Tchad. Evaluation nale. Search Odhiambo, Lewis O. ²². “The Media Environment in Kenya for Common Ground. https://www.sfcg.org/wp-content/up- Since ¼¼².” African Studies ª ():¼±-®. loads/²±/²/CHA-±²-Evaluation-nale-VF-ok.pdf

46 Schmidt, Christoph (Ed.). ². Kenya’s Media Landscape: A Suc- Tsegyu, Santas, and Ezekial Asemah. ². “Public Perception of cess Story with Serious Structural Challenges. DW Akademie, the Role of Rural Broadcasting in Rural Development in Nige- International Media Studies, Vol. ª. Bonn: DW Akademie. ria.” Journal of Sustainable Development in Africa. ± (½).

Search for Common Ground. ²a. Baseline Study of the United Nations Educational, Scienti c and Cultural Organiza- Perception of Youth About the Role of Media in Promoting tion. ². Global Media and Information Literacy Assessment Dialogue for Peacebuilding. https://www.sfcg.org/wp-content/ Framework: Country Readiness and Competencies. Paris: uploads/²/²/DANIDA-Key-Findings-of-Media-Baseline.pdf Unesco.

Search for Common Ground. ²b. Baseline Study Report on Van Dijck, José. ²²¼. “Users like you? Theorizing Agency in The Peacebuilding Initiative in Nepal. User-generated Content.” Media Culture Society ():-±®. https://www.sfcg.org/wp-content/uploads/²/²®/Nep_BL_ Vincent, Katharine and Tracy Cull. ². “Ten Seeds: How Mo- Jan_Peacebuilding-Initiative-in-Nepal.pdf biles Have Contributed to Development in Women-led Farming Sowka, A., Klimmt, C., Hefner, D., Mergel, F. & Possler, D. (²±). Cooperatives in Lesotho.” Information Technologies & Interna- “Die Messung von Medienkompetenz. Ein Testverfahren für die tional Development ¼ () Spring ²:½–®. Dimension ‚Medienkritikºähigkeit’ und die Zielgruppe ‚Jugendli- Whitehead, Ann and Dzodzi Tsikata. ²². “Policy Discourses che.’” Medien & KommunikationswissenschaÄ ª ():ª-®. on Women's Land Rights in Sub–Saharan Africa: The Implica- Spurk, Christoph. ². Baseline study part I, Results of Listener tions of the Return to the Customary.” Journal of Agrarian Survey. Unpublished report for DW Kenya. Change (/):ª½-.

Spurk, Christoph. ²±. “What to do Without Baselines? - Not Worku, Zelalem Tesfaye ². “E ect of Mass Media Interven- OÄ en Used Methods in Media Evaluation.” Presentation at th, tion on HIV and AIDS realted Stigma and Discrimination in Symposium Forum Media and Development “Focus on Impact: Ethiopia.” Journal of Development and Communication Stud- Advanced Methods and Concepts in Media Development.” ies, (/): ¼-. http://www.devcomsjournalmw.org/sites/ Bonn: DW Akademie. default/ les/papers/jdcs____zelalem.pdf

Spurk, Christoph; Dingerkus, Filip. ²±. The Need and the Opportunities for Sustainability – The Case of Local Radio Sta- tions in Tanzania, Presentation at Academic Conference “News Media Development and Sustainability in Africa,” University of Neuchâtel, - October ²±.

Srinivasan, Shranath; Abreu Lopes, Claudia. ². Africa's Voic- es Project Description. University of Cambridge. http://www. cghr.polis.cam.ac.uk/research-themes/pdtm/africas_voices/ap- plied_pilot/Africas_voices_project_description

Studio Social. ²±. Social Media Report in Palestine ². Internet: https://docs.google.com/ le/d/²B-®jJSVHIcmTFJnd- FVDbWlzMVk/edit?pref=&pli= (½..²±)

Please contact us for further information

Dr. Jan Lublinski Petra Berner Deutsche Welle Head, Research and Evaluation Head, Strategy and Consulting Services Bonn, Germany DW Akademie DW Akademie dw-akademie.com T +±.°.± ° T +±.°.± ±° [email protected] [email protected] [email protected] facebook.com/DWAkademie

Edition DW Akademie 47 Dennis Reineck

Dennis Reineck is a project manager for research and evaluation at DW Akad- Project Manager emie. He has done research on various topics surrounding Strategy and Consulting Services media development, the quality of journalism and media and DW Akademie information literacy. He also does monitoring and evaluation [email protected] consulting. Before joining DW Akademie, he was a researcher and lecturer at the Rudolf Augstein Professorship for Quality Journalism Practice, specializing in methods and empirical research. He started his professional career at ProSiebenSat. Media’s division for internal communication.

Dr. Laura Schneider

Dr. Laura Schneider is project manager in DW Akademie’s Strategy and Consult- Project Manager ing Services department. Her work focuses on the topics of Strategy and Consulting Services media viability, audience research, and media development DW Akademie policy. Another area of Schneider’s expertise is international [email protected] media freedom measurement: In several publications she has critically analyzed the existing press freedom rankings. Her PhD thesis provides ideas for a new instrument for assessing media freedom globally. Ms. Schneider has been a consultant for UNESCO, an analyst for Freedom House, and project co- ordinator at the International Media Center in Hamburg. She has worked as a journalist for several German media outlets such as Spiegel Online and the German press agency dpa, and was a radio and newspaper reporter in Mexico.

Christoph Spurk

Christoph Spurk is a lecturer and media researcher at the Institute of Applied Lecturer and media researcher Media Studies at Zurich University of Applied Sciences. He Institute of Applied Media Studies mainly conducts research on quality of journalism and com- Zurich University of Applied Sciences munication in Africa and on the in¤uence of mass media on [email protected] the democratization process. He is working in advancing the methods of evaluation, measuring results in communication eorts and mass media programmes, and supporting media development organisations in monitoring and evaluation. He has worked amongst others with Deutsche Welle Acad- emy, Swiss Development Cooperation, Fondation Hirondelle, UNESCO, and many African universities. Recently, he started to combine content analysis with audience research in eorts to measure media eects, for example with regard to peace processes.

48 Dr. Esther Dorn-Fellermann

Dr. Esther Dorn-Fellermann studied political science, media studies and European culture Senior Lecturer studies at Friedrich-Wilhelms University in Bonn, where she Deutsche Welle wrote her PhD in media studies on “Civil engagement through [email protected] community radios. Opportunities and limits of participatory media work in South Africa”. She has been at Deutsche Welle since , currently serving, amongst other things, as conceptual program manager for the Global Media Forum (dw.com/gmf). Beyond this, she also teaches in the Interna- tional Media Studies M.A. Program at DW Akademie.

Dr. Charles Nyambuga

Dr. Charles Ongadi Nyambuga is a senior lecturer if mass communication at Maseno Uni- Chairman versity and and researcher of in the area o audience survey Department of Communication and media content. He has been a lead and Co researcher in a and Media number of studies commissioned by DW among other collab- Maseno University Kenya orators. He has authored a number if journal articles on me- [email protected] dia and democracy and development communication among other areas. He is an author of a university level book titled introduction to communication published by the Oxford University Press. Dr. Nyambuga holds a doctoral degree from Nelson Mandela Metropolitan University in South Africa. He also has a Master’s degree in Mass Communication and Jour- nalism, Bachelor’s degree in Journalism and a bachelor of Arts degree in English and Public Administration.

Dr. Roland Schuerho

Dr. Roland Schuerho works as a project manager for DW Akademie. He was engaged Project Manager as a research assistant at the Center of Psychosomatic Medi- Strategy and Consulting Services cine in Gießen. Within the project “Psychosocial Determi- DW Akademie nants of AIDS”, which the Federal Ministry for Youth, Families, roland.schuerho@dw.com Women and Health has supported, he obtained his doctorate. Mr. Schuerho has worked as market researcher at the In- stitut für Demoskopie Allensbach and for over years as a media researcher at Deutsche Welle, providing reach measurement, global estimates, indicators for the development of international media markets, target group analyses as well as employee surveys.

Edition DW Akademie 49 facebook.com/DWAkademie dw.de/newsletter-registration

@dw_akademie dw.com/mediadev dw-akademie.com