This IDRC working document ptesents the results of a recent IDRC-supported research activity. Unless otherwise stated, copy- right is held by rhe authors. To obtain additional copies, please con- tact the authors or the contact person listed on the title page. This document is also available from the IDRC Library. Le present document présente les résultats d'un travail de recherche recent qui a bénéfIcie du soutien du CRDI. A moms d'indication contraire, les aureurs détiennent les droits d'aureur. Pour obtenir des exemplaires supplémentaires, prière de communiquer avec les auteurs ou 'a personne-ressource dont le nom figure sur page de tirre. On peut se procurer egalemenr d'autres exemplaires auprès de la bibliotheque du CRDI. Este documento de trabajo del GIlD presenta los resultados de una reciente actividad de invesrigación financiada pot el GIlD. A menos que se indique lo contrario, los derechos de autor corresponden a los autores. Para obrener copias adicionales, sirvase contactar a los autores o a la persona cuyo nombre aparece en la portada. Este doc- umento rambién esta disponible en la biblioreca del GIlD. IDRC. Lib. TELE CENTRE EVALUATION A GLOBAL PERSPECTIVE PAN GLOBAL NETWORKING INTERNATIONAL DEVELOPMENT RESEARCH CENTRE (IDRC) TELECENTRE EVALUATION A GLOBAL PERSPECTIVE Report of an International Meeting on Telecentre Evaluation Edited by Ricardo Gómez and Patrik Hunt, IDRC FAR HILLS INN, QUEBEC, CANADA SEPTEMBER 28-30, 1999 CONTENTS CONTENTS .1 PREFACE 3 REPORT ON THE MEETING 5 Guiding Princifiles of Telecentre Evaluation 5 Key issues and concerns ... Synthesis and Direction 7 Telecentre Evaluation and Research: a global perspective (R. GOMEZ, P. HUNT, B. LAMOUREUX) 13 PARTICIPANT CONTRIBUTIONS. 29 Knowledge Systemfor Sustainable Food Securzty (V BAIAJI) 31 Experiments in community access to new communication and information technologies in Bogota (Luis F. BARON) 35 Socialise the modem ofproduction - The role of telecentres in development (Peter BENJAMIN e.9'MonaDAI-IMS) 49 The UniversalAccess Model — Cabinas Publicas in Pern (Yuri HERRERA BURSTEIN).. 69 Computer and Communications Use in Low-Income Communities (Steve CISLER, et aL) .. 77 LearnLinle, Assessment and Evaluation (Eduardo CONTRERAS-BUDGE) 91 For the Educated People only ... Reflections on a Visit to two Multipurpose Community Tele centres in Uganda (Mona DAH!s'IS) 95 Establishing a Public Internet Centre in RuralAreas ofMongolia (Narangerel DANDAR)105 Learning Lessonsfrom Telecentres in Latin America and the Caribbean (Karin DELGADILLO, 109 Education forAll in the Information Age (Laurel ElMER) 121 Evaluating Tele centres within National Policies for ICTs in Developing Countries (Roger W HARRiS) 129 Gender analysis of telecentre evaluation methodology (Rebecca HOlMES) .137 Designing Research for Telecentre Evaluation (Heather E. HUDSON) 147 Telecentre Evaluation: A Tele-community Perspective (Loyola JOSEPH) 163 Buwama and Nabweru Community Telecentres: Baseline Surveys in Uganda (Samuel KYABWE Richard KtB 0MB 0) 169 RadioNet: Community Radio, Telecentres and Local Development (Emmanuelle LAMOUREUX) 193 Impact of the Internet: some conceptual and methodological issues, or how to hit a moving target behind the smoke screen (MichelJ. MENOU) 201 Multipurpose Community Telecenters in Selected Philippine Barangays (Merlita OPEIIA)21 7 On Estimating Telecentre Demand in Mexican Rural Municipios (Scott S. ROBINSON). 223 Telecenter evaluation issues - the Salvadoran experience (Clemente SAN SEBASTIAN)... 229 Telecentre Evaluation Methods and Instruments: What works and why? (George SCHARFFENBERGER) 235 Outcome Mapping: A Methodfor Reporting on Results SM UTYL 0) 249 Understanding the Role of Community Telecentres in Development - A Proposed Approach to Evaluation (Anne WHYTE) 269 2 PREFACE The contributions in this volume are the result of an international working meeting on telecentre evaluation, held at Far Hills, Quebec from September 28-30, 1999. At IDRC, we have a strong interest in learning about the introduction and use of information and communication technologies for development. Among the various approaches to increase access to information and communication technologies (ICTs), telecentre experiences are mushrooming around the world. Despite important differences in location, ownership, funding, operation, technology, and services, common telecentre experiences and lessons are emerging. We do not attempt here to provide a unique model for telecentres. Instead, the idea is to foster the creation and dissemination of knowledge relevant to the design, operation and evaluation of telecentres. IDRC supports a variety of telecentre experiences in Asia, Africa and Latin America; some represent collaborations with ITU, UNESCO, and other partners. A number of commissioned studies and papers have been prepared, and efforts toward systematic and useful evaluation frameworks and methodologies are currently being developed in Africa and Latin America. This global working session on telecentre evaluation has helped to create awareness about and strengthen these local, national and regional initiatives, while providing our partners in the South with an opportunity to exchange experiences and initial results with each other. The objectives of this meeting were the following: • To explore in depth the challenges and opportunities of telecentre evaluation in Latin America, Asia and Africa; • To understand and compare emerging evaluation frameworks and methodologies; • To assess the needs and resources available for telecentre baseline evaluation, monitoring, impact assessment, and to identify salient issues affecting telecentre performance; • To provide an opportunity for telecentre operators, project leaders, and researchers to exchange experiences and lessons across regions. These objectives may seem to be too simple for some. However, recognizing the emergent nature of telecentre issues, we preferred to start with humble expectations and work through a process of collective exploration and dialogue. One of the major results of this meeting has been the creation of a dynamic network of committed telecentre practitioners and researchers. We have now developed a common language and understanding of the key issues surrounding telecentre evaluation, as well as some practical tools. Our sincere thanks go out to the meeting participants who have contributed their rich collection of telecentre knowledge, as represented here. Ricardo Gomez, IDRC Ottawa, October 1999 The Output/Outcome/Downstream Impact Blues by Terry Let us tell you 'bout a word Now the donors say Getting too much use (And this is a fact) In development research Get out there and show us It's prone to abuse Your impact It's become an obsession You must change peoples' lives Now we're all in the act And help us take the credit 'Cause survival depends Or next time you want funding On the elusive... "impact". You just might not get it. Chorus: Chorus Impact any place Impact anytime Impact, they find, You may find it 'round the corner When it does occur, Or much farther down the line Comes from many factors It may happen in a way And we're not always sure That you did not choose, Which ones We got them output/outcome! Came from who Downstream impact blues. 'Cause impact is the product Of what many people do When donors look for impact They really want to see Hey donor wake up A pretty little picture From your impossible dream Of their fantasy You drop in your funding They've done with impact A long way upstream What donors often do These waters they flow, Jump on a bandwagon They mingle and blend Without thinking it through. So who can take credit What comes Out lfl the end? 4 REPORT ON THE MEETING Guiding Principles of Telecentre Evaluation The following synopsis is based on a brainstorming session involving all meeting participants, held at the conclusion of the gathering. It represents the group's collective determination of basic principles to guide our work in telecentre evaluation. Ideas were generated in response to this query: What basic, guiding will ensure that telecentre evaluation: (a) is useful; (b) isfinancially responsible,' (c) builds local capacity; and (d) enables shared learning. Guiding Principles: People-focused. participatory evaluation design. sensitive to gender and the local context (Focus on people, Participatory, Horizontal participation, Users and non-users, Contextually sensitive, Involve stakeholders early in the process, Build up from local cultural paradigms, Identify target groups, Enhance women's participation) Design the evaluation process to ensure quality and sustainability (Dynamic costing arrangements, Viable business, Constant quality of service with diminishing external support, Ensure that cost vs. resources are realistic) EL2: (Evaluation = listen and learn) (Use available expertise, Listen!) Both successes and failures will be analyzed through the evaluation process (Learn from failure and innovation) The evaluation result will be disaggregated and used to conduct training workshops (Oriented towards strengthening and empowering practitioners and intermediaries, Builds capacity tools) Clarify purpose (Make sure all stakeholders share a vision of what questions the evaluation needs to answer and how this information will be used, Focus on critical issues) Use adaptable and replicable methods (Replicable research design, Easy to do, Evolving Design for local context and project, Variety of methods, Adaptable, responsive frameworks, Transcend project boundaries)