Informes e Participações Empowerment evaluation in Brazil: building capacity and facilitating self-determination David Fetterman School of Medicine And School of Education Stanford University Resumo mpowerment evaluation is a global The effort was sponsored by Cesgranrio phenomenon. It is being practiced in the Foundation. Ana Carolina Letichevsky and United States, Finland, United Kingdom, Angela Dannemann helped organized the South Africa, Japan, and Brazil, among conference. This brief discussion highlights many other countries.1 Dr. Thereza Penna the three steps of empowerment evaluation Firme recently invited Dr. Fetterman to and the 10 principles guiding the approach. discuss the most recent developments in empowerment evaluation and provide an Dr. Thereza Penna Firme empowerment evaluation in Rio de Janeiro, and Dr. Fetterman at the Brazil, December 2004. Cesgranrio Foundation Overview Empowerment evaluation is the use of evaluation concepts, techniques, and findings to foster improvement and self-determination (FETTERMAN, 2000; FETTERMAN; KAFTARIAN; WANDERSMAN, 1996). It is guided by a commitment to truth and honesty (FETTERMAN, 1998). It is designed to help people help themselves and improve their Empowerment Evaluation Workshop in programs using a form of self-evaluation and Rio de Janeiro Poster reflection. Program participants – including 1 See empowerment evaluation web pages including http://homepage.mac.com/profdavidf and http://www.stanford.edu/~davi- df/empowermentevaluation.html. Ensaio: aval. pol. públ. Educ., Rio de Janeiro, v.12, n.45, p. 1065-1075, out./dez. 2004 1066 David Fetterman, School of Medicine And School of Education Stanford University clients, consumers, and staff members – open session with as many staff members conduct their own evaluations; an outside and participants as possible. evaluator often serves as a coach or additional facilitator depending on internal program Participants are asked to generate key capabilities. By internalizing and phrases that capture the mission of the institutionalizing self-evaluation processes and program or project. This is done even when practices, a dynamic and responsive approach an existing mission statement exists, because to evaluation can be developed. there are typically many new participants and the initial document may or may not have There are three steps involved in helping been generated in a democratic open forum. others learn to evaluate their own programs: Proceeding in this fashion allows fresh new 1) developing a mission, vision, or unifying ideas to become a part of the mission and it purpose; 2) taking stock or determining also allows participants an opportunity to where the program stands, including voice their vision of the program. It is strengths and weaknesses; and 3) planning common for groups to learn how divergent for the future by establishing goals and their participants’ views are about the helping participants determine their own program, even when they have been working strategies to accomplish program goals and together for years. The evaluator records objectives. In addition, empowerment these phrases, typically on a poster sheet. evaluators help program staff members and participants determine the type of evidence Then a workshop participant is asked to required to document and monitor progress volunteer to write these telescopic phrases into credibly toward their goals. These steps a paragraph or two. This document is shared combined help to create a “communicative with the group, revisions and corrections are space” (VANDERPLAAT, 1995) to facilitate made in the process, and then the group is emancipatory and “communicative action” asked to accept the document on a consensus (HABERMAS,1984). basis: that is, they do not have to be in favor of 100% of the document; they just have to be willing to live with it. The mission statement represents the values of the group, and as Empowerment Evaluation such, represents the foundation for the next 3 Steps step, taking stock. 1. Mission 2. Taking Stock 3. Planning for the Future Mission Mission G Facilitate development of The first step in an empowerment statement evaluation is to ask program staff members G Group and participants to define their mission. This G Democratic step can be accomplished in a few hours. G Making meaning & giving voice An empowerment evaluator facilitates an Ensaio: aval. pol. públ. Educ., Rio de Janeiro, v.12, n.45, p. 1065-1075, out./dez. 2004 Empowerment evaluation in Brazil: building capacity and facilitating self-determination 1067 Taking Stock The second phase of taking stock involves rating the activities. Program staff members and The second step in an empowerment participants are asked to rate how well they are evaluation is taking stock. This step can also be conducted in a few hours, and has two sections. doing concerning each activity on a 1 to 10 The first involves generating a list of key activities scale, with 10 as the highest level and 1 as the that are crucial to the functioning of the program. lowest. The staff members and participants only Once again, the empowerment evaluator serves have minimal definitions about the components as a facilitator, asking program staff members or activities at this point. Addition clarification and participants to list the most significant features can be pursued as needed; however, detailed and/or activities associated with the program. A definition and clarification become a significant list of 10 to 20 activities is sufficient. After part of the later dialogue process. (The group generating this list, it is time to prioritize and will never reach the rating stage if each activity determine which are the most important activities is perfectly defined at this point. The rating meriting evaluation at this time. process then sets the stage for dialogue, clarification, and communication.) One tool used to minimize the time associated with prioritizing activities involves voting with dots. Typically, participants rate each of the The empowerment evaluator gives each activities while in their seats on their own participant five dot stickers, and asks the piece of paper. Then they are asked to come participants to place them by the activity on which up to the front of the room and record their the participant wants to focus. The participant ratings on a poster sheet of paper. This can distribute them across five different activities allows for some degree of independence in or place all five on one activity. Counting the rating. In addition, it minimizes a long stream dots easily identifies the top 10 activities. The 10 of second-guessing and checking to see activities with the most dots become the prioritized what others are rating the same activities. list of activities meriting evaluation at that time. (This process avoids long arguments about why At the same time, there is nothing one activity is valued more than another is, when confidential about the process. Program staff both activities are included in the list of the top members and participants place their initials 10 program activities anyway.) at the top of the matrix and then record their ratings for each activity. Contrary to most research designs, this system is designed to Taking Stock ensure that everyone knows and is influenced Part I by each other’s ratings (after recording them on the poster sheet). This is part of the G List socialization process that takes place in an empowerment evaluation, opening up the G Prioritize (dots) discussion and stepping toward more open Activities Priorization with Dots disclosure – speaking one’s truth. Communication GGGG Product GGGGGGGG Fundraising GGGGGG The taking stock phase of an empowerment evaluation is conducted in an open setting for Empowerment Evaluation Brazilian three reasons: 1) it creates a democratic flow Workshop Participant “Taking Stock Part I” of information and exchange of information; Ensaio: aval. pol. públ. Educ., Rio de Janeiro, v.12, n.45, p. 1065-1075, out./dez. 2004 1068 David Fetterman, School of Medicine And School of Education Stanford University 2) it makes it more difficult for managers to empowerment evaluator facilitates a retaliate because it is in an open forum; and discussion about the ratings. A survey would 3) it increases the probability that the have accomplished the same task up to this disclosures will be diplomatic, because point. However, the facilitator probes and program staff members and participants must asks why one person rated communication remain in that environment. Open discussions a 6, whereas two others rated it a 3 on the in a vacuum, without regard for workplace matrix.2 Participants are asked to explain norms, are not productive. They are often their rating and provide evidence or unrealistic and can be counter-productive. documentation to support the rating. This plants the seeds for the next stage of Staff members and participants are more empowerment evaluation, planning for the likely to give their program a higher rating if future, where they will need to specify the they are only asked to give an overall or gestalt evidence they plan to use to document that rating about the program. Consequently, it is their activities are helping them accomplish important that program staff members and their goals. The empowerment evaluator participants be asked to begin by assessing serves as a critical friend during this stage, individual program activities. They are more facilitating discussion and making sure likely to give some activities low ratings if they everyone is heard, and at the same time are given an equal opportunity to speak being critical and asking, “What do you positively about, or rate, other activities highly. mean by that?” or asking for additional The ratings can be totaled and averaged by clarification and substantiation about a person and by activity. This provides some particular rating or viewpoint. insight into routinely optimistic and pessimistic participants. It allows participants to see where Participants are asked for both the positive they stand in relation to their peers, which helps and negative basis for their ratings.