Focusing the Evaluation

Once you have developed a logic model that depicts your program’s main components, you’ll need to determine which program activities in your logic model should be evaluated based on your stakeholders’ needs and available resources. How do you choose the focus of your evaluation? 1. Decide on the purpose of your evaluation. The purpose of your evaluation is what you intend to learn from the evaluation. This serves as the basis for the evaluation design, questions, and methods. Examples of purposes include: to improve program operations, determine the effects of the program in the target population(s), and gain knowledge about program activities. 2. Identify the users of your evaluation results. The users of your evaluation are the specific persons who receive and use the evaluation findings (e.g., stakeholders). Support from the users increases the likelihood that the evaluation results will be used to improve the program or program activity under evaluation. 3. Identify the uses of your evaluation results. The uses of your evaluation are the specific ways that program staff and stakeholders plan to utilize the evaluation findings. To identify the uses of your evaluation, program staff and the various stakeholders should discuss the different expectations and needs they have for the evaluation. Examples of uses of evaluation findings include: deciding how to allocate resources, deciding whether to expand the locations where a program activity is carried out, identifying program areas that need improvement, documenting the level of success in achieving objectives, and soliciting more funds. Once program staff and stakeholders have identified their expectations for conducting an evaluation, you will need to prioritize the various uses of the evaluation. 4. Identify the stage of development of your program to determine if the focus of your evaluation is realistic. With the implementation of a new STD program activity, consider conducting a process evaluation to determine if the program is being delivered as planned and if improvements are needed. An evaluation that included outcomes would make little sense at this stage. With a more mature program, you may want to conduct an outcome evaluation to assess your program’s effectiveness and to demonstrate that it is making productive use of resources.

TIP: Learn more about process and outcome evaluations by reviewing the related short-guide on Types of Evaluation.

For more information and examples, see Step 3.1 in the Practical Use of Program Evaluation among STD Programs manual. http://www.cdc.gov/std/program/pupestd.htm

National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention Division of STD Prevention

CS249668 How do you prioritize your evaluation focus? Determining which component of a program to evaluate is a critical step in the evaluation planning process and will require prioritization. The list below provides key criteria you may want to consider adopting as part of your prioritization process. You may also identify pertinent criteria not on this list. Each program should discuss potential prioritization criteria with appropriate stakeholders to decide which combination of criterion is relevant for the program in deciding which program component to evaluate at a particular time. Criteria for Prioritizing Evaluation Focus Areas You may consider looking at activities within each of the major program components separately so there’s no need to apply the same criteria for each component. The same criteria may not fit well with all parts of your program. For example, criteria that fit partner services might not apply to surveillance or partnership activities.

Criterion Question to Ask High scores on your selected criteria indicate good candidates for targeted evaluation Information need How critical is the evaluation information for making near-term decisions? Stakeholder interest What is the level of interest for this activity among our stakeholders? Use How likely are the intended audiences to use results or recommendations from this evaluation? Cost What financial resources do we have for this activity? Labor/time intensity How much staff time will we need to invest for this activity? Maturity What is the stage of development or implementation for this activity? Prior evaluation Have we evaluated this activity before? Pilot Do we plan to expand this activity? Purpose What do we intend to gain from the evaluation activities? Plausible outcomes Can this activity reasonably be expected to lead to relevant outcomes? Centrality How connected is this activity to our STD partners across the state? Plan alignment How closely aligned is this activity with our state STD plan? Sustainability How much does this activity contribute to the sustainability of the state STD program? Improvements Would evaluating this activity likely result in recommendations for programmatic improvement? Challenges Are we (or do we anticipate) struggling with this activity? Reach How many people in our state are (or could be) affected by this activity? Focus Does this activity affect those most burdened by STD? Disparities Will this activity reduce STD disparities?

Consider selecting a limited number of criteria in order to make the prioritization process more manageable. In a priority-setting process such as this, deciding which criteria are not important is just as vital as deciding which are important to the stakeholders. A final list of criteria should be selected and each activity ranked as high, medium, or low priority against each criterion. Those activities ranked highest across multiple criteria are the best candidates to be considered for evaluation.

Source: CDC. Learning and Growing through Evaluation: State Asthma Program Evaluation Guide. http://www.cdc.gov/asthma/program_eval/Evaluation_Guide_COMBINED_FINAL_BOOKMARKED.pdf

Focusing the Evaluation