ISSUE 4 OCTOBER 2005

WHAT’S YOUR THEORY?

Tips for conducting program evaluation

A program theory is simply a clear explanation of why you do the things you do, given what you hope to accomplish.

All human service programs are designed the things you do – your program activities – to make a difference in the lives of people should lead to the intended results or or to improve our society. But how? benefits.

When you set out to design or redesign a Clarity and plain language are essential. program, you are choosing among many Can you explain it to your neighbor, or to options. For example, if you want to prevent your third cousin at a family reunion? violence or help people heal after being Don’t assume that your funders or staff victims of crime, how do you decide which members are any more sophisticated in activities to include? Since you can’t do their need for a clear description. everything that might help, which services are most important? Which ones need to Not only will a clear program theory help be combined with other services in order others see the sense of your program, but it to be most effective? will help you make sure you are actually providing the package of services that have These questions all get at your underlying the best possible chance of helping program theory. A program theory explains participants. And when it comes to how and why a program is supposed to evaluating your results, a clear program work. Spelling out that theory can be one theory makes it much easier to choose the of the most important things you do for the most appropriate outcomes (results) to success of your program. It provides a measure. logical and reasonable description of why

The if-then connection Program theories can often be captured in “IF facilitators model nonviolent conflict a series of “if-then” statements – IF management strategies and provide something is done with or for the program opportunities for participants to practice participants, THEN something should these strategies, THEN participants will change. develop skills in nonviolent conflict management.” For example, a program to reduce aggression based on social learning theory A program theory should also spell out why could have an underlying theory like this: you expect the changes to happen. Between WHAT’S YOUR THEORY?

Page 2

If-then connection, continued: the “if” and the “then,” there should be some example, many programs have a goal of solid evidence or some well-established changing some type of behavior. However, connection supporting the idea that your there are usually several things that have to service package will accomplish your happen first. People usually change their program goals. behavior after first learning some new information, developing a new skill, or A good program theory also reflects the changing their attitude about something. fact that change happens in stages. For

Sample outline for a program theory

IF a certain set of resources (such as For example, an after-school violence staff, equipment, materials) are prevention program could have the following available, THEN the program can theory: provide a certain set of activities or services to participants. Qualified counselors provide violence IF participants receive these services, prevention training to children. THEN they experience specific changes in their knowledge, attitudes, Children gain knowledge of nonviolent or skills. conflict resolution and coping strategies. IF individuals change their knowledge, attitudes, or skills, THEN they will Children practice non-violent conflict change their behavior and usual resolution and coping strategies. practice. IF enough participants change their Children reduce their violent behavior and behavior and practice, THEN the demonstrate improved social skills. program may have a broader impact on the families or friends of participants As a result of the reduced violent behavior or on the community as a whole. of individual children, violent behaviors in schools will decline.

A gut feeling is not enough

Some human service programs sound To avoid a shaky theory that leads to promising, but do not result in the desired disappointing results, go deeper than changes for participants. Of course, this common assumptions about how certain could be because a good theory is not activities lead to outcomes. Instead, being carried out well, but in some cases consider the available theories and research the problem is the theory itself. Make sure evidence that support these connections. In that your theory not only looks clear and this way, you can be more confident in the makes sense on paper, but that it is based underlying strength of your service on good underlying evidence about what delivery model. makes programs successful and how people really change.

ISSUE 4

Page 3

If you’re not sure what the current research information. Review material presented at is showing, take some time to find out. the best conferences in your field to see Talk with colleagues in the field about what the latest research and evaluation what evidence they’ve seen lately. Look studies are showing. online or in a library for recent

Here’s how to start

To develop a program theory, select one of other programs, published research, or your activities and answer the following consistent feedback from participants)? three questions. IF the activity is provided, THEN what – Repeat the same three questions for each realistically – should be the result for activity or service that you provide. Don’t participants? worry, you don’t need to develop a theory WHY do you believe the activity will for everything! Administrative tasks, such lead to this result? (In other words, as training staff or doing paperwork, what is your assumption about how typically are not included in a program this kind of change occurs? Are you theory. These activities, while a necessary drawing from an established theory part of running a program, are usually not used by others? the important services that produce change What evidence do you have that the in participants. Focus on the main services activity will lead to this result (such as you provide – the ones you most count on previous results from your own or to promote positive results.

How does a theory differ from a logic model?

A program theory is similar in concept to A logic model uses short phrases to logic models, which have become represent things that you explain in more increasingly popular in human services detail in the program theory. Another key programs over the past several years. difference is that, while a logic model can (We’ll go into logic models in an just use an arrow to show that one thing upcoming Evaluation Tip Sheet.) leads to the next, your program theory needs to lay out the evidence to show why In simple terms, a logic model is a picture you believe one thing will lead to the next. of your theory – a drawing that shows how one thing leads to the next, like a flow chart.

TIPS FOR CONDUCTING PROGRAM EVALUATION

Page 4

Quick links to more information Theory of Change http://www.theoryofchange.org

Making Evaluation Integral to Your Asset-Building Initiative: Employing a Theory of Action and Change http://www.search-institute.org/research/knowledge/MakingEvaluationIntegral.html

Theory of Change: A Practical Tool for Action, Results and Learning http://www.aecf.org/initiatives/mc/readingroom/documents/ theoryofchangemanual.pdf

In future tip sheets

What a logic model can do for you Building a logic model Prioritizing evaluation questions

Find previous tip sheets on the web: www.ojp.state.mn.us/grants/index.htm or www.wilderresearch.org.

October 2005 For more information or additional copies, contact: Cecilia Miller Author: Cheryl Hosley Minnesota Office of Justice Programs Wilder Research [email protected] www.wilderresearch.org 651-205-4817