DOCSLIB.ORG

What's Your Theory.Tips for Conducting Program Evaluation

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
What's Your Theory.Tips for Conducting Program Evaluation ISSUE 4 OCTOBER 2005 WHAT’S YOUR THEORY? Tips for conducting program evaluation A program theory is simply a clear explanation of why you do the things you do, given what you hope to accomplish. All human service programs are designed the things you do – your program activities – to make a difference in the lives of people should lead to the intended results or or to improve our society. But how? benefits. When you set out to design or redesign a Clarity and plain language are essential. program, you are choosing among many Can you explain it to your neighbor, or to options. For example, if you want to prevent your third cousin at a family reunion? violence or help people heal after being Don’t assume that your funders or staff victims of crime, how do you decide which members are any more sophisticated in activities to include? Since you can’t do their need for a clear description. everything that might help, which services are most important? Which ones need to Not only will a clear program theory help be combined with other services in order others see the sense of your program, but it to be most effective? will help you make sure you are actually providing the package of services that have These questions all get at your underlying the best possible chance of helping program theory. A program theory explains participants. And when it comes to how and why a program is supposed to evaluating your results, a clear program work. Spelling out that theory can be one theory makes it much easier to choose the of the most important things you do for the most appropriate outcomes (results) to success of your program. It provides a measure. logical and reasonable description of why The if-then connection Program theories can often be captured in “IF facilitators model nonviolent conflict a series of “if-then” statements – IF management strategies and provide something is done with or for the program opportunities for participants to practice participants, THEN something should these strategies, THEN participants will change. develop skills in nonviolent conflict management.” For example, a program to reduce aggression based on social learning theory A program theory should also spell out why could have an underlying theory like this: you expect the changes to happen. Between WHAT’S YOUR THEORY? Page 2 If-then connection, continued: the “if” and the “then,” there should be some example, many programs have a goal of solid evidence or some well-established changing some type of behavior. However, connection supporting the idea that your there are usually several things that have to service package will accomplish your happen first. People usually change their program goals. behavior after first learning some new information, developing a new skill, or A good program theory also reflects the changing their attitude about something. fact that change happens in stages. For Sample outline for a program theory IF a certain set of resources (such as For example, an after-school violence staff, equipment, materials) are prevention program could have the following available, THEN the program can theory: provide a certain set of activities or services to participants. Qualified counselors provide violence IF participants receive these services, prevention training to children. THEN they experience specific changes in their knowledge, attitudes, Children gain knowledge of nonviolent or skills. conflict resolution and coping strategies. IF individuals change their knowledge, attitudes, or skills, THEN they will Children practice non-violent conflict change their behavior and usual resolution and coping strategies. practice. IF enough participants change their Children reduce their violent behavior and behavior and practice, THEN the demonstrate improved social skills. program may have a broader impact on the families or friends of participants As a result of the reduced violent behavior or on the community as a whole. of individual children, violent behaviors in schools will decline. A gut feeling is not enough Some human service programs sound To avoid a shaky theory that leads to promising, but do not result in the desired disappointing results, go deeper than changes for participants. Of course, this common assumptions about how certain could be because a good theory is not activities lead to outcomes. Instead, being carried out well, but in some cases consider the available theories and research the problem is the theory itself. Make sure evidence that support these connections. In that your theory not only looks clear and this way, you can be more confident in the makes sense on paper, but that it is based underlying strength of your service on good underlying evidence about what delivery model. makes programs successful and how people really change. ISSUE 4 Page 3 If you’re not sure what the current research information. Review material presented at is showing, take some time to find out. the best conferences in your field to see Talk with colleagues in the field about what the latest research and evaluation what evidence they’ve seen lately. Look studies are showing. online or in a library for recent Here’s how to start To develop a program theory, select one of other programs, published research, or your activities and answer the following consistent feedback from participants)? three questions. IF the activity is provided, THEN what – Repeat the same three questions for each realistically – should be the result for activity or service that you provide. Don’t participants? worry, you don’t need to develop a theory WHY do you believe the activity will for everything! Administrative tasks, such lead to this result? (In other words, as training staff or doing paperwork, what is your assumption about how typically are not included in a program this kind of change occurs? Are you theory. These activities, while a necessary drawing from an established theory part of running a program, are usually not used by others? the important services that produce change What evidence do you have that the in participants. Focus on the main services activity will lead to this result (such as you provide – the ones you most count on previous results from your own or to promote positive results. How does a theory differ from a logic model? A program theory is similar in concept to A logic model uses short phrases to logic models, which have become represent things that you explain in more increasingly popular in human services detail in the program theory. Another key programs over the past several years. difference is that, while a logic model can (We’ll go into logic models in an just use an arrow to show that one thing upcoming Evaluation Tip Sheet.) leads to the next, your program theory needs to lay out the evidence to show why In simple terms, a logic model is a picture you believe one thing will lead to the next. of your theory – a drawing that shows how one thing leads to the next, like a flow chart. TIPS FOR CONDUCTING PROGRAM EVALUATION Page 4 Quick links to more information Theory of Change http://www.theoryofchange.org Making Evaluation Integral to Your Asset-Building Initiative: Employing a Theory of Action and Change http://www.search-institute.org/research/knowledge/MakingEvaluationIntegral.html Theory of Change: A Practical Tool for Action, Results and Learning http://www.aecf.org/initiatives/mc/readingroom/documents/ theoryofchangemanual.pdf In future tip sheets What a logic model can do for you Building a logic model Prioritizing evaluation questions Find previous tip sheets on the web: www.ojp.state.mn.us/grants/index.htm or www.wilderresearch.org. October 2005 For more information or additional copies, contact: Cecilia Miller Author: Cheryl Hosley Minnesota Office of Justice Programs Wilder Research [email protected] www.wilderresearch.org 651-205-4817 .
Load more

Recommended publications
  • Theory of Change and Organizational Development Strategy INTRODUCTION Dear Reader
    Theory of Change and Organizational Development Strategy INTRODUCTION Dear Reader: The Fetzer Institute was founded in 1986 by John E. Fetzer with a vision of a transformed world, powered by love, in which all people can flourish. Our current mission, adopted in 2016, is to help build the spiritual foundation for a loving world. Over the past several years, we have been identifying and exploring new ways to make our vision of a loving world a reality. One aspect of this work has been the development of a new conceptual frame resulting in a comprehensive Theory of Change. This document aspires to ground our work for the next 25 years. Further, this in-depth articulation of our vision allows us to invite thought leaders across disciplines to help sharpen our thinking. This document represents a moment when many strands of work and planning by the Institute board and staff came together in a very powerful way that enabled us to articulate our Theory of Change. However, this is a dynamic, living document, and we encourage you to read it as such. For example, we are actively developing detailed goals and action plans, and we continue to examine our conceptual frame, even as we make common cause with all who are working toward a shared and transformative sacred story for humanity in the 21st century. We continue to use our Theory of Change to focus our work, inspire and grow our partnerships, and identify the most pressing needs in our world. I look forward to your feedback and invite you to learn about how our work is coming alive in the world through our program strategies, initiatives, and stories at Fetzer.org.
    [Show full text]
  • Sacred Rhetorical Invention in the String Theory Movement
    University of Nebraska - Lincoln DigitalCommons@University of Nebraska - Lincoln Communication Studies Theses, Dissertations, and Student Research Communication Studies, Department of Spring 4-12-2011 Secular Salvation: Sacred Rhetorical Invention in the String Theory Movement Brent Yergensen University of Nebraska-Lincoln, [email protected] Follow this and additional works at: https://digitalcommons.unl.edu/commstuddiss Part of the Speech and Rhetorical Studies Commons Yergensen, Brent, "Secular Salvation: Sacred Rhetorical Invention in the String Theory Movement" (2011). Communication Studies Theses, Dissertations, and Student Research. 6. https://digitalcommons.unl.edu/commstuddiss/6 This Article is brought to you for free and open access by the Communication Studies, Department of at DigitalCommons@University of Nebraska - Lincoln. It has been accepted for inclusion in Communication Studies Theses, Dissertations, and Student Research by an authorized administrator of DigitalCommons@University of Nebraska - Lincoln. SECULAR SALVATION: SACRED RHETORICAL INVENTION IN THE STRING THEORY MOVEMENT by Brent Yergensen A DISSERTATION Presented to the Faculty of The Graduate College at the University of Nebraska In Partial Fulfillment of Requirements For the Degree of Doctor of Philosophy Major: Communication Studies Under the Supervision of Dr. Ronald Lee Lincoln, Nebraska April, 2011 ii SECULAR SALVATION: SACRED RHETORICAL INVENTION IN THE STRING THEORY MOVEMENT Brent Yergensen, Ph.D. University of Nebraska, 2011 Advisor: Ronald Lee String theory is argued by its proponents to be the Theory of Everything. It achieves this status in physics because it provides unification for contradictory laws of physics, namely quantum mechanics and general relativity. While based on advanced theoretical mathematics, its public discourse is growing in prevalence and its rhetorical power is leading to a scientific revolution, even among the public.
    [Show full text]
  • What Scientific Theories Could Not Be Author(S): Hans Halvorson Reviewed Work(S): Source: Philosophy of Science, Vol
    What Scientific Theories Could Not Be Author(s): Hans Halvorson Reviewed work(s): Source: Philosophy of Science, Vol. 79, No. 2 (April 2012), pp. 183-206 Published by: The University of Chicago Press on behalf of the Philosophy of Science Association Stable URL: http://www.jstor.org/stable/10.1086/664745 . Accessed: 03/12/2012 10:32 Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at . http://www.jstor.org/page/info/about/policies/terms.jsp . JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. The University of Chicago Press and Philosophy of Science Association are collaborating with JSTOR to digitize, preserve and extend access to Philosophy of Science. http://www.jstor.org This content downloaded by the authorized user from 192.168.52.67 on Mon, 3 Dec 2012 10:32:52 AM All use subject to JSTOR Terms and Conditions What Scientific Theories Could Not Be* Hans Halvorson†‡ According to the semantic view of scientific theories, theories are classes of models. I show that this view—if taken literally—leads to absurdities. In particular, this view equates theories that are distinct, and it distinguishes theories that are equivalent. Furthermore, the semantic view lacks the resources to explicate interesting theoretical relations, such as embeddability of one theory into another.
    [Show full text]
  • Introduction to Philosophy of Science
    INTRODUCTION TO PHILOSOPHY OF SCIENCE The aim of philosophy of science is to understand what scientists did and how they did it, where history of science shows that they performed basic research very well. Therefore to achieve this aim, philosophers look back to the great achievements in the evolution of modern science that started with the Copernicus with greater emphasis given to more recent accomplishments. The earliest philosophy of science in the last two hundred years is Romanticism, which started as a humanities discipline and was later adapted to science as a humanities specialty. The Romantics view the aim of science as interpretative understanding, which is a mentalistic ontology acquired by introspection. They call language containing this ontology “theory”. The most successful science sharing in the humanities aim is economics, but since the development of econometrics that enables forecasting and policy, the humanities aim is mixed with the natural science aim of prediction and control. Often, however, econometricians have found that successful forecasting by econometric models must be purchased at the price of rejecting equation specifications based on the interpretative understanding supplied by neoclassical macroeconomic and microeconomic theory. In this context the term “economic theory” means precisely such neoclassical equation specifications. Aside from economics Romanticism has little relevance to the great accomplishments in the history of science, because its concept of the aim of science has severed it from the benefits of the examination of the history of science. The Romantic philosophy of social science is still resolutely practiced in immature sciences such as sociology, where mentalistic description prevails, where quantification and prediction are seldom attempted, and where implementation in social policy is seldom effective and often counterproductive.
    [Show full text]
  • Theory Use and Usefulness in Scientific Advancement
    Editorial Theory Use and Usefulness in Scientific Advancement Rita H. Pickler his issue of Nursing Research is focused on theory in This would particularly be the case if theories for nursing Tnursing, including how theories historically devel- science were focused on understanding, explaining, and oped in nursing and how they have evolved over time. predicting those phenomena that are of interest to our dis- Also included in this issue are new thoughts on theory develop- cipline and, by extension, to our practice. Our conundrum ment as well as emergent theoretical ideas. But to what end is here is that the phenomena of interest in nursing science theory in the rapidly changing scientific world? Do theories and practice are highly varied and—importantly—not agreed and, particularly, “grand” theories of the sort that many of us upon by the discipline. Many years ago, the historically impor- of a “certain age” were required to “use” in our own first efforts tant grand theories identified broad phenomena of interest— at research lead to scientific advancement? Is theory useful in person, environment, health, and nursing. In the ensuing today’s seemingly atheoretical age where new scientific “dis- years, these broad phenomena have been challenged, modified, coveries” are announced daily on social media? Do we need or even discounted as meaningless because of their broad- theory to advance science? ness. And yet, it seems that much of what we study as nurse Like most scientists, I was taught very early in my train- scientists does fall into one or more of these broad categories ing that a theory is a set of interrelated concepts and proposi- of phenomena.
    [Show full text]
  • Toc Basics Guide Copy
    ActKnowledge, 365$Fi'h$Avenue,$6th$Floor New$York,$NY$10016 Telephone$212.817.1906 www.actknowledge.org$$ Theory,of,Change,Basics A,PRIMER,ON,THEORY,OF,CHANGE Dana,H.,Taplin,,Ph.D. Heléne,Clark,,Ph.D. March,2012 1 Theory of Change Basics Introduction August 26, 2011 Volume VII Theory of transparent so that everyone involved knows what change is a is happening and why. To be clear, every outcome in rigorous yet the theory is explicitly defined. All outcomes should participatory be given one or more indicators of success. As process implementation proceeds, organizations collect and whereby analyze data on key indicators as a means of groups and monitoring progress on the theory of change. stakeholders in Indicator data show whether changes are taking a planning place as forecast or not. Using the indicator data process program staff can adjust and revise their change articulate their model as they learn more about what works and long-term goals what does not. and identify the conditions they Rationales in a theory of change explain the believe have to connections between the outcomes and why one unfold for outcome is needed to achieve another. those goals to Assumptions explain the contextual be met. These underpinnings of the theory. Often, rationales and conditions are assumptions are supported by research, strengthening the plausibility of the theory and the modeled as 1 desired likelihood that its stated goals can be achieved . The graphic model in theory of change is accompanied outcomes, arranged graphically in a causal framework. by a written narrative that explains the logic of the framework.
    [Show full text]
  • Plato's Theory of Desire Author(S): Charles H
    Plato's Theory of Desire Author(s): Charles H. Kahn Source: The Review of Metaphysics, Vol. 41, No. 1 (Sep., 1987), pp. 77-103 Published by: Philosophy Education Society Inc. Stable URL: https://www.jstor.org/stable/20128559 Accessed: 20-05-2019 12:36 UTC JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at https://about.jstor.org/terms Philosophy Education Society Inc. is collaborating with JSTOR to digitize, preserve and extend access to The Review of Metaphysics This content downloaded from 145.107.82.110 on Mon, 20 May 2019 12:36:45 UTC All use subject to https://about.jstor.org/terms PLATO'S THEORY OF DESIRE CHARLES H. KAHN 1YJ.Y aim here is to make sense of Plato's account of desire in the middle dialogues. To do that I need to unify or reconcile what are at first sight two quite different accounts: the doctrine of eros in the Symposium and the tripartite theory of motivation in the Repub lic.1 It may be that the two theories are after all irreconcilable, that Plato simply changed his mind on the nature of human desire after writing the Symposium and before composing the Republic. But that conclusion can be justified only if attempts to reconcile the two theories end in failure.
    [Show full text]
  • A Brief History of Mathematics a Brief History of Mathematics
    A Brief History of Mathematics A Brief History of Mathematics What is mathematics? What do mathematicians do? A Brief History of Mathematics What is mathematics? What do mathematicians do? http://www.sfu.ca/~rpyke/presentations.html A Brief History of Mathematics • Egypt; 3000B.C. – Positional number system, base 10 – Addition, multiplication, division. Fractions. – Complicated formalism; limited algebra. – Only perfect squares (no irrational numbers). – Area of circle; (8D/9)² Æ ∏=3.1605. Volume of pyramid. A Brief History of Mathematics • Babylon; 1700‐300B.C. – Positional number system (base 60; sexagesimal) – Addition, multiplication, division. Fractions. – Solved systems of equations with many unknowns – No negative numbers. No geometry. – Squares, cubes, square roots, cube roots – Solve quadratic equations (but no quadratic formula) – Uses: Building, planning, selling, astronomy (later) A Brief History of Mathematics • Greece; 600B.C. – 600A.D. Papyrus created! – Pythagoras; mathematics as abstract concepts, properties of numbers, irrationality of √2, Pythagorean Theorem a²+b²=c², geometric areas – Zeno paradoxes; infinite sum of numbers is finite! – Constructions with ruler and compass; ‘Squaring the circle’, ‘Doubling the cube’, ‘Trisecting the angle’ – Plato; plane and solid geometry A Brief History of Mathematics • Greece; 600B.C. – 600A.D. Aristotle; mathematics and the physical world (astronomy, geography, mechanics), mathematical formalism (definitions, axioms, proofs via construction) – Euclid; Elements –13 books. Geometry,
    [Show full text]
  • Theory Ahead of Business Cycle Measurement (P
    Federal Reserve Bank of Minneapolis Modern Business Cycle Analysis: A Guide to the Prescott-Summers Debate (p. 3) Rodolfo E. Manuelli Theory Ahead of Business Cycle Measurement (p. 9) Edward C. Prescott Some Skeptical Observations on Real Business Cycle Theory (p. 23) Lawrence H. Summers Response to a Skeptic (p. 28) Edward C. Prescott Federal Reserve Bank of Minneapolis Quarterly Review Vol. 10, NO. 4 ISSN 0271-5287 This publication primarily presents economic research aimed at improving policymaking by the Federal Reserve System and other governmental authorities. Produced in the Research Department. Edited by Preston J. Miller and Kathleen S. Rolte. Graphic design by Phil Swenson and typesetting by Barb Cahlander and Terri Desormey, Graphic Services Department. Address questions to the Research Department, Federal Reserve Bank, Minneapolis, Minnesota 55480 (telephone 612-340-2341). Articles may be reprinted it the source is credited and the Research Department is provided with copies of reprints. The views expressed herein are those of the authors and not necessarily those of the Federal Reserve Bank of Minneapolis or the Federal Reserve System. Federal Reserve Bank of Minneapolis Quarterly Review Fall 1986 Theory Ahead of Business Cycle Measurement* Edward C. Prescott Adviser Research Department Federal Reserve Bank of Minneapolis and Professor of Economics University of Minnesota Economists have long been puzzled by the observations restricted to those that generate the growth observa- that during peacetime industrial market economies tions. The process governing the technology parameter display recurrent, large fluctuations in output and is selected to be consistent with the measured tech- employment over relatively short time periods.
    [Show full text]
  • Passmore, J. (1967). Logical Positivism. in P. Edwards (Ed.). the Encyclopedia of Philosophy (Vol. 5, 52- 57). New York: Macmillan
    Passmore, J. (1967). Logical Positivism. In P. Edwards (Ed.). The Encyclopedia of Philosophy (Vol. 5, 52- 57). New York: Macmillan. LOGICAL POSITIVISM is the name given in 1931 by A. E. Blumberg and Herbert Feigl to a set of philosophical ideas put forward by the Vienna circle. Synonymous expressions include "consistent empiricism," "logical empiricism," "scientific empiricism," and "logical neo-positivism." The name logical positivism is often, but misleadingly, used more broadly to include the "analytical" or "ordinary language philosophies developed at Cambridge and Oxford. HISTORICAL BACKGROUND The logical positivists thought of themselves as continuing a nineteenth-century Viennese empirical tradition, closely linked with British empiricism and culminating in the antimetaphysical, scientifically oriented teaching of Ernst Mach. In 1907 the mathematician Hans Hahn, the economist Otto Neurath, and the physicist Philipp Frank, all of whom were later to be prominent members of the Vienna circle, came together as an informal group to discuss the philosophy of science. They hoped to give an account of science which would do justice -as, they thought, Mach did not- to the central importance of mathematics, logic, and theoretical physics, without abandoning Mach's general doctrine that science is, fundamentally, the description of experience. As a solution to their problems, they looked to the "new positivism" of Poincare; in attempting to reconcile Mach and Poincare; they anticipated the main themes of logical positivism. In 1922, at the instigation of members of the "Vienna group," Moritz Schlick was invited to Vienna as professor, like Mach before him (1895-1901), in the philosophy of the inductive sciences. Schlick had been trained as a scientist under Max Planck and had won a name for himself as an interpreter of Einstein's theory of relativity.
    [Show full text]
  • The Example of Self-Determination Theory-Based Interventions Marlene N
    Silva, Marques, & Teixeira STD, theory and practice original article Testing theory in practice: The example of self-determination theory-based interventions Marlene N. Silva Leonardo da Vinci once tested the application of the TCS, investigating i) the University of Lisbon said that, “He who loves extent and type of theory use in health behavior ULHT practice without theory is change interventions to increase physical activity and Marta M. Marques like the sailor who boards healthy eating, and ii) the associations between University of Lisbon ship without a rudder and theory use and intervention effectiveness. The Pedro J. Teixeira compass and never knows authors found poor reporting on the application of University of Lisbon where he may cast”. theory in intervention design and evaluation. For Similarly, advancing example, few interventions targeted and measured behavioral science requires a good understanding of changes in all theoretical constructs defined by the how interventions are informed by theory, how they theory or linked all the behavior change techniques can better test theory, and which behavior change to those constructs. Since this meta-analysis tested techniques should be selected as a function of theory the TCS framework with only two theories (Social (or theories). However, simply claiming that an Cognitive Theory [SCT] and the Transtheoretical intervention is theory-based does not necessarily Model), more research is needed to test the fidelity to make it so. Critical evaluation of applied theory is theory in health behavior change interventions based needed for a more integrated understanding of on other frameworks and its differential impact on behavior change interventions, their usefulness, and interventions effectiveness.
    [Show full text]
  • Hart's Methodological Positivism
    University of Pennsylvania Carey Law School Penn Law: Legal Scholarship Repository Faculty Scholarship at Penn Law 1998 Hart's Methodological Positivism Stephen R. Perry University of Pennsylvania Carey Law School Follow this and additional works at: https://scholarship.law.upenn.edu/faculty_scholarship Part of the Ethics and Political Philosophy Commons, Jurisprudence Commons, Law and Society Commons, Legal History Commons, and the Legal Theory Commons Repository Citation Perry, Stephen R., "Hart's Methodological Positivism" (1998). Faculty Scholarship at Penn Law. 1136. https://scholarship.law.upenn.edu/faculty_scholarship/1136 This Article is brought to you for free and open access by Penn Law: Legal Scholarship Repository. It has been accepted for inclusion in Faculty Scholarship at Penn Law by an authorized administrator of Penn Law: Legal Scholarship Repository. For more information, please contact [email protected]. Legal Theory http://journals.cambridge.org/LEG Additional services for Legal Theory: Email alerts: Click here Subscriptions: Click here Commercial reprints: Click here Terms of use : Click here Hart's Methodological Positivism Stephen R. Perry Legal Theory / Volume 4 / Issue 04 / December 1998, pp 427 - 467 DOI: 10.1017/S1352325200001105, Published online: 16 February 2009 Link to this article: http://journals.cambridge.org/abstract_S1352325200001105 How to cite this article: Stephen R. Perry (1998). Hart's Methodological Positivism. Legal Theory, 4, pp 427-467 doi:10.1017/S1352325200001105 Request Permissions : Click here Downloaded from http://journals.cambridge.org/LEG, IP address: 130.91.146.35 on 10 Dec 2013 Legal Theory, 4 (1998), 427-467. Printed in the United States of America Copyright © Cambridge University Press 1352-3252/98 $9.50 HART'S METHODOLOGICAL POSITIVISM Stephen R.
    [Show full text]