DOCSLIB.ORG

Working with Assumptions: Understanding How Evaluators Make Decisions About Capturing Reality/Context* Jonathan A

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Working with Assumptions: Understanding How Evaluators Make Decisions About Capturing Reality/Context* Jonathan A Working with Assumptions: Understanding how Evaluators Make Decisions about Capturing Reality/Context* Jonathan A. Morell, Ph.D. 4.669 Evaluation and Planning Director of Evaluation, Syntek Technologies Editor, Evaluation and Program Planning American Evaluation Association October 29th – November3rd Cleveland Ohio * This presentation is drawn from a report commissioned by Catholic Relief Services: Revealing Implicit Assumptions: Why, Where, and How? Thanks to CRS staff in Ghana and Malawi for their time and cooperation in the preparation of this report. For a copy of the report contact Guy Sharrock [email protected] URL for this slide presentation: http://jamorell.com/documents/Assumptions_presentation_AEA_2018_final.pdf A tacit assumption or implicit assumption is an assumption that includes the underlying agreements or statements made in the development of a logical argument, course of action, decision, or judgment that are not explicitly voiced nor necessarily understood by the decision maker or judge. Often, these assumptions are made based on personal life experiences, and are not consciously apparent in the decision-making environment. These assumptions can be the source of apparent paradoxes, misunderstandings and resistance to change in human organizational behavior. https://en.wikipedia.org/wiki/Tacit_assumption Where do assumptions live? Single Portfolios of Whole programs similar programs organization Budgets People’s heads Funding proposals Theories of change Organizational culture Requests for proposals Mission/goal statements Makeup of human capital History of previous activities Schedules (of varying timeframes) How can knowledge about assumptions be used? Process ▪ Time driven ▪ Event driven ▪ Ongoing, incremental influence ▪ Analysis of trends and frequency over time ▪ Immediate application by person/group with authority Promising places to suspect presence of implicit assumptions ▪ System dynamics ▪ Perverse consequences ▪ Interface across partners ▪ Range of near-term impact How can questions be asked that will reveal implicit assumptions? 5 Whys Exploit skeptical opinion Ruminate on success and failure Focus on outcome, ignore the program Query negative & unexpected outcomes Ishikawa (fishbone) root/contributing factors Fertilizer use Agricultural practices Field maintenance Irrigation Crop selection Quality of life Transitions among multiple outcomes derived from agricultural Assumption set #1 Current price knowledge producton ? Accounting expertise Assumption set #2 Assumption set #3 Business practice Education and Improved Employment job training quality of life Assumptions in Theories of Change Technical assistance Increased yield Agriculture Technical assistance Quality of life Marketing Technical assistance Increased AND There is no consequential difference between “and” and Agriculture yield “or” relationships Technical assistance AND Quality of Marketing life Technical assistance Increased OR Agriculture yield Technical assistance OR Quality of Marketing life Line thickness = certainty in relationship Technical assistance Increased yield We are equally certain of all relationships in the TOC Agriculture Technical assistance Quality of life Marketing Technical assistance Increased yield All causal connections are of equal strength Agriculture Technical assistance Quality of life Marketing Assumptions in Theories of Change Outcome Outcome Program Outcome Outcome Outcome Outcome Outcome It is possible to specify relationships unambiguously Outcome Outcome Program Outcome Outcome Outcome Outcome Outcome Provide Participants Income/ Global quality Recruit technical apply new resources of life participants assistance expertise increase improves Time and rates do not matter Provide Participants Income/ Global quality Recruit technical apply new resources of life participants assistance expertise increase improves 20 per 10 per 1 year lag, application 2 year lag, month month to increase income to QOL Assumptions in Theories of Change Soon Later Much later Outcome Outcome It is possible to specify an outcome chain beyond Program Outcome Outcome Outcome near-term outcomes Outcome Outcome Unknowable, Program Outcome unpredictable things Outcome happen Outcome Outcome Outcome Outcome If a model is correct locally it is correct globally Program Outcome Outcome Outcome Assumptions in Theories of Change Guide for Building Multiple Models Reason to Construct Model Explanation of program Evaluation of program Advocacy Stakeholder operation and impact operation and impact for program Funder #1 Funder #2 One version of a TOC Funder n will suffice Implementer #1 Implementer #2 Implementer n Service recipient group #1 Service recipient group #2 Service recipient group n Others… Country in which NGO resides Outcome Program 1 Outcome Outcome Outcome Outcomes stem only from program action Outcome Program 2 Outcome Outcome Outcome Outcome Program n Outcome Outcome Outcome.
Load more

Recommended publications
  • The Nature of Student Thinking and Its Implications for the Use of Learning Progressions to Inform Classroom Instruction
    The Nature of Student Thinking and Its Implications for the Use of Learning Progressions to Inform Classroom Instruction Alicia Alonzo, Michigan State University, [email protected] Andrew Elby, University of Maryland, [email protected] Abstract: Underlying much of the work on learning progressions (LPs) is a strong though often tacit assumption that student thinking is theory-like and context-independent. In this work-in-progress, we use both theoretical perspectives on the nature of novices’ knowledge and empirical evidence of the context-dependent variability of students’ reasoning in physics to question this assumption and to argue that characterizing students in terms of LP “levels” inadequately captures their understanding of force and motion. We then analyze one teacher’s use of LP-based data to reason about student thinking and instructional responses. While the teacher reasoned fluidly using LP levels, he more frequently used finer-grained knowledge elements and contextual factors to interpret student thinking; and these finer-grained interpretations led to more actionable instructional implications. Thus, while recognizing LPs as models (imperfect representations) of student thinking, we argue that their assumption of “levels” of student understanding may limit their utility for classroom decision-making. Introduction Learning progressions (LPs)—“descriptions of the successively more sophisticated ways of thinking about a topic” (National Research Council [NRC], 2007, p. 219)—have been touted as having the potential to influence educational policies and practices including standards, curriculum, instruction, and assessment (e.g., Black, Wilson, & Yao, 2011; National Assessment Governing Board, 2008; NRC, 2007, 2012). Although the NRC (2007, 2012) has tended to emphasize large-scale applications and LPs that span many years of instruction, LPs have also been promoted as tools to support teachers’ instructional decision-making (Alonzo, 2011; Black et al., 2011; Furtak, 2012).
    [Show full text]
  • Hindu Fundamentalism and Christian Response in India
    HINDU FUNDAMENTALISM AND CHRISTIAN RESPONSE IN INDIA Rev. Shadakshari T.K. (Bangalore, India and Pastoring Divyajyothi Church of the Nazarene) Introduction One of the purposes of religion, humanly speaking, is to enable people to live a responsible life. One desire is that religious people may not disturb the harmonious life; rather, they may contribute towards it. Today, religions have become a source of conflict and violence in many Asian societies. This is very evident in India where the inter-relationship among religions is breaking up. The contemporary problem in India is the question of nationalism and the issue of marginalized identities. Christians are caught between two: participation in the nationalism in the one hand and commitment to the cause of the marginalized on the other. There is an awakening of nationalism, which bears strong religious stamp, which is strongly promoted by the Hindutva ideology. At the same time there is a strong awakening of the Tribals and Dalits. In this context, the question comes to our mind: how do Christians in India serve both nationalism and marginal groups when both of them are opposing each other? I am not promising the absolute answer for the question raised. However, this article provides some clues by analyzing the historical development of religious fundamentalism and suggesting an appropriate response for Christians. Though there are several religious fundamental groups in the history of India (including Hinduism, Islam, Sikhism, and others), this article limits its study to the religious fundamentalism of Hinduism. The importance of Hindu fundamentalism lies in its very contemporary and nationalistic scope, compared to other, more regional expressions.
    [Show full text]
  • The Misconceived Assumption About Constitutional Assumptions
    Georgetown University Law Center Scholarship @ GEORGETOWN LAW 2009 The Misconceived Assumption About Constitutional Assumptions Randy E. Barnett Georgetown University Law Center, [email protected] This paper can be downloaded free of charge from: https://scholarship.law.georgetown.edu/facpub/852 http://ssrn.com/abstract=1285485 103 Nw. U. L. Rev. 615-661 (2009) This open-access article is brought to you by the Georgetown Law Library. Posted with permission of the author. Follow this and additional works at: https://scholarship.law.georgetown.edu/facpub Part of the Constitutional Law Commons THE MISCONCEIVED ASSUMPTION ABOUT CONSTITUTIONAL ASSUMPTIONS Randy E. Barnett* INTRODUCTION Whether or not they are originalists, most constitutional scholars and observers assume that the basic assumptions held at the time the Constitution was enacted or amended are relevant to ascertaining its original meaning. Some originalists use constitutional assumptions to constrain judges in their interpretations of the more abstract passages of the text. Others reject originalism precisely because they object to the outmoded assumptions that prevailed at the time of enactment, and they assume that these assumptions must shape the original meaning of the text. For example, some claim that because of widespread concern about the continued vitality of the militia, the original meaning of the Second Amendment’s right to keep and bear arms is limited to those who are in service to the militia. Others think that because many in the Thirty-Ninth Congress and elsewhere assumed the continued existence of segregated government schools or the inferiority of women, the original meaning of the Fourteenth Amendment must be consistent with segregated schools and the common law rules of coverture.
    [Show full text]
  • Personal Knowledge: Towards a Post-Critical Philosophy
    PERSONAL KNOWLEDGE PERSONAL KNOWLEDGE Towards a Post-Critical Philosophy by MICHAEL POLANYI London First published 1958 corrected edition 1962 by Routledge & Kegan Paul Ltd © 1958, 1962 Michael Polanyi Routledge is an imprint of the Taylor & Francis Group This edition published in the Taylor & Francis e-Library, 2005. “To purchase your own copy of this or any of Taylor & Francis or Routledge's collection of thousands of eBooks please go to www.eBookstore.tandf.co.uk.” All rights reserved. No part of this book may be reprinted or reproduced or utilized in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library. ISBN 0-203-44215-6 Master e-book ISBN ISBN 0-203-75039-X (Adobe e-Reader Format) ISBN 0-415-15149-X (Print Edition) To SIR THOMAS AND LADY TAYLOR PREFACE THIS is primarily an enquiry into the nature and justification of scientific knowledge. But my reconsideration of scientific knowledge leads on to a wide range of questions outside science. I start by rejecting the ideal of scientific detachment. In the exact sciences, this false ideal is perhaps harmless, for it is in fact disregarded there by scientists. But we shall see that it exercises a destructive influence in biology, psychology and sociology, and falsifies our whole outlook far beyond the domain of science. I want to establish an alternative ideal of knowledge, quite generally.
    [Show full text]
  • Sabotaging Progress: the Cultural Economy of Resentment in Late Neoliberalism William Davies
    Sabotaging Progress: The Cultural Economy of Resentment in Late Neoliberalism William Davies The contemporary populist moment appears perplexing, even irrational, from the perspective of most orthodox traditions of social science, with their tacit assumption of utilitarian psychology and government. Whether allied to the Left or the Right, rational-choice paradigms seek to interpret individual and collective activity in terms of the pursuit and maximization of tangible interests. It’s not clear how Brexit, nativism, and “Trumpism” advance anyone’s interests, other than the financial interests and media careers of those who are personally invested in the new political formations. Centrist technocrats, such as those left over from the “Third Way” era of the late 1990s, make similar allegations about the populists of the Left, such as Corbyn, Mélenchon, and Podemos, pointing to Syriza’s capitulation to the Troika as evidence of the naiveté of these new political forces. There are two riddles in particular that need to be solved, but which defeat a rational-choice perspective. First, how to understand the apparent absence of any macroeconomic logic of the new political forces, which seem not only to have broken free of any model of accumulation, growth, or progress, but often to be sabotaging those very notions. If events such as Brexit do the harm that many economists expect, then surely people must be laboring under some kind of false consciousness or have been victims of propaganda (with or without the intervention of the Kremlin). Is it possible to consciously and reasonably seek economic regress? Secondly, how to understand the power and popularity of the super-rich within nativist movements, combined with extreme antipathy towards “liberal elites” such as the “mainstream media,” “mainstream politicians,” and universities.
    [Show full text]
  • Assumptions to Unravel the Tangle of Complexity, Values, and Cultural Responsiveness
    Working with Assumptions to Unravel the Tangle of Complexity, Values, and Cultural Responsiveness Session 1: How can assumptions be recognized? Apollo Nkwake Katrina Bledsoe Jonny Morell Presenters Jonathan Morell, Ph.D. Katrina L. Bledsoe, Ph.D. Principal, 4.669 Evaluation and Apollo M. Nkwake, CE. Ph.D. Research Scientist, Planning/Editor, Evaluation International Technical Education Development and Program Planning Advisor, Monitoring and Center/Principal Phone: (734) 646-8622 Evaluation, Education Consultant Katrina Email: [email protected] Development Center Bledsoe Consulting Website www.jamorell.com Email: [email protected] Email: [email protected] Blog:https://evaluationuncertai nty.com/ YouTubeTinyurl.com/morell- YouTube Session 1: How can assumptions be • Facilitators walk through scenarios that illustrate applications recognized? 1/14/19 and use tools and processes that are designed to help reveal and monitor the use of assumptions. We will help participants learn to gauge the tools' and processes' appropriateness, their strengths and limitations, and when to use which tools. • This session will cover assumptions about how programs work Session 2: The role of assumptions in the and relationships among outcomes; assumptions about program design and conduct of evaluation. 1/28/19 logic; assumptions about evaluation and assumptions about program purpose. • Discuss the intersection of cultural responsivity and values Session 3: Assumptions, Values and Cultural assumptions in program design and evaluations. Reflect on Responsiveness. 2/11/19 case studies and scenarios. Session 1 Outline • Propositions to frame the discussion • The tangle of complexity, assumptions, values and responsiveness • Tools and processes for examining assumptions • Diagnostic assumptions focused tools/processes • Prescriptive Assumptions focused tools/processes • Causal Assumptions focused tools/processes • Findings from an Assumptions survey • Principles • Next steps One program’s overelaboration is Carol Weiss – evaluator another program’s theory.
    [Show full text]
  • Probing Neural Language Models for Human Tacit Assumptions
    Probing Neural Language Models for Human Tacit Assumptions Nathaniel Weir, Adam Poliak, and Benjamin Van Durme Department of Computer Science, Johns Hopkins University fnweir,azpoliak,[email protected] Abstract Prompt Model Predictions A has fur. dog, cat, fox, ... Humans carry stereotypic tacit assumptions (STAs) (Prince, 1978), or propositional beliefs about generic concepts. Such A has fur, is big, and has claws. cat, bear, lion, ... associations are crucial for understanding natural language. A has fur, is big, has claws, has bear, wolf, cat, ... We construct a diagnostic set of word prediction prompts to teeth, is an animal, eats, is brown, evaluate whether recent neural contextualized language mod- and lives in woods. els trained on large text corpora capture STAs. Our prompts are based on human responses in a psychological study of con- Figure 1: The concept bear as a target emerging as the highest ceptual associations. We find models to be profoundly effec- ranked predictions of the neural LM ROBERTA-L (Liu et al., 2019) tive at retrieving concepts given associated properties. Our re- when prompted with conjunctions of the concept’s human-produced sults demonstrate empirical evidence that stereotypic concep- properties. tual representations are captured in neural models derived from semi-supervised linguistic exposure. Keywords: language models; deep neural networks; concept representations; norms; semantics defining properties across multiple metrics; this perfor- mance discrepancy is consistent with many other lan- Introduction guage understanding tasks (Wang et al., 2018). We also find that models associate concepts with perceptual cat- Recognizing generally accepted properties about con- egories of properties (e.g. visual) worse than with non- cepts is key to understanding natural language (Prince, perceptual ones (e.g.
    [Show full text]
  • Three Principles Underlying the Formulation 6F Questionnaires By
    Survey Research Center February, I9h9 University of Hichigan THE ART OF ASKING WHY Three Principles Underlying The Formulation 6f Questionnaires By Paul F..Lazarsfeld Reprinted From National Marketing rteview Summer 193$ - Vol, I - He. 1. I. Ascertaining What A Question Means; The Principle of Specification,. Asking for reasons and giving answers are commonplace habits of everyday life. V-'e have all had the experience of acting under certain impulses and certain influences so many times that we are sure that our fellow men have had the same experiences and reasons for their own actions. And we are seldom disappointed if we inquire. Our respondent not only has had reasons for his actions; he usually knows, also, in which reason we might be especially interested, and it is upon this assumption that he bases his answer. If a friend explains why he had come to see me, he does not start to tell ne that he was once born, and that he moved to this city two years ago, although these, too, are reasons for his being here today. He is aware that most of these rea• sons are known as well to me as to him, and he picks out the reason which he hopes will contribute especially to a mutual understanding of the present situation. In market research, the question-and-answer business is not so simple, and the ease of furnishing answers in everyday life may in• volve dangerous pitfalls. In social intercourse, it is most likely that what is important 'for our respondent is important also for us who have made the inquiry.
    [Show full text]
  • Mistake and Disclosure in a Model of Two-Sided Informational Inputs
    Missouri Law Review Volume 73 Issue 3 Summer 2008 Article 2 Summer 2008 Mistake and Disclosure in a Model of Two-Sided Informational Inputs Michael J. Borden Follow this and additional works at: https://scholarship.law.missouri.edu/mlr Part of the Law Commons Recommended Citation Michael J. Borden, Mistake and Disclosure in a Model of Two-Sided Informational Inputs, 73 MO. L. REV. (2008) Available at: https://scholarship.law.missouri.edu/mlr/vol73/iss3/2 This Article is brought to you for free and open access by the Law Journals at University of Missouri School of Law Scholarship Repository. It has been accepted for inclusion in Missouri Law Review by an authorized editor of University of Missouri School of Law Scholarship Repository. For more information, please contact [email protected]. Borden: Borden: Mistake and Disclosure in a Model MISSOURI LAW REVIEW VOLUME 73 SUMMER 2008 NUMBER 3 Mistake and Disclosure in a Model of Two- Sided Informational Inputs Michael J Borden* I. INTRODUCTION ............................................................................................ 667 II. CURRENT LAW AND COMMENTARY ............................................................ 671 A . The D octrine........................................................................................ 671 B. The Comm entary ................................................................................. 675 1. K ronm an's A pproach .................................................................... 676 2. Levm ore's Contribution ...............................................................
    [Show full text]
  • Jung, Christianity, and Buddhism
    Jung, Christianity, and Buddhism James W. Heisig An abbreviated version of this paper was presented at a conference on ‘“Buddhism and Depth Psychology: Japan and America,” held in Kyoto in May 1999. The fuller paper was prepared for presentation in Japanese at the 1999 annual meeting of the Japan Society for Buddhist-Christian Studies. Throughout the past generation of Christianity’s encounter with Buddhism, the role played by Jungian psychology has been ancillary at best. True, Jung’s ideas are cited with a certain regularity, but for some reason the systematic body of thought he left behind has not attracted Christians or Buddhists as a common forum for mutual criticism and enrichment. In this essay I would like to draw attention to what I see as an unnecessary closure in Jung’s idea of the psyche and to suggest how its opening could nudge Christianity and Buddhism into closer contact with each other and with the shifts that have taken place in the general spirituality of our age. The body of writings Jung left behind is at once forbidding and fascinating. The sheer volume of his output, which continues to grow as notes from his seminars and other unpublished material are added, makes it more and more unlikely that any but a small coterie of devoted specialists will take the time to become familiar with the whole. Even so, the academic background the specialist needs to read with understanding all but ensures that large blocks of his work will simply be skimmed over uncritically. For the same reason, general readers, in which group one must include the greater part of Jungian analysts, commentators, and critics, tend to dip into his writings randomly according to their needs and to rely on secondary sources for general outlines of his thought.
    [Show full text]
  • Incredible Utility 1
    Incredible utility 1 Incredible utility: The lost causes and causal debris of psychological science John E. Richters Rockville, Maryland Direct correspondence to: John E. Richters 13200 Valley Drive Rockville, MD 20850 [email protected] Incredible utility 2 Abstract .......................................................................................................................................................................................... 3 The incredible utility of perpetual motion ................................................................................................................................ 4 Incredible utility thesis ................................................................................................................................................................ 4 The plumbing and wiring of individual differences methodology ........................................................................................ 6 Table 1. .............................................................................................................................................................................. 7 Individual-level theoretical models ................................................................................................................................. 11 Aggregate-level empirical models .................................................................................................................................. 12 The inferential passage from aggregates to individuals ..............................................................................................
    [Show full text]
  • Insiders and Outsiders in the Sociology of Science, Or How Can We Foster
    INSIDERS& OUTSIDERSIN THE SOCIOLOGYO F SCIENCEO; R, HOW CAN WE FOSTER AGNOSTICISM? Bruno Latour Volumesh ave been written evaluatingt he prospectsf or developmento f social scientificm ethodologya s rigorousa s that of the natural sciences. They ask, should social scientists attempt to experiencet he "lived world" of the peoplet hey are studying?O r, in the pursuito f objectivity, shoulds ocial scientistsa ttempt to distancet hemselvesa s much as possi- ble from their subjects?C an sociologistsc onfidentlye xpectt he eventual appearanceo f a Newton of the social world? Or should they ignore the canon of the hard sciencesa nd formulate new forms of measurement, deficientb y the standardso f natural scientifici nquiry but suited to the distinctivec haracteristicso f the socialw orld? However interestingt hese Knowledgea nd Society:S tudiesi n the Sociologyo f Culture Past and Present,v olume 3, pages1 9-216. Copyright @ f98f by JAI PressI nc. All rights of reproduction in any forrn reserved. ISBN: 0-89232-161-X 2OO BRUNOL ATOL. questionsm ay be from the standpointo f most social scientists,t hey arc peculiarly inappropriatef or those sociologistsw ho are concernedt o er- plain the conduct of sciencei tself. All methodologicala rgumentsi n the social sciencesa re basedo r onc' tacit assumption:t hat scientifica ctivity is distinct from all other forms oi activity.t All methodologicadl isputesc an be viewed as argumentsa bour the place that social scientific method occupies on the objective- subjectivec ontinuum.2M ost of the debatesc oncern the direction the continuums ocialr esearchs houldt ake. But whethert he conclusioni s that more objectivity or more subjectivityi s neededi n socials cientifici nquiry.
    [Show full text]