CHILDREN AND FAMILIES The RAND Corporation is a nonprofit institution that helps improve policy and EDUCATION AND THE ARTS decisionmaking through research and analysis. ENERGY AND ENVIRONMENT HEALTH AND HEALTH CARE This electronic document was made available from www.rand.org as a public service INFRASTRUCTURE AND of the RAND Corporation. TRANSPORTATION INTERNATIONAL AFFAIRS LAW AND BUSINESS Skip all front matter: Jump to Page 16 NATIONAL SECURITY POPULATION AND AGING PUBLIC SAFETY Support RAND SCIENCE AND TECHNOLOGY Purchase this document TERRORISM AND Browse Reports & Bookstore HOMELAND SECURITY Make a charitable contribution For More Information Visit RAND at www.rand.org Explore the RAND National Defense Research Institute View document details Limited Electronic Distribution Rights This document and trademark(s) contained herein are protected by law as indicated in a notice appearing later in this work. This electronic representation of RAND intellectual property is provided for non- commercial use only. Unauthorized posting of RAND electronic documents to a non-RAND website is prohibited. RAND electronic documents are protected under copyright law. Permission is required from RAND to reproduce, or reuse in another form, any of our research documents for commercial use. For information on reprint and linking permissions, please see RAND Permissions. This report is part of the RAND Corporation tool series. RAND tools may include models, databases, calculators, computer code, GIS mapping tools, practitioner guide- lines, web applications, and various toolkits. All RAND tools undergo rigorous peer review to ensure both high data standards and appropriate methodology in keeping with RAND’s commitment to quality and objectivity. RAND Suicide Prevention Program Evaluation TOOLKIT Joie D. Acosta, Rajeev Ramchand, Amariah Becker, Alexandria Felton, C O R P O R A T I O N Aaron Kofner NATIONAL DEFENSE RESEARCH INSTITUTE RAND Suicide Prevention Program Evaluation TOOLKIT Joie D. Acosta, Rajeev Ramchand, Amariah Becker, Alexandria Felton, Aaron Kofner Prepared for the Office of the Secretary of Defense Approved for public release; distribution unlimited The research described in this report was prepared for the Office of the Secretary of Defense (OSD). The research was conducted within the RAND National Defense Research Institute, a federally funded research and development center sponsored by OSD, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense Intelligence Community under Contract W74V8H-06-C-0002. Library of Congress Cataloging-in-Publication Data is available for this publication. ISBN: 978-0-8330-8072-1 The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. R® is a registered trademark. Cover images: iStockphoto/Thinkstock © Copyright 2013 RAND Corporation Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Copies may not be duplicated for commercial purposes. Unauthorized posting of RAND documents to a non-RAND website is prohibited. RAND documents are protected under copyright law. For information on reprint and linking permissions, please visit the RAND permissions page (http://www.rand.org/publications/ permissions.html). Published 2013 by the RAND Corporation 1776 Main Street, P.O. Box 2138, Santa Monica, CA 90407-2138 1200 South Hayes Street, Arlington, VA 22202-5050 4570 Fifth Avenue, Suite 600, Pittsburgh, PA 15213-2665 RAND URL: http://www.rand.org To order RAND documents or to obtain additional information, contact Distribution Services: Telephone: (310) 451-7002; Fax: (310) 451-6915; Email: [email protected] RAND Suicide Prevention Program Evaluation Toolkit Preface Evaluations are critical for assessing the impact of U.S. Department of Defense (DoD) invest- ments in suicide prevention and can be used as the basis for decisions about whether to sustain or scale up existing efforts. The Defense Centers of Excellence for Psychological Health and Traumatic Brain Injury asked the RAND Corporation to draw from the scientific literature and create a toolkit to guide future evaluations of DoD-sponsored suicide prevention programs (SPPs). The overall goal of the toolkit is to help those responsible for SPPs determine whether their programs produce beneficial effects and, ultimately, to guide the responsible allocation of scarce resources. The toolkit is accompanied by a companion document (available at http://www.rand. org/pubs/research_reports/RR283.html) that provides additional background for those who are interested in learning about the toolkit’s development. To develop the toolkit, RAND researchers examined the peer-reviewed literature and other evaluation toolkits and elicited feedback from program staff responsible for implementing DoD-sponsored SPPs. The contents of this toolkit will be of particular interest to DoD SPP managers and sui- cide prevention coordinators and should also be useful to policymakers in other sectors who sponsor or manage SPPs more generally. This research was sponsored by the Office of the Assistant Secretary of Defense for Health Affairs and the Defense Centers of Excellence for Psychological Health and Traumatic Brain Injury and conducted within the Forces and Resources Policy Center of the National Defense Research Institute, a federally funded research and development center sponsored by the Office of the Secretary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense Intelligence Community. For more information on the Forces and Resources Policy Center, see http://www.rand. org/nsrd/ndri/centers/frp.html or contact the director (contact information is on the web page). iii RAND Suicide Prevention Program Evaluation Toolkit Contents Preface ........................................................................................................... iii Tools ..............................................................................................................vii Figures ........................................................................................................... ix CHAptER ONE Introduction and Overview ................................................................................... 1 Intended Audience ............................................................................................... 1 Toolkit Goals and Specific Aims ............................................................................... 1 Evaluation Challenges Addressed .............................................................................. 2 How the Toolkit Was Developed .............................................................................. 2 User’s Guide ...................................................................................................... 4 Start Using the Toolkit .......................................................................................... 5 Summary .......................................................................................................... 7 CHAptER TWO Identify Your Program’s Core Components and Build a Logic Model ............................... 9 Specify Core Components ...................................................................................... 9 Use Core Components to Build a Logic Model ............................................................ 38 Assess the Quality of Your Program’s Logic Model .........................................................41 Summary ........................................................................................................ 46 CHAptER THREE Design an Evaluation for Your Program ..................................................................47 Identify Your Program Type ...................................................................................47 Learn How Similar Programs Have Been Evaluated ...................................................... 50 Learn the Types of Evaluation Designs .......................................................................51 Identify Issues Pertinent to Your Evaluation Design ........................................................52 Select an Evaluation Design ...................................................................................59 Assess the Quality of the Design ..............................................................................61 Summary .........................................................................................................61 v RAND Suicide Prevention Program Evaluation Toolkit CHAptER FOUR Select Evaluation Measures for Your Program ...........................................................63 Select Process Evaluation Measures ...........................................................................63 Select Outcome Evaluation Measures ....................................................................... 68 Assess the Quality of Selected Measures .....................................................................73 Summary .........................................................................................................74 CHAptER FIVE Analyze Your Program’s Evaluation Data .................................................................75 How to Use This Chapter ......................................................................................75