Evaluation Guide for Pedestrian Safety ACKNOWLEDGMENTS Authors Safe States Alliance • Jamila Porter, DrPH, MPH, Director of Programs and Evaluation • Ina Robinson, MPH, Evaluation and Technical Assistance Coordinator Research and Evaluation Group • Shenee Bryan, MPH, MPA, Director of Programs and Evaluation • LaTasha Barnwell, MPH, Research Associate Transtria • Laura Brennan, PhD, MPH, Chief Executive Ofcer • Brandye Mazdra, MPH, Director of Projects and Operations Advisors to the Authors • Ross Brownson, PhD, Bernard Becker Professor of Public Health, Washington University in St. Louis • Sarah Strunk, MHA, Strategic Advisor, Active Living By Design • Ian Thomas, PhD, State and Local Program Director, America Walks • Denise Yeager, MPH, Healthcare Program Consultant and Operation Analyst, Georgia Division of Public Health, Injury Prevention Section The Evaluation Guide for Pedestrian Safety was developed by the Safe States Alliance, with support from the National Highway Trafc Safety Administration (NHTSA). Content and structure for the guide were provided by Research and Evaluation Group and Transtria. Web development, design, and hosting of the guide were provided by Banyan Communications. Evaluation Guide for Pedestrian Safety Accreditation | 2 TABLE OF CONTENTS About the Evaluation Guide ..............................................4 Section 4: Methodology and Data Collection ....................... 59 Create Your Evaluation Plan ........................................... 10 Step 1: Identify data sources or design data Resource Center ........................................................... 11 collection methods................................................ 59 Step 2: Create a matrix to connect your evaluation Section 1: Evaluation Purpose and Partners ....................... 12 questions, indicators, and data sources ....................... 64 Step 1: Convene partners to plan the evaluation ..................... 12 Step 2: Identify your population of interest and your Section 5: Data Analysis ................................................. 66 vision for change .................................................. 17 Step 1: Convene data analysis partners ................................ 66 Step 3: Consider the intervention stage and corresponding Step 2: Identify variables and themes .................................. 67 types of evaluation ................................................ 18 Step 3: Organize data and ensure data quality ........................ 70 Step 4: Determine evaluation audiences and the relevance Step 4: Agree on data analysis methods ............................... 72 of the evaluation to these audiences .......................... 21 Step 5: Interpret results ................................................... 75 Step 5: Create a clear, concise purpose statement ................... 23 Step 6: Create a data analysis plan ...................................... 78 Step 6: Design your collaborative, including a governance Section 6: Reporting and Dissemination ............................ 79 structure and process, partner responsibilities, Step 1: Identify audiences and develop a dissemination plan ......... 79 and budget and time constraints ............................... 25 Step 2: Select key messages for specifc purposes ................... 80 Section 2: Describing the Intervention .............................. 31 Step 3: Select communication methods and create evaluation Step 1: Describe your Pedestrian Safety Intervention ................ 31 products ............................................................ 80 Step 2: Create a Logic Model ............................................. 36 Step 4: Monitor Dissemination Eforts .................................. 86 Section 3: Evaluation Design ........................................... 39 Glossary ..................................................................... 87 Step 1: Develop evaluation questions aligned with the Evaluation Templates Appendix ..................................... 96 evaluation purpose statement .................................. 39 Tools and Resources Appendix .......................................116 Step 2: Identify the population or subpopulations, intervention goals and objectives, and types of evaluation for each evaluation question ................... 42 Step 3: Specify the evaluation design to maximize causal inferenceand internal validity for each evaluation question ............................................................ 44 Step 4: Determine sampling strategies to maximize external validity in the selected population or subpopulations for each evaluation question ................ 53 Evaluation Guide for Pedestrian Safety Table of Contents | 3 ABOUT THE EVALUATION GUIDE The Evaluation Guide for Pedestrian Safety is a robust and comprehensive resource designed to help you and your partners efectively evaluate your pedestrian safety interventions. How You Can Use This Guide You can use this guide to: • Plan your evaluation of a pedestrian safety intervention using a helpful, step-by-step process. • Obtain useful tips to help you: • Collaborate successfully with partners. • Infuse principles associated with social, health, and economic equity into your evaluation. • Implement and manage your evaluation process efectively. • Capitalize on assets and overcome anticipated barriers during the evaluation process. • Access templates, tools, resources, and examples from the feld to inform and enhance your evaluation. Why Evaluation Is Essential There are a variety of reasons that it is essential to evaluate pedestrian safety interventions. Evaluation can: 1. Improve interventions by identifying what works to increase pedestrian safety. Evaluationfndingscan: • Refect strengths and weaknesses of your partnership’s structure and processes. • Strengthen the design and implementation of your pedestrian safety intervention. • Highlight the reach of your pedestrian safety intervention to vulnerable populations. • Demonstrate impacts of your pedestrian safety intervention on population health behaviors, outcomes, and quality of life. • Assess sustainability of your pedestrian safety intervention over time. 2. Build support for collaborative partnerships that can mobilize resources for pedestrian safety initiatives. Evaluation partnerships can: • Raise awareness and visibility of pedestrian safety interventions. • Leverage political and community support, funding, and resources for pedestrian safety interventions.1 Evaluation Guide for Pedestrian Safety About the Evaluation Guide | 4 3. Establish an evidence base for scale up of efective pedestrian safety policies and systems. Systematicreviewsofevaluationfndingscan: • Facilitate state, regional, and local decision-making about priority pedestrian safety interventions. • Provide state, regional, and local model practices or implementation guidelines. 4. Make the connections between pedestrian safety and population health explicit. 2,3 “Field-building”basedonevaluationfndings(andresearch)can: • Provide data support to show pedestrian safety interventions increase active travel. • Show how increases in active travel increase the proportion of the population meeting physical activity recommendations and decreases in auto use reduce sedentary behaviors. • Demonstrate that increases in physical activity and reductions in sedentary behaviors reduce morbidity and mortality (e.g., obesity, cardiovascular disease, cancer). • Draw connections between active travel/reduced auto use and cleaner air, community development, economic growth, sustainability, and other related outcomes. “Evidence-based public health utilizes the current best available evidence to make decisions in the public health service, and also to develop action plans, public health programs, and policies for addressing public health issues.” 4 FOOTNOTES 1. Redmon, T., D. Gelinne, L. Walton, J. Miller. Jan/ Feb 2012. Spotlight on pedestrian safety. Publication Number: FHWA-HRT-12-002 75(4). Retrieved from: https://www.fhwa.dot.gov/publications/publicroads/12janfeb/03.cfm 2. U.S. Centers for Disease Control and Prevention. 2010 CDC Recommendations for Improving Health through Transportation Policy. Retrieved from: https://www.cdc.gov/transportation/docs/fnal-cdc-transportation-recommendations-4-28-2010.pdf 3. U.S. Department of Transportation. Walking and Biking are Good for Public Health. Retrieved from: https://www.transportation.gov/safer-people-safer-streets 4. Brownson, R. C., E. A. Baker, A.D. Deshpande, K. N., Gillespie. Evidence-Based Public Health. 3rd Ed. New York: Oxford University Press; 2018. Retrieved from: https://global.oup.comacademic/product/evidence-based-public-health-9780190620936?cc=us&lang=en& Evaluation Guide for Pedestrian Safety About the Evaluation Guide | 5 What Interventions You Can Evaluate The Evaluation Guide for Pedestrian Safety is designed to evaluate pedestrian safety interventions, specifcally pedestrian safety action plans (PSAPs) and pedestrian safety education campaigns and promotions. • Pedestrian Safety Action Plans (PSAPs) create a framework for state and local ofcials and a variety of partners to address pedestrian safety problems in specifc geographic areas to identify, implement, and evaluate optimal solutions to improve pedestrian safety. • Pedestrian safety education campaigns and promotions are coordinated eforts designed to improve pedestrian safety for a defned population by targeting knowledge, attitudes, awareness, beliefs, behaviors, and/ or social norms