The author(s) shown below used Federal funds provided by the U.S. Department of Justice and prepared the following final report: Document Title: Evaluation of Internet Child Safety Materials Used by ICAC Task Forces in School and Community Settings, Final Report Author(s): Lisa M. Jones, Ph.D., Kimberly J. Mitchell, Ph.D., Wendy A. Walsh, Ph.D. Document No.: 242016 Date Received: April 2013 Award Number: 2009-SN-B9-0004 This report has not been published by the U.S. Department of Justice. To provide better customer service, NCJRS has made this Federally- funded grant report available electronically. Opinions or points of view expressed are those of the author(s) and do not necessarily reflect the official position or policies of the U.S. Department of Justice. Evaluation of Internet Child Safety Materials Used by ICAC Task Forces in School and Community Settings (PROJECT NUMBER 2009-SN-B9-0004) NIJ EVALUATION FINAL TECHNICAL REPORT Lisa M. Jones 10 West Edge Dr., Suite 106 Crimes Against Children Research Center (CCRC) University of New Hampshire Durham, NH 03824 603-862-2515 [email protected] Kimberly J. Mitchell Wendy A. Walsh 603-862-24533 603-862-1026 [email protected] [email protected] Submission Date: December 31, 2012 This project was supported by Grant No. 2009-SN-B9-0004 awarded by the National Institute of Justice, Office of Justice Programs, U.S. Department of Justice. Opinions, findings, and conclusions or recommendations expressed in this publication/program/exhibition are those of the authors and do not necessarily reflect those of the Department of Justice. This document is a research report submitted to the U.S. Department of Justice. This report has not been published by the Department. Opinions or points of view expressed are those of the author(s) and do not necessarily reflect the official position or policies of the U.S. Department of Justice. NIJ EVALUATION FINAL TECHNICAL REPORT Evaluation of Internet Child Safety Materials Used by ICAC Task Forces in School and Community Settings ABSTRACT Research Goals and Objectives: This project involves content and process evaluations of current internet safety education (ISE) program materials and their use by law enforcement presenters and schools. Despite a proliferation of internet safety programs over the last decade, there is little information that can guide law enforcement, policy makers or the public in determining which materials or delivery methods are most likely to increase children’s online safety. The design of the proposed content and process evaluation is based on the perspective that, despite widespread dissemination, internet safety education is still in a formative stage. It is not clear that ISE messages have been formulated around a careful analysis of the risk and the ways that youth experience problems online, or that they have applied research-based prevention strategies. While outcome evaluation will be critical to determining the effectiveness of internet safety programs in the future, it is important to identify problems in ISE delivery and create guidelines for developing more promising programs. Research Design and Methodology: The study was divided into four subprojects. First, a systematic review or “meta-synthesis” was conducted to identify effective elements of prevention identified by the research across different youth problem areas such as drug abuse, sex education, smoking prevention, suicide, youth violence, and school failure. The process resulted in the development of a KEEP (Known Elements of Effective Prevention) Checklist. Second, a content analysis was conducted on four of the most well-developed and long-standing youth internet safety curricula: i-SAFE, iKeepSafe, Netsmartz, and Web Wise Kids. Third, we conducted a process evaluation to better understand how internet safety education programs are being implemented. The process evaluation was conducted via national surveys with three different groups of respondents: Internet Crimes Against Children (ICAC) Task Force commanders (N=43), ICAC Task Force presenters (N=91), and a sample of school professionals (N=139). Finally, we developed an internet safety education outcome survey focused on online harassment and digital citizenship. The intention for creating and piloting this survey was to provide the field with a research-based tool that can be used in future evaluation and program monitoring efforts. This tool, along with other research and evaluation information on internet safety will be placed in an Internet Safety Education Resource Center on the ICAC Task Force website. Research Results and Conclusions: The internet safety education (ISE) content and process evaluation results indicated that the educational approach and messages of current ISE fail to incorporate critical elements of effective prevention education, including: 1) research-based messages; 2) skill-based learning objectives; 3) opportunities for youth to practice new skills; and 4) sufficient time for learning. Our analyses indicate that the ISE field has been slow to include research-based information on internet predators and online harassment and there is no research to support the assumption that many of the popular educational slogans messages around privacy and digital reputation concerns (e.g., “Think Before You Click”) will lead to improved youth online behavior. The failure to define research-supported program logic means that most ISE is a highly speculative and experimental undertaking, whose success cannot be assumed. Recommendations are made for re-conceptualizing ISE and developing a more effective approach to helping protect youth. This document is a research report submitted to the U.S. Department of Justice. This report has not been published by the Department. Opinions or points of view expressed are those of the author(s) and do not necessarily reflect the official position or policies of the U.S. Department of Justice. NIJ EVALUATION FINAL TECHNICAL REPORT Evaluation of Internet Child Safety Materials Used by ICAC Task Forces in School and Community Settings TABLE OF CONTENTS EXECUTIVE SUMMARY……………………………………………………………………………………….. i-xiv DRAFT FINAL REPORT Introduction ……………………………………………………………………………………………………… 1-5 Project Methodology and Results Subproject A: Prevention Research Meta-Synthesis to Identify Critical Components of Effective Prevention ……………………………………………………….……… 5-11 Subproject B: A Content Evaluation of Established Internet Safety Programs ………………………………………………………………………………………………………………………… 11-20 Subproject C: Evaluating the Delivery of Internet Safety Education: A Survey of Law Enforcement and School Professionals ………………………………………………………. 20-27 Subproject D: Internet Safety Education Resource Center and Evaluation Toolkit…………………………………………………………………………………………………………….. 27-33 Overall Project Conclusions ……..…………………………………………………………………………. 33-35 Policy And Practice Recommendations ………………………………………………………………... 35-44 REFERENCES …………………………………………………………………………………………………. 45-51 PROJECT DATA TABLES ………………………………………………………………………………… 52-78 APPENDICES APPENDIX A: Internet Safety Education Content Evaluation Long Coding Form…….. 80-88 APPENDIX B: UNH Internet Safety Education Survey……………………………………………. 89-96 This document is a research report submitted to the U.S. Department of Justice. This report has not been published by the Department. Opinions or points of view expressed are those of the author(s) and do not necessarily reflect the official position or policies of the U.S. Department of Justice. NIJ EVALUATION FINAL TECHNICAL REPORT Evaluation of Internet Child Safety Materials Used by ICAC Task Forces in School and Community Settings EXECUTIVE SUMMARY The rapid development of new technology over the past two decades is likely to be considered a hallmark of our times, but publicity about online predators and cyberbullying has raised alarms about risks for youth. Law enforcement has been active in delivering materials to communities and dissemination efforts have been very successful. A recent national survey of youth internet users found that in 2010, 47% of youth reported attending an internet safety program hosted by law enforcement in the previous year (Mitchell et al., 2012a). This is up from 21% of youth who reported ISE exposure from law enforcement in 2005. Unfortunately, the broad dissemination of ISE prevention programs and materials happened so quickly that much of it has been put into place before substantial research was available on the nature of the problems youth were experiencing online and their causes. To prepare for outcome evaluation, it is important to first understand how the field is currently delivering ISE. The current study was designed to: 1) create checklists to help policy- makers, program developers and consumers better identify programs that use proven methods and incorporate research; 2) evaluate how well a sample of ISE programs meet these criteria, looking at lessons from four long-standing and well-established ISE programs: IKeepSafe, I-SAFE, Netsmartz, and Web Wise Kids; 3) understand how ISE program materials are being used by educators and law enforcement presenters; and 4) provide pilot data on an ISE outcome measure that can be used in future evaluation. METHODOLOGY To achieve these project aims, our study was divided into four subprojects (see Figure 1). i This document is a research report submitted to the U.S. Department of Justice. This report has not been published by the Department. Opinions or points of view expressed are those of the