PDF, 300 Pages
Total Page:16
File Type:pdf, Size:1020Kb
Developing Web Analytics Best Practices for NIH Report Semphonic 16 Digital Drive Suite 220 Novato, CA 94949 www.semphonic.com December 15, 2011 The Needs Assessment for Developing Best Practices for Trans-NIH Web Analytics was funded by the NIH Evaluation Set-Aside Program, (Reference number: 10-4031 OD-OCPL), administered by the Office of Program Evaluation and Performance, DPCPSI, Office of the Director Contents Acknowledgments ......................................................................................................................................... ii Executive Summary .................................................................................................................................. ES.1 Commitment of Senior Management .............................................................................................. ES.2 Organizational Resources and Responsibilities ................................................................................ ES.2 Process for NIH-Wide and IC-Specific Web Analytics Governance .................................................. ES.3 Web Analytics Metrics and Deep-Dive Analysis ............................................................................... ES.4 Web Analytics Solutions, Methods, and Opportunities ................................................................... ES.4 Web Analytics Training, Data Analysis, and Interpretation of Data ................................................ ES.5 More Information ............................................................................................................................ ES.5 Roadmap .......................................................................................................................................... ES.6 Introduction .................................................................................................................................................. 1 Project Methodology .................................................................................................................................... 3 Findings and Recommendations ................................................................................................................... 1 1. Commitment of Senior Management ............................................................................................... 1 2. Organizational Resources and Responsibilities ................................................................................. 5 3. Process for NIH-Wide and IC-Specific Web Analytics Governance ................................................. 13 4. Web Analytics Metrics and Deep-Dive Analysis ............................................................................. 20 5. Web Analytics Solutions, Methods, and Opportunities .................................................................. 44 6. Web Analytics Training, Data Analysis ,and Interpretation of Data................................................ 46 Appendix A. Survey Analysis: Developing Web Analytics Best Practices for NIH ...................................... A.1 Appendix B. NIH Web Analytics Requirements Summary ......................................................................... B.1 Appendix C. Process for Collecting Accurate Analytics Data ..................................................................... C.1 Appendix D. Success Metrics Calculations .................................................................................................D.1 Appendix E. Dashboard Content and Sample ............................................................................................ E.1 Appendix F. Functionalism: A New Approach to Web Analytics ............................................................... F.1 Appendix G. How to Conduct A/B Testing ................................................................................................ G.1 Appendix H. Strategies for Social Media Measurement ........................................................................... H.1 Appendix I. Evaluation of Web Analytics and Measurement Tools ........................................................... I.1 Appendix J. Strategies for Google Analytics Implementation .................................................................... J.1 Appendix K. Piwik Evaluation: A Free, Open-Source Web Analytics Tool ................................................. K.1 Appendix L. Glossary for NIH Web Analytics Best Practices ...................................................................... L.1 Acknowledgments i Acknowledgments Semphonic was honored to be selected to partner with the On-Line Information Branch of the Office of Communications and Public Liaison, Office of the Director, on the Developing Web Analytics Best Practices for NIH Project. We understand that this is the first time a project of this breadth and scope has been undertaken within a Federal Government agency. I’d like to acknowledge the following people who have made this project possible and created a body of work that we hope will serve the NIH community for years to come: Ann Poritzky, Web Analytics Lead, On-Line Information Branch, whose vision, tenacity, and dedication to using Web analytics throughout NIH led to the creation and funding of this project. In addition, Ann’s leadership during the project, review of the project deliverables, and feedback about the new approaches we’re taking on digital analytics for a Federal Government agency have been invaluable and will no doubt enable the recommendations presented here to also be used by Federal agencies outside NIH. Dennis Rodrigues, Chief, On-Line information Branch, who supported Ann’s vision of this project and whose commitment to the use of Web analytics as a means to guide NIH Web strategy is an example of forward-looking leadership that will enable Federal agencies to thrive in the digital age. The Developing Web Analytics Best Practices Project Team members, who provided excellent feedback on the project’s presentations and research instruments: Cheryl Thompson, Web Manager, National Institute of Environmental Health Sciences; Craig Hicks, User Experience Designer, National Heart, Lung, and Blood Institute; Dennis Rodrigues, Chief, On-Line Information Branch, Office of the Director; Eric Djiva Kamal, Web Analytics System Administrator, National Library of Medicine; Fred Wood, Science Program Leader/Computer Scientist, National Library of Medicine; Ilene France, Digital Information Specialist, National Institute of Allergy and Infectious Diseases; James Petersen, Technical Writer/Editor, National Institute of Mental Health; Karina Boehm, Chief, Health Education Branch and Web Analytics Lead, National Institute of Dental and Craniofacial Research; Louise Marshall, Technical Information Specialist, National Institute of Mental Health; Mark Malamud, Web Analytics Program Manager and 508 Coordinator, National Heart, Lung, and Blood Institute; Sue Feldman, Web Analytics Program Manager, National Cancer Institute. My Semphonic colleagues who contributed their expertise and review of the Best Practices, as well as helping me develop new metrics and methods for conducting digital analytics for a Federal agency. I’d like to specifically recognize Gary Angel, Greg Dowling, Kevin Dupzyk, Jesse Gross, Chelsea Leonard, and Chris Meares. Phil Kemelor, Semphonic ES.ii Developing Web Analytics Best Practices for NIH, December 2011 Executive Summary The Developing Web Analytics Best Practices for NIH Project is unique among U.S. Federal Government agencies in terms of its breadth and scale. No other Federal agency has conducted research to identify the Web and digital analytics information needs of the whole agency and, then, how to make Web analytics tools — often relegated to Information Technology (IT) departments — relevant and useful to the agency’s communications and Web site development staff. The NIH On-Line Communications Branch obtained evaluation set-aside funds for this initiative and selected Semphonic, a vendor-neutral Web analytics consulting firm, to conduct the research and develop recommendations and the reference materials that are the deliverables of this project. While the project recommendations require NIH Institutes and Centers (IC) to have Web analytics solutions, they do not require any particular vendor’s product. The main goal of the project was to develop specific recommendations to improve the quality, consistency, and comparability of Web site measurement among public-facing sites throughout NIH. Another important goal was to help NIH and the ICs use Web-measurement data for strategic guidance and as support for decisions about and investments in NIH’s online initiatives. To understand how Web site measurement at NIH works now and to improve it for the future, the project focused on answering these questions: How do NIH ICs measure Web sites overall now (for example, site traffic, usability, customer satisfaction)? What specific Web-measurement tools do ICs use (for example, Webtrends, Omniture, Google Analytics, American Customer Satisfaction Index (ACSI) surveys)? How can ICs do a better job of using Web and digital analytics? Semphonic consultants worked closely with NIH staff. The project began by inviting the whole NIH Web and communications community to presentations about Web site and digital measurement at NIH. The introductory presentations had about 300 attendees (in person and online). Between January and May 2011, Semphonic interviewed 76 NIH staff from the