RESEARCH PROGRAM PEER REVIEW: PURPOSES, PRINCIPLES, PRACTICES, PROTOCOLS DR. RONALD N. KOSTOFF OFFICE OF NAVAL RESEARCH 800 N. QUINCY STREET ARLINGTON, VA 22217 INTERNET:
[email protected] PHONE: 703-696-4198 FAX: 703-696-4274 (THE VIEWS IN THIS REPORT ARE SOLELY THOSE OF THE AUTHOR AND DO NOT REPRESENT THE VIEWS OF THE DEPARTMENT OF THE NAVY) TABLE OF CONTENTS I. ABSTRACT II. EXECUTIVE SUMMARY - PEER REVIEW PRINCIPLES III. INTRODUCTION, DEFINITIONS, AND BACKGROUND IV. PEER REVIEW PURPOSES AND PRINCIPLES V. PEER REVIEW PRACTICES VI. PEER REVIEW PROTOCOLS VI-A. APPENDIX I - REVIEW PANEL SELECTION APPROACHES VI-B. APPENDIX II - PROGRAM PEER REVIEW PROTOCOL VI-C. APPENDIX III - USE OF PUBLISHED PAPERS IN RESEARCH PROGRAM EVALUATION VI-D APPENDIX IV – NETWORK-CENTRIC PEER REVIEW VI-E APPENDIX V – SIGNIFICANT PEER REVIEW RESOURCES V-1. PROPOSAL PEER REVIEW V-2. PEER REVIEW GUIDE V-3. BIOMEDICAL PEER REVIEW CONGRESSES VI-F LARGE AGENCY PEER REVIEW VI-G DETAILED PEER REVIEW PROTOCOL VII. BIBLIOGRAPHY AND RELATED REFERENCES KEYWORDS: peer review; GPRA; evaluation criteria; metrics; research assessment; retrospective studies; roadmap; data mining; text mining; research merit; research approach; blind review; research evaluation; research impact; bibliometrics; decision aids. Page 1 Form Approved Report Documentation Page OMB No. 0704-0188 Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302.