REPORT TO THE PRESIDENT Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods Executive Office of the President President’s Council of Advisors on Science and Technology September 2016 REPORT TO THE PRESIDENT Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods Executive Office of the President President’s Council of Advisors on Science and Technology September 2016 About the President’s Council of Advisors on Science and Technology The President’s Council of Advisors on Science and Technology (PCAST) is an advisory group of the Nation’s leading scientists and engineers, appointed by the President to augment the science and technology advice available to him from inside the White House and from cabinet departments and other Federal agencies. PCAST is consulted about, and often makes policy recommendations concerning, the full range of issues where understandings from the domains of science, technology, and innovation bear potentially on the policy choices before the President. For more information about PCAST, see www.whitehouse.gov/ostp/pcast. The President’s Council of Advisors on Science and Technology Co-Chairs John P. Holdren Eric S. Lander Assistant to the President for President Science and Technology Broad Institute of Harvard and MIT Director, Office of Science and Technology Policy Vice Chairs William Press Maxine Savitz Raymer Professor in Computer Science and Honeywell (ret.) Integrative Biology University of Texas at Austin Members Wanda M. Austin Christopher Chyba President and CEO Professor, Astrophysical Sciences and The Aerospace Corporation International Affairs Princeton University Rosina Bierbaum S. James Gates, Jr. Professor, School of Natural Resources and John S. Toll Professor of Physics Environment, University of Michigan Director, Center for String and Roy F. Westin Chair in Natural Economics, Particle Theory School of Public Policy, University of University of Maryland, College Park Maryland Christine Cassel Mark Gorenberg Planning Dean Managing Member Kaiser Permanente School of Medicine Zetta Venture Partners v Susan L. Graham Ed Penhoet Pehong Chen Distinguished Professor Emerita Director in Electrical Engineering and Computer Alta Partners Science Professor Emeritus, Biochemistry and Public University of California, Berkeley Health University of California, Berkeley Michael McQuade Barbara Schaal Senior Vice President for Science and Dean of the Faculty of Arts and Sciences Technology Mary-Dell Chilton Distinguished Professor of United Technologies Corporation Biology Washington University of St. Louis Chad Mirkin Eric Schmidt George B. Rathmann Professor of Executive Chairman Chemistry Alphabet, Inc. Director, International Institute for Nanotechnology Northwestern University Mario Molina Daniel Schrag Distinguished Professor, Chemistry and Sturgis Hooper Professor of Geology Biochemistry Professor, Environmental Science and University of California, San Diego Engineering Professor, Center for Atmospheric Sciences Director, Harvard University Center for Scripps Institution of Oceanography Environment Harvard University Craig Mundie President Mundie Associates Staff Ashley Predith Diana E. Pankevich Executive Director AAAS Science & Technology Policy Fellow Jennifer L. Michael Program Support Specialist vi PCAST Working Group Working Group members participated in the preparation of this report. The full membership of PCAST reviewed and approved it. Working Group Eric S. Lander (Working Group Chair) Michael McQuade President Senior Vice President for Science and Broad Institute of Harvard and MIT Technology United Technologies Corporation S. James Gates, Jr. William Press John S. Toll Professor of Physics Raymer Professor in Computer Science and Director, Center for String and Integrative Biology Particle Theory University of Texas at Austin University of Maryland, College Park Susan L. Graham Daniel Schrag Pehong Chen Distinguished Professor Emerita Sturgis Hooper Professor of Geology in Electrical Engineering and Computer Professor, Environmental Science and Science Engineering University of California, Berkeley Director, Harvard University Center for Environment Harvard University Staff Diana E. Pankevich Kristen Zarrelli AAAS Science & Technology Policy Fellow Advisor, Public Policy & Special Projects Broad Institute of Harvard and MIT Writer Tania Simoncelli Senior Advisor to the Director Broad Institute of Harvard and MIT vii Senior Advisors PCAST consulted with a panel of legal experts to provide guidance on factual matters relating to the interaction between science and the law. PCAST also sought guidance and input from two statisticians, who have expertise in this domain. Senior advisors were given an opportunity to review early drafts to ensure factual accuracy. PCAST expresses its gratitude to those listed here. Their willingness to engage with PCAST on specific points does not imply endorsement of the views expressed in this report. Responsibility for the opinions, findings, and recommendations in this report and for any errors of fact or interpretation rests solely with PCAST. Senior Advisor Co-Chairs The Honorable Harry T. Edwards Jennifer L. Mnookin Judge Dean, David G. Price and Dallas P. Price United States Court of Appeals Professor of Law District of Columbia Circuit University of California Los Angeles Law Senior Advisors The Honorable James E. Boasberg The Honorable Pamela Harris District Judge Judge United States District Court United States Court of Appeals District of Columbia Fourth Circuit The Honorable Andre M. Davis Karen Kafadar Senior Judge Commonwealth Professor and Chair United States Court of Appeals Department of Statistics Fourth Circuit University of Virginia David L. Faigman The Honorable Alex Kozinski Acting Chancellor & Dean Judge University of California Hastings College of United States Court of Appeals the Law Ninth Circuit Stephen Fienberg The Honorable Cornelia T.L. Pillard Maurice Falk University Professor of Statistics Judge and Social Science (Emeritus) United States Court of Appeals Carnegie Mellon University District of Columbia Circuit viii The Honorable Charles Fried The Honorable Jed S. Rakoff Beneficial Professor of Law District Judge Harvard Law School United States District Court Harvard University Southern District of New York The Honorable Nancy Gertner The Honorable Patti B. Saris Senior Lecturer on Law Chief Judge Harvard Law School United States District Court Harvard University District of Massachusetts ix EXECUTIVE OFFICE OF THE PRESIDENT PRESIDENT’S COUNCIL OF ADVISORS ON SCIENCE AND TECHNOLOGY WASHINGTON, D.C. 20502 President Barack Obama The White House Washington, DC 20502 Dear Mr. President: We are pleased to send you this PCAST report on Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods. The study that led to the report was a response to your question to PCAST, in 2015, as to whether there are additional steps on the scientific side, beyond those already taken by the Administration in the aftermath of the highly critical 2009 National Research Council report on the state of the forensic sciences, that could help ensure the validity of forensic evidence used in the Nation’s legal system. PCAST concluded that there are two important gaps: (1) the need for clarity about the scientific standards for the validity and reliability of forensic methods and (2) the need to evaluate specific forensic methods to determine whether they have been scientifically established to be valid and reliable. Our study aimed to help close these gaps for a number of forensic “feature-comparison” methods—specifically, methods for comparing DNA samples, bitemarks, latent fingerprints, firearm marks, footwear, and hair. Our study, which included an extensive literature review, was also informed by inputs from forensic researchers at the Federal Bureau of Investigation Laboratory and the National Institute of Standards and Technology as well as from many other forensic scientists and practitioners, judges, prosecutors, defense attorneys, academic researchers, criminal-justice-reform advocates, and representatives of Federal agencies. The findings and recommendations conveyed in this report, of course, are PCAST’s alone. Our report reviews previous studies relating to forensic practice and Federal actions currently underway to strengthen forensic science; discusses the role of scientific validity within the legal system; explains the criteria by which the scientific validity of feature-comparison forensic methods can be judged; and applies those criteria to the selected feature-comparison methods. x Based on our findings concerning the “foundational validity” of the indicated methods as well as their “validity as applied” in practice in the courts, we offer recommendations on actions that could be taken by the National Institute of Standards and Technology, the Office of Science and Technology Policy, and the Federal Bureau of Investigation Laboratory to strengthen the scientific underpinnings of the forensic disciplines, as well as on actions that could be taken by the Attorney General and the judiciary to promote the more rigorous use of these disciplines in the courtroom. Sincerely, John P. Holdren Eric S. Lander Co-Chair Co-Chair xi Table of Contents The President’s Council of Advisors on Science and Technology ................................................... v PCAST Working Group ..................................................................................................................