How Forensic Science Lost Its Way and How It Might Yet Find It Michael J
Total Page：16
File Type：pdf, Size：1020Kb
ANRV359-LS04-08 ARI 15 October 2008 14:37 Failed Forensics: ANNUAL REVIEWS Further Click here for quick links to Annual Reviews content online, How Forensic Science including: • Other articles in this volume Lost Its Way and How • Top cited articles • Top downloaded articles • Our comprehensive search It Might Yet Find It Michael J. Saks1 and David L. Faigman2 1College of Law and Department of Psychology, Arizona State University, Tempe, Arizona 85287; email: [email protected] 2Hastings College of the Law, University of California, San Francisco, California 94102; email: [email protected] Annu. Rev. Law Soc. Sci. 2008. 4:149–71 Key Words First published online as a Review in Advance on courts, Daubert, science, scientific evidence July 8, 2008 The Annual Review of Law and Social Science is Abstract online at lawsocsci.annualreviews.org A group of nonscience forensic sciences has developed over the past cen- This article’s doi: tury. These are fields within the broader forensic sciences that have little 10.1146/annurev.lawsocsci.4.110707.172303 or no basis in actual science. They are not applications of established 1550-3585/08/1201-0149$20.00 basic sciences, they have not systematically tested their own hypothe- ses, and they make unsupported assumptions and exaggerated claims. This review explains the nature and origins of those nonscience forensic fields, which include the forensic individualization sciences and certain other areas, such as fire and arson investigation. We explore the role of the courts in maintaining the underdeveloped state of these fields and consider suggestions for improving this state of affairs (addressing the potential role that could be played by these fields themselves, by the courts, and by normal sciences). 149 ANRV359-LS04-08 ARI 15 October 2008 14:37 INTRODUCTION produced such time-tested technologies as bloodletting and phrenology. Surprisingly little of what most people think The fields that most resemble those ancient they know about major aspects of the foren- nonscience sciences are the forensic identifi- sic sciences is as they imagine it to be. This cation sciences, as well as certain other spe- review focuses on what a recent report of the cialties within forensic science. By identifi- National Institute of Justice (2007) refers to cation sciences we mean those subfields that as the soft forensic sciences, or what might be often are referred to as criminalistics and that more accurately described as the nonscience involve pattern matching in an effort to as- forensic sciences. The public appears to con- sociate a crime scene mark or object with its sider the various forensic sciences as one whole, source. These subfields include the compar- believing them all to be flawless applications of ison of fingerprints, handwriting, bitemarks, sound basic science to problems of law. Courts voiceprints, toolmarks, firearms, tire prints, generally share this view. As a Utah Court of shoe prints, and so on. Their goal is to link a Appeals judge commented in regard to finger- latent fingerprint, a writing, a bitemark, a bul- print expert testimony: “In essence, we have let, or similar objects to the one and only fin- adopted a cultural assumption that a govern- ger, writer, teeth, gun, or other specific object ment representative’s assertion that a defen- in the world that made the markings. The ul- dant’s fingerprint was found at a crime scene timate objective, and postulated achievement, is an infallible fact, and not merely the exam- of forensic identification science is individual- iner’sopinion” (State v. Quintana 2004, Thorne, ization, “[t]he process of placing an object in J., concurring, p. 171). The belief in fingerprint a category which consists of a single, solitary technology may be the most extreme example, unit. Individualization implies uniqueness ...” but it is by no means the only. The trust that (Thornton & Peterson 2008, p. 71). “Crim- is laid upon the forensic sciences generally falls inalistics is the science of individualization” somewhere between uncritical faith and manu- (Osterburg 1969, p. 97). For example, a firearms factured myth. examiner testifying in a federal court claimed to The nonscience forensic sciences, as the be able to identify the unknown weapon “to the paradoxical phrase suggests, are those fields exclusion of every other firearm in the world” within forensic science that have little or no ( 2005, p. 107). basis in actual science. They neither borrow U.S. v. Green Beyond the identification sciences are cer- from established science nor systematically test tain other subfields of forensic science that as- their hypotheses. Their primary claims for va- sert knowledge that was neither borrowed from lidity rest on anecdotal experience and procla- extant basic science nor rigorously tested by mations of success over time. Hypothesis and forensic experts before being offered to courts. supposition are typically considered sufficient. These fields include the areas of fire, arson and Whereas in most scientific fields experience and explosives, gunshot residue, comparative bullet observation are designated as the first steps of lead analysis, and aspects of forensic pathology. the scientific method, for many forensic fields These areas of claimed expertise share with the they constitute the final stages of confirma- forensic identification sciences the irrational tion. Indeed, in a way, many practitioners of the reliance on unspecified, unsystematic “experi- forensic arts have turned the scientific method ence” coupled with plausible-sounding argu- on its head. So long as their hypotheses and ments as the nearly exclusive bases for their suppositions have not been tested, they are as- hypotheses. sumed true. Hypotheses that endure over time, The nonscience forensic sciences have failed rather than actually being tested, are deemed on several levels. They are scientific failures proven. This model was once pervasive in in the sense that science (either in substance applied settings, especially in medicine, and or in methodology) played little more than a 150 Saks · Faigman ANRV359-LS04-08 ARI 15 October 2008 14:37 rhetorical part in the development of these ences might be made scientific and what such a fields (Saks 2007). The word and the ac- world might look like. coutrements of science were exploited to sell these fields to courts and to the public (Cole 2001), but they did not trouble themselves to SCIENTIFIC FAILURES build upon a foundation of substantive scien- Although science is not monolithic, the many tific knowledge or to employ the methods of forms science assumes share certain character- science to self-critically evaluate their wares. istics. Perhaps foremost, good science has a They are also technological failures. The spe- strong element of formality associated with it. cific techniques that have been developed and This formality permits what might be the key deployed in these fields make unknown num- attribute of the scientific method: reproducibil- bers of errors—both false negatives (failing to ity. Although postmodern sensibilities are wont detect what they sought to detect, or to identify to stress objectivity as a keystone of science, vir- what they sought to identify) and false positives tually all science strives toward what Sir Karl (seeming to detect something that did not exist, Popper (1949) called intersubjective testability. mistakenly identifying an innocent person or In a nutshell, this means that the essential char- object as having been connected to the crime acteristics of any scientific test can be under- under investigation). Yet despite these core stood well enough to be replicated by another failures, the nonscience forensic sciences have group of researchers. Researchers must specify been extraordinarily “successful” as measured all the details employed in testing the hypoth- by their admission into trial courts and their esis of interest so that outsiders can know just apparent acceptance by the public. The reasons what was done. This requires clear statements for acclaim in the public domain is a subject be- of operational definitions, full exposition of re- yond the scope of this review, although promi- search methods employed and statistics used, nence on nightly television programs and in and strict limits on the degree of subjectiveness feature-length movies must constitute part of relied upon in reaching judgments or conclu- any explanation. Acceptance among courts and sions. The scientific method, therefore, is not lawyers, however, is somewhat easier to docu- merely a way of testing hypotheses; it permits ment, though possibly as difficult to explain. outsiders to know how the tests were conducted In this review, we focus on the nonscience and allows other researchers to repeat and con- forensic sciences and consider the quandary of firm them. how such profoundly flawed disciplines could From the law’s perspective, as a consumer of achieve such worldly success. In examining this scientific opinion, knowing what methods were puzzle, we explore three fundamental concerns. used to support various theories or hypothe- The first section considers the basic scientific ses should be an imperative. Without knowing failure of these disciplines. This consideration in detail how (and whether) a field tested its includes a brief survey of their historical origins hypotheses, it is impossible for courts to know and a detailed analysis of their compositions, how valuable, if valuable at all, expert opinions in an effort to identify how they came to be are that come from that field. Although anec- and to understand how they persist. The second dotal experience sometimes produces valuable section considers the law’s role in perpetuating knowledge, without verifiable tests it is impos- these disciplines. Although courts provide the sible to know when it does so. primary markets for the experts coming from Ostensibly scientific expert opinion, there- these fields, they have done virtually nothing to fore, must be evaluated on the basis of fun- evaluate critically the bases for the opinions that damental principles of good hypothesis test- representatives from these fields offer into ev- ing.