Pattern recognition and forensic identification: The presumption of scientific accuracy and other falsehoods IR Coyle, D Field and P Wenderoth* Decision-making in forensic contexts where patterns (such as fingerprints) are compared involves processes of perception and cognition which are notoriously fallible in many circumstances. The known or potential rate of error in those scientific methods of forensic identification which have long been accepted by the courts is often higher than would generally be perceived, despite the presumption of accuracy of such techniques. In this article, the authors argue that errors arising from perceptual and cognitive errors in such forensic identification evidence are overwhelmingly due to the misuse and profound lack of understanding of basic epistemological and statistical principles. To avoid miscarriages of justice, these principles need to be understood and safeguards employed so that the legal process is not contaminated by pseudoscience. INTRODUCTION Few things can be more damning to the prospects of a defendant than the unqualified pronouncement of an authoritative expert that there is forensic evidence directly linking the accused to a crime scene. Before the First Fleet sailed into Sydney Harbour, Lord Mansfield issued the following warning, the basic thrust of which is still apposite today: The fact that an expert witness has impressive scientific qualifications does not by that fact alone make his opinion on matters of human nature and behaviour within the limits of normality any more helpful than that of the jurors themselves. But there is a danger that they may think it does.1 Forensic evidence comes in many forms. To name but a few tools of trade of forensic scientists, this may involve comparison of latent fingerprints found at the crime scene with exemplar prints either obtained pursuant to a forensic order or extant in some database; comparison of bite marks on a victim with orthodontic analysis; DNA evidence; hair and fibre matching; or other emerging techniques based on the anthropometric or biomedical characteristics of humans. Whatever their genesis, all of these techniques have one thing in common: the potential for human error. The decision that evidence found at a crime scene can be matched to a particular suspect is made by a human. While mathematical algorithms processed with computational power almost beyond comprehension may reduce to a manageable number the comparisons that need to be made, the ultimate decision is always made by a human. This startlingly simple observation has far-reaching consequences that have not been fully appreciated or accommodated by the legal system. Decision-making, whether in a forensic context or otherwise, is as much a function of the processes of the human mind as of the technology on which such decisions are founded. It is as meaningless to try to isolate the two as it is to try to determine what caused a motor vehicle accident by only considering the vehicles involved and the traffic conditions, whilst ignoring the decisions made by the drivers. Decision-making in forensic contexts, in which patterns of various types are compared, involves processes of perception and cognition. Perception and cognition are notoriously fallible in many * Ian Coyle: Visiting Professorial Fellow, Forensic Psychologist, Forensic Ergonomist and Forensic Psychopharmacologist, Bond University Centre for Forensic Excellence; Principal Consultant Safetysearch Forensic Consultants, Gold Coast, Queensland. David Field: Associate Professor of Law, Director, Bond University Centre for Forensic Excellence. Peter Wenderoth: Professor of Psychology, Macquarie University. The authors would like to thank Professor Don Thomson and the Honourable Tim Carmody for their helpful comments on the manuscript. 1 Folkes v Chadd (1782) 99 ER 589. 214 (2009) 33 Crim LJ 214 Pattern recognition and forensic identification circumstances. This has long been recognised with respect to the identification of an accused person as the perpetrator of a crime by way of eyewitness testimony, and legal safeguards have evolved to limit the consequences of error when such evidence is adduced, albeit that these safeguards are often useless.2 It is therefore curious that such safeguards are, in the main, conspicuously absent when the perceptions of forensic experts that patterns they have observed in evidence (whether they be fingerprints, bite marks, or DNA) match the characteristics of a suspect. This appears to be due in large measure to the misuse and profound lack of understanding of basic epistemological and statistical principles not only among lawyers, but also among scientists and other forensic “experts”. This has led, inter alia, to the legal doctrine of the presumption of scientific accuracy. When a scientific instrument may be said to belong to: [a] class of instruments of a scientific or technical character, which by general experience [are] known to be trustworthy, and are so notorious that the court requires no evidence to the effect that they do fall into such class, before allowing the presumption in question to operate with regard to readings made thereon,3 a court will, at common law, be entitled to take what is called “judicial notice” of its reliability. This means that the results or readings which are derived from such an instrument, may be relied on in evidence when this is relevant to the outcome of a case. This doctrine is founded on the notion that some tests are so notorious in their accuracy that it would require statistical improbability on a vast scale for them to be wrong in any particular case. Unfortunately, the same presumption of accuracy seems to have become applied to so-called scientific testing procedures which rely not upon the use of instruments, but the bare application of human judgment, albeit skilled and experienced judgment. Whatever may be presumed in law, errors in decision-making in tests undoubtedly do occur. These errors are common in the area of forensic identification, and are so ubiquitous as to require a fundamental reassessment of the way such evidence is received by the courts. A convenient place to start when considering this proposition is with biometric identification since this has the longest history. BIOMETRIC IDENTIFICATION Biometric measurement goes back to the 19th century, when Alphonse Bertillon categorised a series of measurements of the human body (forearm length, hand width etc). These were used to describe an individual. Literally tens of thousands of such measurements were collected in England, America and France and used to obtain convictions, typically of habitual criminals. Despite misgivings, the most significant of which was the report of the Royal Commission in 1898 in England, the system continued to be used. Then there occurred the case of Mr Will West. In 1903, Mr West was incarcerated in Leavenworth prison, Kansas, in the United States. His Bertillon measurements were taken and were identical, based on 15 matching points of comparison, to another inmate who had been admitted to Leavenworth two years earlier and was still there. This was the genesis of the requirement to have 16 matches or points of comparison that migrated, through a process of unscientific osmosis, to the system of fingerprint matching. Fingerprint examination replaced the Bertillon system during the early part of the 19th century. Since then, the criteria for obtaining a “match” have varied. In most of Europe, a fingerprint match requires 16 points; in Greece it is 10; and in Turkey eight points are required. In the United States and Australia, no specific criteria are used – the analyst simply forms an opinion that a latent and “exemplar” print (ie one taken from the suspect) match. There is no empirical or statistical basis for these thresholds: none whatsoever. As Thompson and Cole noted: 2 Coyle IR, Field D and Miller G, “The Blindness of the Eye-witness” (2008) 82 ALJ 471. 3 Porter v Kolodzeil [1962] VR 75 at 78. (2009) 33 Crim LJ 214 215 Coyle, Field and Wenderoth Latent Print Examiners (LPEs) have no scientific basis to estimate the probability of a random match between two impressions, and they present no statistics in connection with their testimony. If they find sufficient consistent detail they simply declare a positive identification of individualization, claiming the potential donor for the mark has been reduced to one and only one area of friction ridge skin in the world to the exclusion of all other friction ridge skin in the world.4 And so LPEs in Australia and North America simply state that the latent print matched or did not match the exemplar print and ignore the vexatious issue that they might not be correct 100% of the time when making such judgments. LPEs routinely assert that latent and exemplar prints can be matched by properly trained examiners with no realistic chance of incorrect matching (ie making a false positive error). This is a comforting thought for an accused. It is also wrong. Errors in fingerprint analysis have been known since the 1920s. Typically these have been shrugged off as being due to poor training, poor supervision or difficulties in matching poor quality latent impressions with exemplars in a database. The cases of Brandon Mayfield in the United States and Shirley McKie in Scotland have conclusively demonstrated that these arguments will not fly.5 In both of these cases, the most experienced LPEs in North America and the United Kingdom conclusively and comprehensively identified the wrong person. What processes of decision-making are involved when a forensic scientist compares latent and exemplar evidence of whatever type? The short answer is that no one knows. Because forensic scientists making decisions as to the similarity or otherwise of such evidence produce almost no documentation, it is very difficult (if not impossible) to determine, post facto, what led them to their conclusion. It seems clear from the fact that there is no standard in North American and Australia vis-à-vis the number of indicia that must match before a positive identification is called that LPEs must consider the overall pattern of the latent print as well as individual minutiae, but how this is done is not clear.