Communicating Error and Expertise in Forensic Expert Testimony
Total Page:16
File Type:pdf, Size:1020Kb
Communicating error and expertise in forensic expert testimony Gianni Ribeiro Bachelor of Psychological Science (Hons I) Graduate Certificate in Tertiary and Adult Education https://orcid.org/0000-0002-2594-8311 A thesis submitted for the degree of Doctor of Philosophy at The University of Queensland in 2020 School of Psychology Abstract Forensic comparison evidence, such as fingerprints, plays an important role in criminal investigations and proceedings. For more than 100 years, forensic examiners were permitted to testify that two prints, marks, or samples matched to the exclusion of all other people, with some even claiming that their decisions were infallible. However, numerous wrongful identifications have demonstrated that these claims of infallibility simply are not true. In fact, forensic science testing errors and false or misleading forensic testimony are two common factors leading to the wrongful conviction of innocent individuals. As a result of these misidentifications, forensic expert testimony is now beginning to reflect the uncertainty of forensic comparison evidence but the method in which experts should communicate uncertainty is under debate. Many have supported the use of likelihood ratios — which communicates the probability of an observation given two competing hypotheses: that the two samples share the same source (H1), or that the two samples originate from different sources (H2). However, others have argued that communicating a specific likelihood ratio is a futile endeavour when considering that the probability of a false positive (e.g., mislabelling of samples) is far greater. The overall aim of this thesis is to introduce a diagnostic information approach to communicating forensic expert testimony, which draws on signal detection theory to convey examiners’ accuracy and error. Specifically, I aim to a) determine people’s existing beliefs about accuracy and error in forensic science, b) determine whether likelihood ratios affect sensitivity to exculpatory evidence, such as an alibi, and c) compare the likelihood ratio approach to the diagnostic information approach to determine which results in better sensitivity to the strength of the evidence. There are three main empirical chapters of this thesis. In Chapter 2, I explore laypeople’s perceptions of error and involvement of human judgment in forensic science, both in terms of the stages of the forensic science process (e.g., collection, storage) and for specific forensic techniques. I find that people do not necessarily have blind faith in forensic sciences, as participants believed that there was quite a high likelihood of error, as well as a high level of human judgment involved, in the forensic science process. Furthermore, participants had lower perceptions of accuracy for specific forensic techniques than expected, ranging from 65.18% for document analysis to 89.95% for DNA analysis. i In Chapter 3, I turn to the use of likelihood ratios, investigating whether likelihood ratios affect people’s sensitivity to the strength of exculpatory evidence, such as an alibi, in two preregistered experiments. I found that those who were presented with a likelihood ratio were more likely to conclude that the suspect was the source of the DNA evidence and guilty of the crime compared to those who were not presented with a likelihood ratio. On the other hand, as the strength of the suspect’s alibi increased, people were less likely to conclude that the suspect was the source of the evidence and guilty of the crime. However, unexpectedly, I found that participants who received a likelihood ratio were actually more sensitive to the strength of the suspect’s alibi, but this appeared to be driven largely by the low ratings in the strongest alibi condition. Finally, this pattern of results held across both experiments, despite the likelihood ratio increasing by two orders of magnitude (5,500 in Experiment 1 and 5,500,000 in Experiment 2), indicating that participants were, problematically, not sensitive to the value of the likelihood ratio. In Chapter 4, I compare the likelihood ratio approach to a novel approach to expert testimony, which I call the diagnostic information approach. This approach provides people with information about forensic examiners’ accuracy and error (hits, misses, false alarms, and correct rejections). I find that participants are more sensitive to the strength of the fingerprint evidence when it is presented diagnostically rather than as a likelihood ratio. Further, I find that when both diagnostic information and a likelihood ratio are presented together, people’s resulting judgments about the evidence are driven primarily by the diagnostic information and not the likelihood ratio. Overall, I conclude that laypeople are sensitive to error in forensic sciences. Even before hearing expert testimony, people have their own beliefs about the likelihood of an error occurring during the forensic science process and about the accuracy of various forensic techniques — they do not blindly believe that forensic science is infallible. When hearing expert testimony, people are far better at discriminating between strong and weak evidence when presented diagnostically (i.e., information about examiners’ accuracy and error) than as a likelihood ratio. These findings demonstrate a promising approach towards communicating the uncertainty of forensic decisions in a way that jurors can accurately evaluate. ii Declaration by author This thesis is composed of my original work, and contains no material previously published or written by another person except where due reference has been made in the text. I have clearly stated the contribution by others to jointly-authored works that I have included in my thesis. I have clearly stated the contribution of others to my thesis as a whole, including statistical assistance, survey design, data analysis, significant technical procedures, professional editorial advice, financial support and any other original research work used or reported in my thesis. The content of my thesis is the result of work I have carried out since the commencement of my higher degree by research candidature and does not include a substantial part of work that has been submitted to qualify for the award of any other degree or diploma in any university or other tertiary institution. I have clearly stated which parts of my thesis, if any, have been submitted to qualify for another award. I acknowledge that an electronic copy of my thesis must be lodged with the University Library and, subject to the policy and procedures of The University of Queensland, the thesis be made available for research and study in accordance with the Copyright Act 1968 unless a period of embargo has been approved by the Dean of the Graduate School. I acknowledge that copyright of all material contained in my thesis resides with the copyright holder(s) of that material. Where appropriate I have obtained copyright permission from the copyright holder to reproduce material in this thesis and have sought permission from co-authors for any jointly authored works included in the thesis. iii Publications included in this thesis Ribeiro, G. Tangen, J. M., & McKimmie, B. M. (2020). Does DNA evidence in the form of a likelihood ratio affect perceivers’ sensitivity to the strength of a suspect’s alibi? Psychonomic Bulletin & Review. Advanced online publication. https://doi.org/10.3758/s13423-020-01784-x Ribeiro, G., Tangen, J. M., & McKimmie, B. M. (2019). Beliefs about error rates and human judgment in forensic science. Forensic Science International. 297(1), 138- 147. https://doi.org/10.1016/j.forsciint.2019.01.034 iv Submitted manuscripts included in this thesis Ribeiro, G. McKimmie, B. M., & Tangen, J. M. (in preparation). Diagnostic information produces more accurate judgments about forensic comparison evidence than likelihood ratios. v Other publications during candidature Journal publications Searston, R. A., Thompson, M. B., Robson, S. G., Corbett, B. J., Ribeiro, G., Edmond, G., & Tangen, J. M. (2019). Truth and transparency in expertise research. Journal of Expertise, 2(4), 199-209. Chin, J. M., Ribeiro, G., & Rairden, A. (2019). Open forensic science. Journal of Law and the Biosciences, 6(1), 255-288. https://doi.org/10.1093/jlb/lsz009 Ribeiro, G., & Antrobus, E. A. (2017). Investigating the impact of jury sentencing recommendations using procedural justice theory. New Criminal Law Review, 20(4), 535-568. https://doi.org/10.1525/nclr.2017.20.4.535 Edmond, G., Towler, A., Growns, B., Ribeiro, G., Found, B., White, D., Ballantyne, K., Searston, R. A., Thompson, M. B., Tangen, J. M., Kemp, R. I., & Martire, K. A.(2016). Thinking forensics: Cognitive science for forensic practitioners. Science & Justice, 57(2), 144-154. https://doi.org/10.1016/j.scijus.2016.11.005 Edmond, G., Found, B., Martire, K., Ballantyne, K., Hamer, D., Searston, R. A., Thompson, M. B., Cunliffe, E., Kemp, R., San Roque, M., Tangen, J. M., Dioso-Villa, R., Ligertwood, A., Hibbert, B., White, D., Ribeiro, G., Porter, G., Towler, A., & Roberts, A. (2016). Model forensic science. Australian Journal of Forensic Sciences, 48(5), 496-537. https://doi.org/10.1080/00450618.2015.1128969 Conference presentations and posters Ribeiro, G. (18 September, 2020). Improving understanding of forensic evidence and expertise: A diagnostic approach. Invited talk presented for the 2020 Postgraduate Student Excellence Award. School of Psychology, The University of Queensland. Ribeiro, G., Tangen, J. M., & McKimmie, B. M. (25 June, 2020). Improving understanding of forensic evidence and expertise: A diagnostic approach. Talk presented at the Australian Forensic Psychology Online Conference. Ribeiro, G., Tangen, J. M., & McKimmie, B.M. (6-7 November, 2019). Improving understanding of forensic evidence and expertise. Invited talk presented at the vi New South Wales Police Fingerprint Experts Conference. Paramatta, Australia. Ribeiro, G., Tangen, J. M., & McKimmie, B.M. (6-9 June, 2019). Does DNA evidence in the form of likelihood ratios make perceivers less sensitive to the strength of the suspect’s alibi?. Talk presented at the 13th Society for Applied Research in Memory and Cognition Conference.