Ideas in American FOUNDATION Number 14 Policing May 2012 Learning About Learning From Error James M. Doyle

here has been a lot of tragedies, can be replaced by justice practitioners may be ready learning from error going a new period dedicated to the to give it a try. There are strong T on in American criminal sustained, routine practice of arguments that the policing justice since the publication in learning from error? Criminal community can and should 1996 of the U.S. Department of Justice’s compilation of the first twenty-eight wrongful Ideas in American Policing presents commentary and insight from leading crimi- nologists on issues of interest to scholars, practitioners, and policy makers. The convictions exposed by DNA. papers published in this series are from the Police Foundation lecture series of the Does it indicate that criminal same name. Points of view in this document are those of the author and do not justice practitioners can adopt necessarily represent the official position of the Police Foundation. The full series is available online at http://www.policefoundation.org/docs/library.html. some version of the quality © 2012 Police Foundation. All rights reserved. reform initiatives that have reshaped other high-risk fields James M. Doyle, a Boston attorney, is currently a Visiting Fellow such as aviation and medicine? at the National Institute of Justice, where he is examining the Can the criminal justice system utility of the organizational-accident model for understanding and embrace “a theory of work, avoiding system-level errors in criminal justice. He has written which conceptualize[s] the extensively on the issue of eyewitness identification testimony continual improvement of in criminal cases, including a fourth edition of his treatise for quality as intrinsic to the work lawyers, Eyewitness Testimony: Civil and Criminal, and the itself” (Kenney 2008, 30)? historical narrative, True Witness: Cops, Courts, Science, and the Is it possible that the current Battle Against Misidentification. Mr. Doyle is a member of the era, characterized by episodic Police Foundation’s Research Advisory Committee. patches motivated by high-profile lead the way toward a systems- in the comfort that Scalia and the accepted the Green Book as a oriented approach to known system’s other apologists have call to action. For them, one errors and near misses that begins tried to offer them. The DNA error was too many. Dozens of to move beyond a culture of involve the sort of jurisdictions, independently of blame and builds partnerships bread-and-butter cases everyone each other, mobilized efforts to across all stakeholders in the had handled and would handle address the problems identified in criminal justice system. again, not arcane borderland the Green Book. The goal here is to outline a specimens. All veteran criminal The initial leadership came path for future exploration, not practitioners had seen mistakes in from different players in different to sell a fixed set of prescriptions. similar cases, and many veterans places. Former attorney general The possibilities I have in mind had themselves been involved Janet Reno, who decided are complex and difficult ones. with one—or at least with an that the Green Book would Still, we do have a place to start. uncomfortably close near miss. include commentary from It came as no surprise to veteran the full spectrum of criminal The Exonerations and detectives that eyewitnesses make justice system actors, provided mistakes; veteran detectives have an influential template. She Their Aftermath been to a lot of lineups and convened technical working Attempts by U.S. Supreme watched witnesses identify lots of groups under the auspices of NIJ, Court Justice Antonin Scalia, known innocent “fillers.” which brought together diverse among others, to dismiss the For the frontline troops, the stakeholders to hammer out and exonerations listed in the rarefied utilitarian calculations of publicize new criminal justice best National Institute of Justice’s error rate that fascinated Justice practices. These groups addressed (NIJ) Convicted by Juries, Scalia missed the point. Because crime scene investigations, Exonerated by Science: Case practitioners in all roles were death investigations, and Studies in the Use of DNA drowning in heavy caseloads, eyewitness evidence, among Evidence to Establish Innocence they could readily see that even other topics. Peter Neufeld After Trial (Connors, Lundregan, very low rates of error still and Barry Scheck, cofounders Miller, and McEwen 1996) result in a very high absolute of the , who (known to practitioners as the number of tragedies. Police had been among Reno’s Green Green Book) as a catalogue of practitioners confronting the Book commentators, called for a freakish mishaps have gained very early exonerations were uniquely learning-from-error initiative. little traction (Kansas v. Marsh sensitive to a key fact: whenever In North Carolina, the 2006). In part, this is because the wrong guy was convicted, the first impetus came from the whenever these arguments are right guy got away and claimed conservative Republican chief put forward the Innocence further victims. Of course, many justice of the North Carolina Project exposes yet another people managed to shrug this Supreme Court. In Boston and horrifying wrongful conviction. off as just the unavoidable cost Minneapolis, it came from elected But a more fundamental reason of doing business. But other district attorneys; in Illinois, that the influence of the DNA practitioners—again, particularly from Northwestern University’s cases continues to police practitioners—saw Center on Wrongful Convictions grow is that the criminal justice avoiding errors as a matter of and the Governor’s Commission system’s frontline practitioners— professionalism, workmanship, on Capital Punishment; and in the people who actually do the and, ultimately, self-respect, New Jersey, from a Republican work on the streets and in the not as an issue of social policy attorney general (Doyle 2005). courts—show very little interest (Bittner 1990). This early group More recently, the International

—— 2 —— Association of Chiefs of Police has decided to convene a wrongful convictions summit. If we can learn, as Because exonerations take place at (or many years past) the medicine has learned, to end of the normal criminal justice process, many of the pioneering treat errors as sentinel learning-from-error initiatives that responded to the DNA events to be studied rather exonerations were dominated by the lawyers who operate the than embarrassments to be system’s terminal phases. A group of lawyers in these circumstances buried, we can all do a tends to luxuriate in examining the question, “How did the much better job. police screw this one up?” and then to share their answers generously with the media. Some in the policing Others have catalogued evaluation of errors? It may be community have resisted the these efforts and considered that the police have the most temptation to duck these their merits. I want to look to gain. In the age of DNA, gatherings. This was not true at them as precursors and ask there is no point in pretending of everyone, of course; plenty where these first steps could that mistakes do not happen; of people just hoped the whole lead. There is potential here for illusions of infallibility have thing would blow over. In many a new approach to mistakes and been placed irretrievably out of places, nothing has changed. near misses that promises more reach. As things stand, the police This is not yet a mass movement. than the penitential baring of take the largest share of public But in all instances where police throats to media abuse blame for wrongful convictions anything positive has grown out and lawyers’ criticisms. Tragic anyway. In this environment, a of the lessons of exonerations mistakes inspired medicine and public commitment to the frank it has been because the police aviation to blaze trails towards confrontation of known errors have at least acquiesced in the cultures of safety in ways that and near misses can provide an process, and usually because illuminate what we might develop important bulwark for police they have actively participated in within the criminal justice system. legitimacy. Besides, there are or led it. Practitioners who do If we can learn, as medicine has errors other than exonerations— not share much else share this learned, to treat errors as sentinel wrongful releases, cold cases much: they all hate wrongful events to be studied rather than that stayed cold too long, hot convictions. Every time judges, embarrassments to be buried, we spots that were not identified or cops, prosecutors, or Innocence can all do a much better job. A eliminated, avoidable street stops Network lawyers took steps conscious effort to see errors as and frisks of harmless citizens— forward, they found allies from an opportunity to find abiding that can yield valuable lessons. all points of the criminal justice root causes can drastically reduce The approaches learned by system, often from among their future risks. medicine, aviation, and other courtroom adversaries (Saloom Do the police have the high-risk endeavors can knit 2010). most to lose from the frank policing and policing research

—— 3 —— back into the development of the lineup or photo array be were mistaken but they were criminal justice system policy administered by an investigator sincere, and the police had and practice. Pursuing those who: (1) does not know which usually gone by-the-book as their approaches can identify a member of the group is the book then stood. There were common ground on which suspect; (2) instructs the witness no obvious miscreants to hunt to mobilize and share the that the real perpetrator may or for and punish. The eyewitness insights of the past decade of may not be present; (3) displays reforms, largely by accident, evidence based and problem lineup suspects and the fillers were generated in a non-blaming solving policing expertise and to individually (sequentially) rather context. Just as importantly, recapture a role that has been than in a group (simultaneously), because of the nature of the muted since the Warren Court’s as in traditional practice; and (4) underlying psychological findings, original interventions in criminal solicits a confidence statement the remedial program the investigative practice in the from the witness at the time of eyewitness cases provoked marked 1960s. any choice. Advocates of this a dramatic departure from method argue that it prevents the criminal justice reformers’ The Wrong Man, The the inadvertent steering of a usual strategy: it did not try witness toward a suspect and to augment the retrospective Wrong Patient, The an unconscious bolstering of inspection of eyewitness cases at System’s Errors the witness’s confidence, while the adversary trial by inserting To see the potential in this new it also mutes the dangerous psychological testimony by orientation, it helps to take a “looks-most-like” properties of defense experts. The new reforms brief detour through one of the the traditional lineup because are forward-looking, aimed at best-known and most productive it converts a multiple-choice the prevention of eyewitness efforts to salvage something comparison test into a true/ errors before they happen, not from tragic mistakes: the recent false recognition test. Laboratory at catching them later. They fell wave of reforms that integrate studies of the model indicate into police territory, not into the into investigative operations that it produces a lower rate of lawyers’ courtroom realm. psychological findings concerning “false-positive” identifications These features resonate . of innocent lineup members at with contemporary medicine’s Innocent men who were the cost of a slightly higher rate quiet revolution in patient safety convicted by the testimony of of “false misses”—failures to (Kenney 2008). Just as the sincere but mistaken eyewitnesses identify the perpetrator when he criminal justice system is haunted dominate the DNA exonerations is present in the lineup (Wells, by the fact that it sometimes list. Reforms to the eyewitness Steblay, and Dysart 2011). convicts the wrong man, investigative process have moved In effect, the double-blind medicine is haunted by the fact forward in a diverse range of sequential method provides a that it sometimes operates on the jurisdictions. These reforms new, more conservative screening wrong patient. But when modern incorporate into local practice test for guilt. medical researchers began to new science-based procedures Two characteristics of the look carefully into wrong-patient for lineups and photo arrays— eyewitness exoneration experience events, they uncovered surprising principally the double-blind and the reforms it generated insights. For example, one sequential procedure—advocated stand out. To begin with, the intensive examination of a wrong- by psychological researchers. eyewitness wrongful conviction patient surgery discovered not This protocol requires that cases were generally no-villain just one but at least seventeen tragedies. The eyewitnesses errors (Chassin and Becher

—— 4 —— 2002). The patient’s face was errors of many individuals (active or, although properly collected draped so that the physicians errors) converge and interact and tested in the lab, distorted could not see it; a resident left with system weaknesses (latent in the courtroom presentation. the lab assuming the attending conditions), increasing the Cell phone, Metrocard, or other physician had ordered the likelihood that individual errors alibi information could have invasive surgery without telling will do harm. The practitioners been ignored or considered him; conflicting charts were and organizations involved in insignificant. Tunnel vision, overlooked; and contradictory these tragedies did not choose augmented by clearance rate and patient stickers were ignored. to make errors; they drifted caseload pressures from above, But the crucial point for the into them (Dekker 2011). may have overwhelmed the researchers was that no single These events involved normal investigators and the prosecutors. one of the seventeen errors they people, doing normal work, in Poorly funded or untrained catalogued could have caused the normal organizations (Dekker defense counsel may have wrong-patient surgery by itself. 2007). They suffered, in Charles failed to investigate alternative Analysis showed not Perrow’s (1984) memorable explanations or to execute only mistakes by individual phrase, “normal accidents.” Like effective cross-examination. The doctors and nurses, but also the Challenger launch decision, witness erred; the cops erred; the latent systemic problems. they were “mistake[s] embedded technicians erred; the prosecutors Communication between staff in the banality of organizational erred; the defense erred; the was terrible; computer systems life” (Vaughan 1996, xiv). judge and the jury erred; the did not share information. When These insights apply to a appellate court erred, too. No teams failed to function, no one wrong-man conviction. Our single one of these errors would was surprised or bothered because traditional wrongful conviction have been enough without the of a culture of low expectations narrative (the witness picked the others. The errors combined that “led [staff] to conclude wrong guy; the cops and the and cascaded; then there was a that these red flags signified not district attorney believed her; so tragedy. unusual, worrisome harbingers did the jury) is not adequate. The right answer to the but rather mundane repetitions Nor is it adequate to isolate the question, “Who is responsible of the poor communication to imperfections of one operator for this wrongful conviction?” is which they had become inured” or one investigative technique usually “Everyone involved, to (Chassin and Becker 2002). employed in the case—for one degree or another,” if not by Deviations from good practice example, the traditional non- making a mistake, then by failing had become normal, and a blind, simultaneous lineup—as to catch one. And “everyone” tragedy resulted. either a sole cause or a silver includes not only cops and What this meant to medical bullet vehicle to a stable solution. lawyers at the sharp end of the reformers was that the lessons Lots of things have to go system, but also legislators, policy of closely studied events such as wrong before the wrong man makers, funders, and appellate the Chernobyl meltdown and is convicted. Yes, the witness judges far from the scene of the Challenger launch tragedy has to choose the wrong man the event, who dictated the could be applied to health from an array, but the police conditions under which the sharp- care. Like those tragedies, the have to put him into the array end operators work. Look twice wrong-patient surgery was an in the first place and design the at the DNA-exposed wrongful organizational accident. No format of the array. Forensic convictions and you see that, as single error is sufficient to cause evidence on the crime scene Charles Perrow (1984, 9) noted, an organizational accident; the could have been overlooked “[T]ime and again, the operator

—— 5 —— is confronted by unexpected and when environments change or then discipline him: charge him, usually mysterious interactions science advances. No new set of sue him, fire him, at the very among failures, [so that] saying best practices or checklists can least shame him and exhort him that he should have zigged cover every circumstance, so an to do better. The bad-apple instead of zagged is possible only irreducible zone of discretion explanation is not only wrong as after the fact.” This is as true always survives, and operators a descriptive matter but it has a of a whole spectrum of criminal are forced to manage life within crippling impact on any remedial justice errors—mistaken releases, that zone. Every new checklist potential. prisoners lost in prisons, and cold is under immediate and constant In medicine, the endemic cases that stayed cold too long— assault from caseload pressure assumption had always been as of wrongful convictions. and other environmental factors “good man, good result.” As The habit of treating horrific from the moment it is written. Dr. Lucian Leape wrote in his wrongful convictions as single- Drift toward failure remains a seminal 1994 essay, Error in cause events, and then totaling threat to our new best practices Medicine: up, ranking, and prioritizing just as it was to their discredited Physicians are expected to these causes has produced predecessors (Dekker 2011). function without error, an useful reforms, but it does No one had more checklists expectation that physicians not really engage the nature than NASA; NASA launched translate into the need to be of the problem. The solutions Challenger anyway. infallible. One result is that physicians, not unlike test it has generated stop short of pilots, come to view error fundamentally improving future The Hunt for as a failure of character— system reliability. Although a new Bad Apples you weren’t careful enough, set of best practices or checklists you didn’t try hard enough. can be a helpful thing, they The single-cause approach is This kind of thinking lies have to be operationalized and especially flawed when everyone behind a common reaction by physicians: How can there be executed. And they have to be assumes, as we tend to do, that an error without negligence? maintained, monitored, evaluated, the single cause must be a bad and perhaps junked and replaced apple. If someone can be blamed, Transplant Leape’s description of medical culture into criminal justice, and homicide detective, prosecutor, defender, forensic scientist, or judge substitutes Practitioners do not want smoothly for physician. In this familiar conception, any error is to be blamed or to be an operator error: some surgeon, police officer, nurse, forensic part of the unpredictable scientist, or lawyer at the site was lazy, ill-trained, venal, or machinery of blaming careless. The task of conscientious professionals in this vision is colleagues; thus, nothing to act as the custodians of a presumptively safe system and gets reported. to protect it from incompetent and destructive humans (Dekker 2007).

—— 6 —— Health care reformers leaders oblige and undertake held accountable influences how quickly noticed that the first the hunt for the bad apple. But we make decisions and the quality effect of the focus on blaming when internal affairs, the grand of our decisions. How, and to culprit-operators is that it jury, or the civil rights division whom, we expect to be called on drives valuable reports of errors actually investigates the event to account for our performance and near misses underground closely, it almost invariably turns can affect our performance in (Berwick 1989). Practitioners out to have been a complex explicit and implicit ways (Woods do not want to be blamed or organizational accident with 2005.) Expectations for what will to be part of the unpredictable many contributing causes and no be seen as an adequate account machinery of blaming colleagues; villain suitable for shouldering the (and the consequences if the thus, nothing gets reported. exclusive blame. Unless things account is seen as inadequate) are This tendency affects agencies are so bad that a scapegoat is critical parts of a cycle of giving as well as individuals. In a absolutely required, a report is accounts and being called to blame-oriented environment, issued, expressing regret and account that shapes our conduct when sentinel events cannot be explaining that no individual will (Woods 2005, 3). This is as true buried completely, the pressure be punished. in the criminal justice system as it intensifies to keep them in-house That may be fair enough in is anywhere else. or to try to shift the blame to regard to the individual targets, The history of the someone else’s “house.” But but it is a bad place to stop. No exoneration cases illuminates since no individual “house” can one learns anything useful from a missing element in the fully explain an organizational these exercises and, to make architecture of the criminal accident, weaknesses that might matters worse, when fruitless justice system: the capacity for a be studied and understood are hunts for bad apples result in “forward-looking accountability” glossed over and remain latent little or no action, they actually that balances demands for in the system, waiting for the tend to decrease police legitimacy. individual responsibility with next patient or the next case to When an airline official on an the need for learning and come along. A search for the air crash scene says, “Let’s wait improvement (Sharpe 2003). cause blocks understanding of for the National Transportation We lack a vehicle for accounting how complex systems fail through Safety Board report,” the public for tragic outcomes that allows the confluent, cascading errors, and the media are generally for working on continuous active and passive, of multiple content to wait. When police quality improvement—a means contributors from many “houses” officials say after an exoneration, for anticipating and preventing (Woods 2005). “We’re doing a report,” the the next catastrophe before it There is a sense in which public response is more likely happens. everyone knows this, because we to be, “Let’s wait for the Detectives speak of making watch it play out all the time. whitewash.” cases; lawyers speak of trying Now and then, a tragic wrongful them. In this respect, the conviction or similar error Accounting for work of criminal justice system generates a public clamor for the actors mimics work in other punishment of the responsible Our Errors enterprises. The police operate bad apple. The most common Our decisions take place in the a production stage in which understanding of accountability context of an expectation that they make the case, and the holds that a tragic event requires we may be called on to give lawyers and judges run an that tragic punitive consequences accounts of those decisions to inspection stage, during which fall on somebody. The system’s various parties. The way we are

—— 7 —— the legal system evaluates the process inspection to improve adversary trial inspection stage investigators’ product. reliability and quality control. so that it comes out (from the Inspection during an It is axiomatic in these other prosecutors’ perspective) the adversary trial before a lay jury industries that all end-of-process right way. Exculpatory material is a permanent feature of our inspection schemes, although is hidden by prosecutors in system. It expresses fundamental they are necessary components order to convict the people the American convictions about of their system, are poor prosecutors believe are guilty the relationship between the routes to overall system quality without interference from red accused individual and the state. (Berwick 1989). Practitioners herrings that defense lawyers Besides, the lay jury’s one-time who are subject to inspection might have manufactured out of concentration on a specific are resourceful when it comes dissonant facts. narrative provides a bracing to avoiding the inspection The downstream challenge to the professional altogether or to gaming the consequences of hiding practitioner’s endemic tendency inspection when they cannot. exculpatory material are obvious to believe that we know the Those being inspected usually when a wrongful conviction case odds in our fields and therefore end up owning the process, and is seen in hindsight: the trial simply play those odds. their primary goal is usually stage inspectors were foiled and But the goal of the trial their own safety (Berwick prevented from catching the process is to protect this 1989). Criminal justice system mistake before DNA technology innocent citizen from the state. operators are not immune to intervened decades later. Less The DNA exonerations have these tendencies. obvious, but just as important augmented doubts about the Of course, only a tiny if prevention is our goal, are adversary trial’s efficacy even portion of criminal cases the upstream consequences of a in that limited role (Simon actually receive jury scrutiny. looming inspection architecture. 2011), but no one claims that This certainly has something The prosecutors in the the trial’s role is to analyze to do with the costs of wrongful conviction cases, like the investigative and charging trials in terms of time and workers in many production processes and make them more money. But it also reflects processes, evidently adopted a reliable in the future. A jury all professional practitioners’ “covert work system” (Woods that believes it has caught a shared disinclination to submit 2005). They decided to evade faulty investigation says “not to inspection by unpredictable well-known formal disclosure guilty” and nothing more. lay jurors, especially when that requirements and buried Appellate courts review the inspection takes place in an alternative narratives because legal procedures; they do not exposed zero-sum courtroom they believed sharing the reconsider the facts, and their contest where one side wins, exculpatory facts would interfere review is entirely backward- and one side loses, all. The with achieving the real goal looking. Both are necessarily disturbing segment of the assigned to them by officials to uninformative. exoneration list that recounts whom they were accountable. Medicine, aviation, and prosecutors’ failures to turn The police investigators may other modern industries that over exculpatory material does have been encouraged by the achieve high reliability in the not show the prosecutors’ prosecutors (or, mistakenly face of potential catastrophic desire to frame the known believing they had been failures regard as insane innocent, but rather reveals encouraged by the prosecutors, exclusive reliance on end-of- their impulse to shape the decided on their own) to avoid

—— 8 —— exploring and documenting discordant leads that might have falsified the original theory of the case. Both of these practices But the goal of the trial augment the universal human tendency toward tunnel vision process is to protect this by rewarding surrender to tunnel vision with a “cleaner” innocent citizen from trial inspection for the hypothesis that the practitioners the state. prematurely decided is accurate. Tunnel vision makes a causal contribution to the wrongful conviction, but aggravated tunnel vision is also an effect from the victim’s family, and concrete consequences fell on the of the sharp-end operators’ that an eyewitness identification prosecutors. discomfort with the demands did not match Thompson. This case has provoked of the end-stage inspection Thompson’s conviction was a heated campaign for more machinery. overturned on appeal. On retrial, criminal charges and more All of this is on display in a jury exonerated Thompson aggressive discipline of unethical the recent U.S. Supreme Court in only thirty-five minutes. The prosecutors, including more case of Connick v. Thompson public’s most lasting impression severe punishment designed to (2011). John Thompson was was undoubtedly “the cops got hold prosecutors accountable convicted of murder in New the wrong guy; the prosecutors (Gertner and Scheck 2012). We Orleans in 1985. After a trial covered up for the cops.” can never dispense altogether where he opted not to testify, The only available response with disciplinary actions aimed at Thompson was sentenced to for addressing this travesty was a consciously unethical behavior. death and spent the next eighteen civil lawsuit, which by its nature Characters like rogue forensic years in prison, fourteen of them meant hunting for someone scientist Fred Zain, who falsified on death row. A few weeks before to blame. The line prosecutors lab work in dozens of cases, will Thompson’s scheduled execution were dead, judgment proof, or crop up from time to time. But in 1999, a defense investigator immune; the office of the district there is an additional failure in learned that a cancer-stricken attorney denied responsibility accountability embedded in the member of the prosecution team for its staffers’ actions. In other Thompson aftermath—a failure in had confessed on his deathbed words, both the operators and forward-looking accountability. to having withheld crime lab their bosses successfully ran for No one learned anything about results from the defense, as well cover, and the U.S. Supreme the real, abiding issues in the as removing a blood sample from Court held that the government Thompson narrative, and those the evidence room. In addition, could not be held liable under issues were left to surface again in Thompson’s defense learned the 1964 Civil Rights Act since future cases. that the New Orleans district the Thompson wrongful conviction Unless sociopathic Thompson attorney’s office had failed to was (as far as the judges knew) prosecutors consciously set out disclose that Thompson had been a single incident. Thompson, with the goal of convicting a implicated in the murder by a the victim of this misconduct, known innocent, the question person who received a reward was not compensated; no that Thompson raises is not

—— 9 —— whether the choices of either the as if error, not conspiratorial of the mistakes. That is why a district attorney’s office as an malfeasance, is what we are sentinel-event approach is crucial. agency or the individual frontline confronting. A process that would get us prosecutors who hid the evidence Why couldn’t the defenders somewhere would involve calling were wrong. Of course those themselves find the exculpatory to the table representatives of choices were wrong. The real evidence? Why didn’t the all stakeholder roles to analyze question is why did the mistaken detectives know this and follow known errors or near misses, not choices seem to be good choices at up on the court’s failure to make distilling each to a single cause or the time (Dekker 2011). Or, use of the a single villain but appreciating at least, why did the mistaken they had generated during their and describing its complexity. As choices seem like the only, or investigation? Why did they Lucien Leape (1994) noted in least bad, ones. Exculpatory decide to stand by silently and the medical context, “Efficient, evidence has to be turned over. watch the trial unfold? After all, routine, error identification needs Why didn’t the prosecutors know the police were going to take the to be part of hospital practice this? Why, knowing it, did they blame for any error in the end. as does routine investigation decide their best choice was Were the detectives caught in of all errors that cause injury.” not to act on it? While it might the classic administrative double The all-stakeholders approach be perfectly fitting to blame blind, held accountable for an is necessary if we decide to prosecutors who knowingly outcome they did not feel they see a wrongful conviction or disregard legal requirements or had the authority to control or other error as an organizational best practices, what about the influence? accident that required cascading, other participants? There is a Congratulating ourselves confluent contributions from difference between malfeasance on simply seeing that mistaken everyone’s “house.” And, among and error and, further up and choices were made will not the stakeholders, practitioners down the stream from the get us anywhere if we do not not too far divorced from sharp- prosecutors, it looks very much account for the root causes end practice have to be included. Medical reformers are fond of pointing to a mysterious outbreak of central line infections in a Pittsburgh intensive care unit that was solved by the janitor member The best way forward of the error-review team (Kenney 2008.) allows practitioners The missing weapon in our approach to error is not a once- themselves to nominate in-a-decade, blue-ribbon panel of august dignitaries at the their sentinel events, their chief justice and superintendent level, convened to redesign the collaborators, and their architecture of the criminal justice system. We have examples of that approach now, and it cannot formats. be denied that, when the goal is changing structural elements by legislation or rulemaking,

—— 10 —— the political heft of those high- participants and the public expect everywhere will start doing this ranking players is useful, even the National Transportation immediately and all of the time. essential (Saloom 2010). Safety Board to convene a mixed There are some jurisdictions What is missing is a team of specialists to give an where the police leadership knows commitment to regular, routine account of what happened when that the chief public defender is review of known errors and near a plane goes down, criminal a self-aggrandizing loudmouth, misses, conducted by experienced practitioners and the public or the district attorney is a practitioners and stakeholders could learn to expect that we vainglorious idiot, or the unions (for example, victims’ rights will marshal a team, including an are permanently and obstructively professionals) supplemented investigator or patrol supervisor, antagonistic. Even in jurisdictions where appropriate by subject- a prosecutor, a forensic where none of this is true, there matter experts and (at least in scientist, a defender, a judge, a are some sentinel events that will the beginning) by specialists victims’ representative, and the be simply too hot to handle. But in analyzing the sources of jurisdiction’s risk management a lot will be achieved if some system error and in the error- officers, joined by additional people in some places start doing review process itself. As Leape specialists as needed, in a non- this some of the time and share (1994) argued, “The emphasis blaming process of dissecting the product of their work. is on routine. Only when error the facts of what happened and The best way forward allows is accepted as an inevitable, sharing the account they have practitioners themselves to although manageable, part developed. The goal would be nominate their sentinel events, of everyday practice will it be to understand the gritty facts, to their collaborators, and their possible to shift from a punitive do the sort of clinical fact-finding formats. Since nothing will to a creative frame of mind that inevitably suffers when work unless everyone consents, that seeks out and identifies the everyone in a turf-conscious, why not allow practitioners to underlying system failures.” blue-ribbon group is anxiously choose? In this scheme, the looking over his or her shoulder police can go first. The police What If? Continually at potentially sweeping and are best positioned to identify unwelcome legal reforms. the common ground—in this Improving Reliability Continually working on particular example, a hatred What if, when the next wrongful improving system reliability of wrongful convictions and a conviction is announced, the means changing the system’s forward-looking approach to police—usually described as culture, not its architecture. avoiding them—onto which all famously inbred, suspicious, Overhauling institutional stakeholders are willing to enter, paranoid, and wrapped in arrangements, identifying best and (even more importantly) a pathological blue code of practices, and devising checklists, to set the terms on which silence—amaze the world by as difficult as these might be, they and all other stakeholders calling for an all-stakeholders are the easy parts. Working on would be willing to defend examination of the error? Or, changing the culture means that common ground (Woods what if DNA results back from concentrating on giving a primary 2005). Stakeholders will have to the lab six months after an arrest place to workmanship and agree to forego some short-term show that they arrested the professionalism instead of blame pleasures (e.g., paying off in the wrong guy, and police suggest and discipline. media an ancient grudge against that a team examination of this It would take a more a counterpart agency) in order near miss might pay dividends? messianic temperament than my to pursue the longer-term and Just as all aviation industry own to believe that everyone, more fundamental goal of fully

—— 11 —— understanding the root causes of (Andrews International Group would uncover—and not in the the organizational accident under 2010). context of excuse-making—the scrutiny. In all of these examples, operational constraints imposed It is not as if learning concerns about incident liability on the police by actors far from from error in the aftermath of were successfully balanced against the immediate scene of the police catastrophe has never succeeded the need to manage future investigation. It would expose from a police perspective. The risks by identifying root causes everyone’s role in the outcome. Los Angeles Police Department’s that will trigger repetitions. It would illuminate many frank morning-after analysis of Participants recognized that the police choices that are dictated the MacArthur Park disorder new, marginal exposure caused by (or at least heavily influenced) and the role of its own staff a careful evaluation was minimal. by the policy and practices of had disciplinary consequences, The worst that could happen prosecutors, judges, or other but the report’s thrust was not had already happened; if you actors. It would supplement disciplinary. Much was learned were going to be sued, you were the in-house impulse to go from it, and public confidence in going to be sued. The goal of “down and in” to find a broken the LAPD and its leadership rose not getting sued again was kept component with the modern measurably when the report was in mind. safety expert’s recognition that, released (McGreevey and Winton The “keep this in-house” because no organization acts 2007). Milwaukee police regularly impulse is understandable, and in isolation, we also need to participate in the Milwaukee even has its more attractive go “up and out” to assess the Homicide Review Commission’s side—an ethic of responsibility, environment’s impact on existing interdisciplinary investigations of of cleaning up our own mess. procedures and practices (Dekker homicides (O’Brien, Woods, and Still, this admirable impulse 2011). Cisler 2007). feeds a go-it-alone approach that A coherent program of non- A near miss can work just suffers from built-in limitations. blaming, learning from error that as well or better as a sentinel Read any police-only scrutiny includes the evaluation of near event. The Will County, of a mistaken conviction and misses offers rewards both within Illinois, Sheriff’s Department you are left to ask, “Where were local systems as well as across commissioned an investigation the prosecutors?” or “What was scattered systems. A common, by the experienced staff of it about the trial environment national template for error review, Andrews International Group that made this seem like a good enacted locally and informed (including former chiefs John idea?” or “Who funded this crime and challenged by diverse local Timoney and Patrick Hartnett) lab?” or even “Where was the experiences, can substantially to analyze investigative missteps defense?” Comparable efforts mitigate the fragmentation of in the mistaken prosecution of an by prosecutors’ offices to handle American criminal justice. Illinois father who was exonerated conviction integrity entirely in- These advantages can pretrial by DNA in the rape and house suffer from comparable be multiplied if a simple murder of his daughter. The handicaps (see Scheck 2010). Of mechanism—a clearinghouse report shed light on the origins course, an in-house investigation or a wiki-style community of the father’s might ask those questions but it of practitioners, researchers, and on the efficacy of new DNA cannot answer them. and policy makers—could be techniques, while generating An all-stakeholders review, developed for distributing and cogent recommendations for by contrast, would involve commenting on the reports of the future and avoiding personal everyone in the discussion. It errors (Doyle 2010). This process humiliation of any of the actors

—— 12 —— could provide a platform where participants would not be forced to choose between the valuable contributions of evidence-based policing (Weisburd and Neyroud Today’s police lieutenants 2011) and problem-solving policing (Sparrow 2011), but will make better police could mobilize both and perhaps alert both schools to factors captains next year thanks that neither had yet considered. Reading of a distant system’s to their participation experience of completed accidents can alert currently isolated in the rigorous practitioners to the operation of dangerous latent features in organizational accident their own local systems. Reading studies of remote near misses analysis of a known error can reveal both those dangerous latent features as well as potential or near miss. fail-safe devices or procedures not present locally, which provided resilience and kept the near miss from becoming a tragic hit. It can counteract the tendency of where well-meaning actions on arguably indispensable—element today’s best practice to calcify this coast can have disastrous, of a new professionalism (Stone into a ceiling that blocks future unanticipated impacts on the and Travis 2011). Today’s police improvements. far shore. Ignoring this fact will lieutenants will make better fulfill the axiom that the cause of police captains next year thanks problems is solutions. Improve to their participation in the Conclusion your property crime clearance rigorous organizational accident There is truth in the warnings of rate by swabbing everything in analysis of a known error or near David Kennedy (Wessels 2006), sight and you will raise the rape- miss. Bratton’s “collaborate or among others, that conditioning kit backlog at the lab. perish” is a good warning to any decision to act on But there is opportunity keep in mind, and the power of establishing our firm confidence as well as danger in this collaboration to improve forward- in “fixing the criminal justice interdependency of criminal looking accountability is one system” is a recipe for paralysis. justice’s operators. Working more reason for collaboration There is no arrangement of gears steadily on organizational error (Bratton and Tumin 2012). A and switches in criminal justice, analysis creates an increased disciplined commitment to team no system in that sense that system consciousness among analysis of error can lay the we can reach for and fix with a the practitioners who staff the foundation in criminal justice wrench or a hammer. But, like components of the criminal for realizing the new ideal of it or not, the world of criminal process. The forward-looking continuous quality improvement justice is a complex functioning accountability that this practice that is transforming the culture of ecosystem like a pond or a swamp creates can be an important—and contemporary medicine.

—— 13 —— References /opinions/10pdf/09-571.pdf Street Journal. March 16. _ (accessed April 23, 2012). http://online.wsj.com/article Andrews International Group. /SB1000142405270230469 2010. Comprehensive Connors, Edward, Thomas 2804577281852966541834. Operational Assessment, Lundregan, Neal Miller, html (accessed April 24, Criminal Investigative Unit, and Tom McEwen. 2012). Sheriff’s Office, Will County, 1996. Convicted by Juries, Illinois. December 16. Exonerated by Science: Case Kansas v. Marsh. 2006. 548 http://www.scribd.com Studies in the Use of DNA U.S. 163, 199 (concurring /doc/47496706/Andrews- Evidence to Establish Innocence opinion of Justice Antonin Report (accessed April 18, After Trial. Washington, DC: Scalia). http://www 2012). U.S. Department of Justice, .supremecourt.gov/opinions National Institute of Justice. /boundvolumes/548bv.pdf Berwick, Donald. 1989. https://www.ncjrs.gov (accessed April 18, 2012). Continuous quality /pdffiles/dnaevid.pdf improvement as an ideal in Kenney, Charles. 2008. The (accessed April 18, 2012). medicine. New England Best Practice: How the Journal of Medicine 320: Dekker, Sydney. 2011. Drift into New Quality Movement is 53–56. http://www.nejm. Failure: From Hunting Broken Transforming Medicine. org/doi/full/10.1056 Components to Understanding New York: PublicAffairs. /NEJM198901053200110 Complex Systems. Farnham, Leape, Lucian L. 1994. Error (accessed April 18, 2012). UK: Ashgate Publishing Ltd. in medicine. Journal of the Bittner, Egon. 1990. Aspects Dekker, Sydney. 2007. Just American Medical Association of Police Work. Boston: Culture: Balancing Safety and 272: 1851–1857. http:// Northeastern University Accountability. Farnham, UK: jama.ama-assn.org/ Press. Ashgate Publishing Ltd. content/272/23/1851.full .pdf+html (accessed April 18, Bratton, William, and Zachary Doyle, James M. 2005. True 2012). Tumin. 2012. Collaborate Witness: Cops, Courts, Science or Perish! Reaching Across and the Battle Against McGreevey, Patrick, and Richard Boundaries in a Networked Misidentification. New York: Winton. 2007. Bratton issues World. New York: Crown Palgrave MacMillan. report on melee. Los Angeles Business. Times, May 30. http:// Doyle, James M. 2010. articles.latimes.com/2007 Chassin, Mark R. and Elise C. Learning from error in /may/30/local/me-lapd30 Becher. 2002. The wrong American criminal justice. (accessed April 18, 2012). patient. Annals of Internal The Journal of Criminal Medicine 136: 826–833. Law & Criminology 100: O’Brien, Mallory, Laurie Woods, http://www.annals.org 109–148. http://www and Ron A. Cisler. 2007. The /content/136/11/826.full .law.northwestern.edu Milwaukee Homicide Review .pdf+html (accessed April 18, /jclc/backissues/v100 Commission: An interagency 2012). /n1/1001_109.Doyle.pdf collaborative process to (accessed April 18, 2012). reduce homicide. Wisconsin Connick v. Thompson. 2011. Medical Journal 106: 563 U.S.___. http:// Gertner, Nancy, and Barry 385–388. http://www www.supremecourt.gov Scheck. 2012. How to rein .wisconsinmedicalsociety. in rogue prosecutors. Wall

—— 14 —— org/_WMS/publications trials. Vanderbilt Law Review Wells, Gary L., Nancy K. Steblay, /wmj/issues/wmj_v106n7 64: 143–223. http://www and Jennifer E. Dysart. 2011. /106no7_obrien_wms3.pdf .vanderbiltlawreview.org A Test of the Simultaneous vs. (accessed April 18, 2012). /content/articles/2011/01 Sequential Lineup Methods: /Simon-Limited-Diagnosticity- An Initial Report of the Perrow, Charles. 1984. Normal 64-Vand.-L.-Rev.-143-2011. AJS National Eyewitness Accidents: Living with High- pdf (accessed April 18, Identification Field Studies. Risk Technologies. New York: 2012). Des Moines: American Basic Books. Judicature Society. http:// Sparrow, Malcolm K. 2011. Saloom, Stephen. 2010. www.ajs.org/wc/ewid Governing Science, New Adversaries as allies: Joining /ewid_report.asp (accessed Perspectives in Policing, together to prevent criminal April 18, 2012). Executive Sessions on injustice. Albany Law Review Policing and Public Safety, Weisburd, David, and Peter 73: 1235–1243. http:// January. Cambridge, MA: Neyroud. 2011. Policing www.albanylawreview.org Harvard Kennedy School. Science: Toward a New /articles/06%20Saloom.pdf http://www.hks.harvard. Paradigm, New Perspectives (accessed April 18, 2012). edu/var/ezp_site/storage in Policing, Executive Sessions Scheck, Barry. 2010. Professional /fckeditor/file/pdfs on Policing and Public Safety, and conviction integrity /centers-programs January. Cambridge, MA: programs: Why we need /programs/criminal-justice Harvard Kennedy School. them, why they will work, /NPIP-GoverningScience.pdf http://www.hks.harvard. and models for creating (accessed April 18, 2012). edu/var/ezp_site/storage them. Cardozo Law Review /fckeditor/file/pdfs/centers- Stone, Christopher, and Jeremy 31: 2215–2256. http:// programs/programs/criminal- Travis. 2011. Toward a New www.cardozo.yu.edu justice/NPIP-Police%20Science- Professionalism in Policing, /uploadedFiles/Cardozo TowardaNewParadigm.pdf New Perspectives in Policing, /Profiles/burns_ethics-145 (accessed April 18, 2012). Executive Sessions on Policing /Barry%20Scheck.pdf and Public Safety, March. Wessels, Joe. 2006. Expert: (accessed April 18, 2012). Cambridge, MA: Harvard City can reduce crime. Sharpe, Virginia A. 2003. Kennedy School. http:// Cincinnati Post, December Promoting Patient Safety: www.hks.harvard.edu/var 29. http://www.highbeam. An Ethical Basis for Policy /ezp_site/storage/fckeditor com/doc/1G1-156644783 Deliberation. Hastings /file/pdfs/centers-programs .html (accessed April 24, Center Report Special /programs/criminal-justice 2012). Supplement 33, No. 5, S1– /ExecSessionPolicing/NPIP- Woods, David D. 2005. Conflicts S20, July/August. http:// TowardsaNewProfessionalismin between learning and www.thehastingscenter.org Policing-03-11.pdf (accessed accountability in patient /uploadedFiles/Publications April 18, 2012). safety. DePaul Law Review /Special_Reports/patient Vaughan, Diane. 1996. The 54: 485–502. _safety.pdf (accessed April 18, Challenger Launch Decision: 2012). Risky Technology, Culture, and Simon, Dan. 2011. The limited Deviance at NASA. Chicago: diagnosticity of criminal University of Chicago Press.

—— 15 —— ABOUT THE POLICE FOUNDATION The Police Foundation is a national, nonpartisan, nonprofit organization dedicated to supporting innovation and improvement in policing through its research, technical assistance, communication, and professional services programs. Established in 1970, the foundation has conducted seminal research in police behavior, policy, and procedure, and works to transfer to local agencies the best new information about practices for dealing effectively with a range of important police operational and administrative concerns. Motivating all of the foundation’s efforts is the goal of efficient, humane policing that operates within the framework of democratic principles and the highest ideals of the nation

DIVISION OF RESEARCH, EVALUATION, BOARD OF DIRECTORS & PROFESSIONAL SERVICES

Karen L. Amendola David Weisburd Chairman Chief Operating Officer Senior Fellow Weldon J. Rougeau Garth den Heyer Travis Taniguchi Senior Research Fellow Senior Research Associate President Hubert Williams Edwin E. Hamilton LeRoy O’Shield Professional Services Senior Technical Assistance George H. Bohlinger III Director Advisor David D. Cole Raymond Johnston, Sr. Kristin Koval Senior Systems Engineer Research & Administrative Coordinator Clarence Edwards

Paul Helmke RESEARCH ADVISORY COMMITTEE Julie Horney David Weisburd, Chair Hebrew University and George Mason University William H. Hudnut III James M. Doyle David B. Lewis Carney & Bassil Jack R. Greene W. Walter Menninger Northeastern University Elsie L. Scott Gary LaFree University of Maryland Alfred A. Slocum James J. Willis George Mason University Andrew L. Sonner

POLICE FOUNDATION

1201 Connecticut Avenue, NW, Washington, DC 20036-2636 (202) 833-1460 • Fax (202) 659-9149 • E-mail: [email protected] www.policefoundation.org