New Mexico Statistical Analysis Center Quality of the Administrative Office of the Courts’ Disposition and Sentencing Data Prepared by: Kristine Denman Callie Dorsey Formatting and editing support: Ashleigh Maus Graham White Research support: Mariah Simplicio Joel Robinson Vaughn Fortier Shultz Drew Medaris Elizabeth Sabbath Jenna Dole February 2020 This project was supported by Grant # 2017-BJ-CX-K018 from the State Justice Statistics program. The State Justice Statistics program is a component of the Office of Justice Programs, which also includes the Bureau of Justice Statistics, the National Institute of Justice, the Office of Juvenile Justice and Delinquency Prevention, and the Office for Victims of Crime. Points of view or opinions in this document are those of the author and do not represent the official position or policies of the United States Department of Justice. Acknowledgements We would like to thank the Administrative Office of the Courts for their willingness to provide these data to us and for their participation in this project. We would also like to thank the New Mexico Sentencing Commission (NMSC) who stores copies of the data used in this report, and provided the data directly to us. Both Linda Freeman, Executive Director for the NMSC, and Dr. Noah Painter-Davis provided valuable comments as we drafted this report. We appreciate their feedback. Finally, we would like to thank the Bureau of Justice Statistics for providing the funding for this study through their State Justice Statistics Program. i Executive Summary Introduction The Administrative Office of the Courts (AOC) is the principal source for criminal case disposition and sentencing information in New Mexico. Although the primary users of agency-created data are typically those within the agency, others may also access the data, including researchers. Unlike intra-agency users who typically interact with the data on an individual case level, researchers look at the raw underlying data either line-by-line or aggregate the data for analysis. Research results generated from these data can be used to measure criminal justice performance, assess reforms, and inform policy. However, the quality of these data can vary over time, by jurisdiction, and by the specific data fields. The current study evaluates the quality of the AOC’s disposition and sentencing data. We are undertaking this assessment both to provide feedback to the AOC regarding the strengths and limitations of the data, but also to help secondary users like us better understand the data in order to ensure that the data are used responsibly to inform research and policy. In this study we focus on the completeness, validity, uniqueness/duplication, consistency, and accuracy of both disposition and sentencing data.1 Key strengths of AOC data Completeness was very high. o Most cases (92%) with a conviction had a sentence populated. o Most cases (89%) with a conditional discharge had a sentence populated. o Most cases (81%) with a deferred sentence had the amount populated. o Nearly all dispositions were populated, though this is likely an artifact of the parameters of the data pull. Face validity was high for disposition types as well as sentences associated with most disposition types. o All of the disposition types appeared reasonable. o Less than 1% of cases with outright dismissals or acquittals of all charges had a sentence. Duplication of data was low. o Approximately 1% of charges were duplicated. o Most commonly, duplications were the result of disparate data in one of seven categories and likely do not originate with the AOC; rather, duplication was due primarily to the appending process the NMSC uses as they receive updated data. Accuracy of disposition data among selected cases was high. o Approximately 5% of cases had an erroneous disposition. o When erroneous, it most often involved a conditional discharge or deferral recorded as guilty/no contest or conviction. Key limitations of AOC data Completeness of amended or updated sentences may be limited. o Updated sentences may not be applied to all charges in a case. While this is likely reasonable for the AOC, it is something secondary data users need to be aware of. Accuracy of sentence information among selected cases lower than may be expected. 1 Appendix E includes a glossary of key terms used throughout this report. ii o Accuracy of total confinement sentence was lower than the accuracy of total probation sentences (65% accuracy for total confinement, 80% accuracy for total probation). Regardless of sentence type (confinement or probation), the most common errors were omissions. Total confinement was often missing presentence confinement credit or suspensions. o In some cases, court clerks did populate the field “suspend all,” but due to an apparent glitch, this was not subtracted from the total sentence. This is expected to be remedied in the next version of Odyssey. Total probation was most often erroneous because it was not populated, though in some cases it may be that the sentence was not yet decided when the data were pulled by AOC. Consistency varies. o Judgement and Sentence (J&S) documentation differs across judicial districts. Some districts write the J&S in narrative format, some use a form to summarize each of the sentence components, and some write out each portion in a narrative and provide a summary at the end. o Conditional discharges are not always recorded as a disposition. o Sentences are recorded in varying ways. By charge, by case recorded under a single charge, or by case recorded under all charges with a sentence. This causes issues when the sentence is recorded by each charge as it is not clear whether to sum the sentences, calculate the longest sentence, or some combination of the two. While there are flags to indicate if the sentence is concurrent/consecutive we found that 35% of the sample had some issue with the flags. Even when there is no issue the flags could refer to another charge in the case or another case altogether. The most common recording method varies by sentence type (confinement versus probation). The method of recording sentences varies by judicial district and over time. A little less than half of conditional discharges and deferred sentences that are dismissed have a sentence recorded. Credits and suspensions are sometimes, but not always, accounted for in the total sentence. Suggestions for improvement Standardize J&S sentence summaries. o While each judicial district should have the ability to determine how they would like to record sentence information in the J&S, adopting a standard summary format to ensure that all parties receiving the data fully understand the judge’s intent would be beneficial. o This summary should include the total sentence before and after suspensions, credits, and enhancements, and specify each of those credits and enhancement amounts. It iii should also clearly state whether presentence confinement and probation credits will be applied to the incarceration term or probation term. One of the key challenges with using these data is that the sentence can be recorded in a variety of ways. One way to remedy this would be to have a set of fields that summarizes the entire sentence in a case or set of concurrent cases. However, our understanding is that the software developer cannot add that. In the absence of this fix, we recommend the following: o If the sentence reflects the total sentence in the case, it should be entered under a single charge to avoid confusion. This would require that the J&S documents include a summary sentence for a given case or combination of cases. o There are currently flags to indicate whether the charges are concurrent of consecutive. If feasible, we suggest exploring whether values could be added to these fields that would clarify how the sentence can be calculated. For example, it may be helpful to include a value “concurrent to another case” or “concurrent to another charge.” Create or update data entry guide. o It is unclear whether there is a comprehensive data entry guide, as when asked, we received documentation of some components of the system, but not a comprehensive guide. If not available, one should be created. This would help to ensure consistency with data entry across judicial districts. o Review existing documentation or create documentation to ensure that it clearly explains how particular components are to be entered in those areas where we found inconsistencies (e.g., whether PSC was recorded). o Require that conditional discharges be entered as dispositions. Offer data entry guides to secondary data users. o A data entry guide could help secondary users to better understand the data since a codebook is not available. Consider expanding disposition categories available in Odyssey, if possible. o While not a primary focus of this study, we did find a handful of lower court review (LR) cases, all of which involved decisions about pretrial release. The current disposition categories reflect options for a criminal case. Given that New Mexico passed an amendment to reform bail practices, it would be beneficial to include options related to the disposition of these cases (e.g., pretrial release granted or not). Including a disposition appropriate to these cases could help assess the implementation of bail reform. iv Introduction Criminal justice agencies collect and maintain administrative data. One such agency is the Administrative Office of the Courts (AOC). The AOC creates and retains disposition and sentencing data in New Mexico. The AOC has a comprehensive data system (Odyssey) for the recording of court case information. Court clerks are responsible for entering the data into the system, typically within a short time after each event in the case. Clerks may record the disposition and sentence (if applicable) for each charge or for the case overall.