RESEARCH Natural Language Processing-Based Quanti cation of the Mental State of Psychiatric Patients Sankha S. Mukherjee1, Jiawei Yu1, Yida Won1, Mary J. McClay 1, Lu Wang1, A. John Rush 2,3,4, and Joydeep Sarkar1 an open access journal 1Holmusk Inc., Singapore 2National University of Singapore, Singapore 3Duke University School of Medicine, Durham, North Carolina, USA 4Texas Tech University Health Sciences Center, Permian Basin, Texas, USA Keywords: natural language processing, psychiatry, mental health ABSTRACT Psychiatric practice routinely uses semistructured and/or unstructured free text to record the behavior and mental state of patients. Many of these data are unstructured, lack standardization, and are difficult to use for analysis. Thus, it is difficult to quantitatively analyze a patient’s illness trajectory over time and his or her responsiveness to treatment, and it is also difficult to compare different patients quantitatively. In this article, experts in the field of psychiatry, along with machine learning models, have collaboratively transformed patient data available in status assessments generated by physicians into binary vector representations. Data from patients with mental health disorders collected within a Citation: Mukherjee, S. S., Yu, J., Won, Y., McClay, M. J., Wang, L., Rush, A. J., real-world clinical setting from one of the largest behavioral electronic health record (EHR) & Sarkar, J. (2020). Natural language systems in the United States have been used for generating these representations. The binary processing-based quantification of the mental state of psychiatric patients. vector representation of these health records is shown to be useful in various clinical tasks, Computational Psychiatry, 4, 76–106. https://doi.org/10.1162/cpsy_a_00030 such as disease phenotyping, characterizing the suicidality of patients, and inferring diagnoses. To summarize, this approach can transform semistructured free-text summaries of DOI: https://doi.org/10.1162/cpsy_a_00030 patients’ status assessments into a structured, quantifiable format, which enriches the data that reside within EHR systems. This allows for effective intra- and interpatient quantifications Supporting Information: and comparisons, which are much needed in the field of mental health. With the aid of these http://doi.org/10.1162/cpsy_a_00030 binary representations, patients’ mental states can be systematically tracked over time, as can Received: 11 October 2019 Accepted: 6 November 2020 their responses to medications at the individual and population levels. Competing Interests: The authors declare no conflict of interest. Corresponding Author: INTRODUCTION Sankha S. Mukherjee [email protected] Mental disorders are among the most challenging illnesses to treat due to the paucity of biomarkers that identify and quantify the severity of disease, as is standard in other therapeutic areas. Diagnosis in mental health is based upon observations as specified by systems such Copyright: © 2020 Computational Psychiatry, Inc. and the as the Diagnostic and Statistical Manual of Mental Disorders, fifth edition (DSM-5; American Massachusetts Institute of Technology. Published under a Creative Commons Psychiatric Association, 2013) and the International Classification of Diseases (ICD; World Attribution 4.0 International Health Organization, 1993). Categorization within such systems predates modern neuroscience (CC BY 4.0) license. (Marshall, 2020), hence the contextualization of advances in neuroscience within such clas- sification systems for disease and severity is rather difficult. Furthermore, unlike in other areas The MIT Press Quantification of the Mental State of Psychiatric Patients Mukherjee et al. of health care, external experiences and social constructs typically have a significant influence on the prognosis and efficacy of the treatment of mental health disorders. There exists a significant gap between advances in neuroscience and their ultimate trans- lation into treatment decisions. This has led to a call for a more rigorous, evidence-based sys- tem, called the Research Domain Criterion (RDoC; Cuthbert, 2014; Cuthbert & Insel, 2013; Cuthbert & Kozak, 2013), that will attempt to classify disorders based upon a combination of domains/constructs of behavior and mental capacity and units of analysis such as genes, molecules, cells, brain circuits, physiology, behavior, self-reports, and other paradigms. This is a radical step away from a purely behavior-based system to a more evidence-based sys- tem. Computational psychiatry has been proposed to be a bridge that can further accelerate the translation of neuroscientific research into clinical practice (Huys, Maia, & Frank, 2016). Greater adoption of RDoC and similar systems will enable large, multiscale models to be used in conjunction with traditional therapeutics to improve the treatment of mental health disorders. The MindLinc Global Database (MGD) is a repository of longitudinal patient records with behavioral health disorders from more than 25 hospitals that use the MindLinc electronic health record (EHR; Beyer, Kuchibhatla, Gersing, & Krishnan, 2005). It is one of the largest longitudinal behavioral health databases in the world to capture clinical data in a real-world setting. MGD comprises records of more than 500,000 patents, with more than 14 million certified visits. As of early 2016, there were more than 42 million records of diagnostic data, 68 million records of prescription data, 22 million records of information pertaining to substance abuse, and 330 million records of mental status examinations.1 The Mental Status Examination (MSE; Koita, Riggio, & Jagoda, 2010; Martin, 1990) is an important clinical assessment in psychiatric practice. It is an essential tool that is used in part for mental health diagnostics. Clinicians assess a patient’s mental state using various methods. Primarily, the patient’s electronic medical record (EMR) history and the clinician’s expert opinion on the patient’s in-clinic behavior during the visit formulate the basis for the diagnosis, but it may be enhanced by conversations between the clinician and the patient and elements of the formal MSE results. For example, the MSE may assess a patient’s memory, recall, and cognitive ability by asking the patient to serially count down from 100 in steps of 7 or the ability of a patient to formulate abstract thoughts by assessing the patient’s ability to interpret proverbs. The result is a clinician note that is a mix of symptoms (what the patient complains about), signs (what the clinician observes about the patient), the patient’s physical appearance, and responses to formal assessments of specific brain functions that are used in the clinic in a nonsystematic way but are borrowed from the full formal MSE. We hypothesize that clinical observations, reports by the patient, and the assessment of brain function, as achieved by using elements of the formal MSE, together may reflect the mental health and/or cognitive status of a patient. We have developed a way to systematically extract these data from each clinic visit, allowing the integration of information across visits and patients over time. For ease of discussion, we will call this system the status assessment (SA) for each clinical visit. Within the MGD, most physicians appear to summarize patients’ clinical assessments as a single component, which includes the formal MSE, ad hoc observations, and symptoms, 1 The anonymization of clinical records is performed on-site, following “Safe Harbor” specifications of the Health Insurance Portability and Accountability Act (1996), with the full consent of the individual hospitals, before it is extracted and put into a common database. These anonymized data are available to all hospitals that also contribute data to the MGD for academic research. Computational Psychiatry 77 Quantification of the Mental State of Psychiatric Patients Mukherjee et al. Figure 1. Boxplots showing the distribution of the percentage of visits that record different psychiatric assessments within the MindLinc Global Database (MGD) over the entire population of patients. It is worth noting that the Clinical Global Impression (CGI), status assessment (SA), and diagnosis are generally recorded for the majority of the visits, while the Global Assessment of Functioning (GAF) and symptom information are infrequently recorded within the MGD. within the SA rather than to record symptom information within structured EHR fields. There- fore, structured symptom information is unavailable for most encounters (median 0%), whereas the SA is available for the majority of the visits (median 67%), along with the diagnosis (median 80%) and the Clinical Global Impression (CGI; Guy, 1976) score (median 60%), as shown in Figure 1. Although the SA allows clinicians to enter information about the mental health of the pa- tient in a very flexible manner, it also promotes the accumulation of unstructured data in place of structured fields. In MGD, for example, the information for SA is present as a (category, sign) tuple. Among the 25 hospitals that contribute data to the MGD, both the categories and Computational Psychiatry 78 Quantification of the Mental State of Psychiatric Patients Mukherjee et al. the signs lack standardization. SA data are categorized into 69 separate categories, many of which represent the same information, and should be regrouped into a smaller set of standard categories. The sign is usually represented as a free-text description. SA is one of the richest