STRUCTURAL VALIDITY of MENTAL DISORDERS 1 Running Head
Total Page:16
File Type:pdf, Size:1020Kb
STRUCTURAL VALIDITY OF MENTAL DISORDERS 1 Running Head: STRUCTURAL VALIDITY OF MENTAL DISORDERS Structural Validity and the Classification of Mental Disorders Robert F. Krueger, Ph.D. Nicholas R. Eaton, M.A. University of Minnesota Published as: Krueger, R. F., & Eaton, N. R. (2012). Structural validity and the classification of mental disorders. In K. S. Kendler & J. Parnas (Eds.), Philosophical issues in psychiatry II: Nosology (pp. 199-212). Oxford, United Kingdom: Oxford University Press. STRUCTURAL VALIDITY OF MENTAL DISORDERS 2 Structural Validity and the Classification of Mental Disorders In conceptualizing mental disorders, reliability and validity are fundamental concepts. Perhaps most frequently, the mental health field focuses on investigations of reliability, by one of several possible routes. Reliability studies often examine a particular assessment instrument—for instance, how similar are two scores on the Beck Depression Inventory when administered one hour apart? In addition to evaluating an assessment instrument’s reliability, one can also examine the reliability of mental disorder diagnoses. For instance, investigators might have several trained clinicians interview each of a number of patients and then examine inter-rater agreement and reliability of their resulting diagnoses. In general, the reliability of mental disorder diagnoses can be thought of as a fraction, with a numerator representing the signal of interest (true score) and a denominator representing the signal plus noise (observed score). As the noise associated with each diagnosis decreases, the numerator and denominator approach equivalence. Thus, when the noise is zero, the two quantities are the same, and the diagnosis (true score) can be made in an error free manner. In that case, the reliability would equal 1.00. As this illustration demonstrates, when one defines and classifies mental disorders based on reliability concerns, the primary goal is to reduce diagnostic noise, typically by reducing measurement error in assessment instruments. Reliability is of greater than purely academic importance. Efforts to maximize diagnostic reliability have had a major impact on the way that mental disorders are classified, and thereby, on the way they are typically conceptualized in clinical settings. We need only look back to the precursors to the current nosology, DSM-IV-TR, to see this is the case. The early DSMs classified psychopathology into somewhat generalized disorders, often defined by a narrative paragraph designed to describe the “flavor” of the disorder in question. Later DSMs switched to STRUCTURAL VALIDITY OF MENTAL DISORDERS 3 providing a more specific list of diagnostic criteria, and diagnostic thresholds (e.g., five of nine diagnostic criteria must be present). Why was this change made? It was made primarily to enhance reliability. Narrative paragraphs produced unreliability in that they failed to maximize signal. The move to clearly delineated criterion sets helped to solve this problem, as it laid out specifically which signal(s) were of interest, which were not, and how much signal needed to be received before a diagnosis should be made. External Validity and its Limitations There is no denying that the changes to the DSMs over time—efforts to raise reliability by including sets of clearly delineated diagnostic criteria—were watershed events in the history of mental disorder nosology. The focus on reliability had a price, however, in that it shifted focus away from deeper questions of validity. In other words, the later DSMs, including DSM-IV-TR, strive to reduce noise. But are we focusing on the correct signals? This question is deep but cannot be avoided. Indeed, there is a growing realization in the mental health field that most of the diagnoses enshrined in the DSMs are probably not true categorical entities in nature. Instead, many appear to index underlying dimensions of severity, and these dimensions seem to cut across current diagnostic constructs rather than following DSM-defined boundaries. Indeed, some have argued that the reification of diagnostic classifications as they stand is a major impediment to research that might eventuate in a valid nosology (Hyman, 2010). As with reliability, there are multiple types of validity that one can consider. The type of validity often invoked to evaluate mental disorder diagnoses is external validity. Researchers taking this approach might demonstrate that a disorder diagnosis is associated with poor functioning, such as undergoing divorce, being severed at the workplace, spending more days STRUCTURAL VALIDITY OF MENTAL DISORDERS 4 hospitalized, and so on. This perspective on validity is unquestionably important, and it highlights the marked impairments associated with mental disorder. However, these external validity approaches do not speak directly to the question of the optimal way to parse psychopathological phenomena. To provide a concrete example: it may be that major depression is associated with more days hospitalized, but it is likely that the presence of panic disorder, borderline personality disorder, schizophrenia, alcohol dependence, and sleep disorders are similarly related to increases in hospitalization, when persons meeting criteria for these disorders are compared with persons who do not meet criteria. Contemporary DSM categories are associated with poorer functioning, but this does not mean that contemporary DSM categories represent the true nature of psychopathological variation. To illustrate this point further, let us consider a particular disorder. DSM-IV-TR includes many culture-bound syndromes, such as hwa-byung, a Korean folk syndrome with symptoms of fatigue, fear of death, insomnia, panic, indigestion, anorexia, palpitations, aches, and dysphoric affect. Thus, hwa-byung seems to cut across the DSM-IV-TR diagnostic categories: It appears to include aspects of mood disturbance as well as panic disorder, sleep disorder, hypochondriasis, and other disorders. Clearly, this disorder does not fit neatly into the categories Western psychiatric nosologies include, and academics in the United States or Europe probably would not argue that hwa-byung represents a fundamental diagnosis akin to major depression or schizophrenia. However, if one were to do an external validity study, it is almost assured that European or North American persons meeting criteria for hwa-byung would be more impaired than a comparison group; indeed, any one of the many diverse symptoms in the hwa-byung category would likely be associated with impairment. While this example might seem strange to STRUCTURAL VALIDITY OF MENTAL DISORDERS 5 some, it is no different from external validity studies used in “support” of the diagnostic validity of any disorder enshrined in the DSM-IV-TR. Structural Validity If external validity is not a sufficient concept for carving psychopathology at its joints, what concept might be better suited to framing this task? The answer may be found in the work of Jane Loevinger, who explicated various conceptions of validity in a seminal 1957 monograph. Loevinger (1957) laid out the concept of structural validity, particularly in the context of psychological assessment. By structural validity, she was referring primarily to parallelism between psychological test item structure (the way test items tend to be correlated with each other) and the structure of extra-test behaviors (the way behaviors outside of the test, that should be correlated with the test, are correlated with each other). We believe that this notion of structural validity can be profitably applied to mental disorder classification and can help clarify how psychopathology might best be conceptualized. Before we proceed further, an illustration of structural validity seems warranted. Consider the idea of a general and continuous dimension underlying intellectual (IQ) test performance—the so-called g factor. This g factor saturates the sub-tests of IQ batteries and leads to a positive manifold, wherein all sub-tests are positively correlated with one another. Indeed, it was the observation of this positive correlation matrix that first gave rise to the notion of a possible general factor of intelligence, g, in the first place. The existence of such a general factor is also inferred from evidence that essentially all reliable intellectual assessment instruments measure the same g (i.e., g estimates are very highly correlated between instruments), albeit to different extents (Johnson et al., 2004). If we apply the notion of structural validity here, we can state that extra-test behaviors should show much the same structure as the STRUCTURAL VALIDITY OF MENTAL DISORDERS 6 test itself. For example, persons who perform well in mathematics classes should tend to perform well in language classes. This would be a reasonable hypothesis given that people who do well on IQ test scales indexing mathematical ability also tend to do well on scales indexing verbal ability. Importantly, these patterns refer to the structure of individual differences, not the structure of abilities or qualities within specific people. Moreover, these patterns are probabilistic. Hence, it is possible to find specific persons who may be harder to fit within the model (e.g., a person with unusually strong mathematical ability but very weak verbal ability). However, to the extent that the model fits observed data and is structurally valid, such persons should be rare relative to persons where both mathematical and verbal talents are found