WARHOL CREDIT HERE by Scott O. Lilienfeld, James M. Wood and Howard N. Garb What’s Wrong with This PICTURE PICTURE? PHOTOGRAPHS BY JELLE WAGAANER PSYCHOLOGISTS OFTEN USE THE FAMOUS RORSCHACH INKBLOT TEST AND RELATED TOOLS TO ASSESS PERSONALITY AND MENTAL ILLNESS. BUT RESEARCH SHOWS THAT INSTRUMENTS ARE FREQUENTLY INEFFECTIVE FOR THOSE PURPOSES www.sciam.com SCIENTIFIC AMERICAN 41 What if you were asked to describe images you saw in an inkblot or to invent a story for an ambiguous illustration— say, of a middle-aged man looking away from a woman who was grabbing his arm? To comply, you would draw on your own emotions, experiences, memories and imagination. You would, in short, project yourself into the images. Once you did that, many practicing psychologists would assert, trained evaluators could mine your musings to reach conclusions about your personality traits, unconscious needs and overall mental health. But how correct would they be? The answer is important Standardization is needed because seemingly trivial differences because psychologists frequently apply such “projective” in- in the way an instrument is administered can affect a person’s struments (presenting people with ambiguous images, words responses. Norms provide a reference point for determining or objects) as components of mental assessments, and because when someone’s responses fall outside an acceptable range. the outcomes can profoundly affect the lives of the respondents. In the 1970s John E. Exner, Jr., then at Long Island Uni- The tools often serve, for instance, as aids in diagnosing men- versity, ostensibly corrected those problems in the early tal illness, in predicting whether convicts are likely to become Rorschach test by introducing what he called the Comprehen- violent after being paroled, in assessing the mental stability of sive System. This set of instructions established detailed rules parents engaged in custody battles, and in discerning whether for delivering the inkblot exam and for interpreting the re- children have been sexually molested. sponses, and it provided norms for children and adults. We recently reviewed a large body of research into how well In spite of the Comprehensive System’s current popularity, the projective methods work, concentrating on three of the most Rorschach generally falls short on two additional, and critical, cri- extensively used and best-studied instruments. Overall our find- teria: scoring reliability and validity. A tool possessing scoring re- ings are unsettling. liability yields similar results regardless of who tabulates and in- terprets the responses. A valid technique measures what it aims to Butterflies or Bison? measure: its results are consistent with those produced by other THE FAMOUS RORSCHACH inkblot test—which asks people to trustworthy instruments or are able to predict behavior, or both. describe what they see in a series of 10 inkblots—is by far the To understand the Rorschach’s scoring reliability problems, most popular of the projective methods, given to hundreds of it helps to know something about how reactions to the inkblots thousands, or perhaps millions, of people every year. The com- are interpreted. First, a psychologist rates the collected reactions ments that follow refer to the modern, rehabilitated version, on more than 100 characteristics, or variables. The evaluator not to the original construction, introduced in the 1920s by records, for instance, whether the person looked at whole blots Swiss psychiatrist Hermann Rorschach. or just parts, notes whether the detected images were unusual The initial tool came under severe attack in the 1950s and or typical of most test takers, and indicates the aspects of the 1960s, in part because it lacked standardized procedures and inky swirls (such as form or color) that most determined what a set of norms (averaged results from the general population). the respondent saw. 42 SCIENTIFIC AMERICAN MAY 2001 Then he or she compiles the findings representative of the U.S. population into a psychological profile of the indi- and mistakenly make many adults and vidual. As part of that interpretive children seem maladjusted. For in- process, psychologists might conclude stance, in a 1999 study of 123 adult that focusing on minor details (such as volunteers at a California blood bank, stray splotches) in the blots, instead of on one in six had scores supposedly in- whole images, signals obsessiveness in a dicative of schizophrenia. patient and that seeing things in the white The inkblot results may be even spaces within the larger blots, instead of more misleading for minorities. Sever- in the inked areas, reveals a negative, al investigations have shown that RORSCHACH TEST contrary streak. scores for African-Americans, Native For the scoring of any variable to be Americans, Native Alaskans, Hispan- Wasted Ink? considered highly reliable, two different ics, and Central and South Americans assessors should be very likely to produce differ markedly from the norms. To- “It looks like two dinosaurs with huge heads similar ratings when examining any giv- gether the collected research raises se- and tiny bodies. They’re moving away from en person’s responses. Recent studies rious doubts about the use of the each other, looking over their shoulders. The demonstrate, however, that strong agree- Rorschach in the psychotherapy office black blob in the middle reminds me of a ment is achieved for only about half the and in the courtroom. spaceship.” characteristics examined by those who score Rorschach responses; evaluators Doubts about TAT Once deemed an “x-ray of the mind,” the might well come up with quite different ANOTHER PROJECTIVE TOOL—the Rorschach inkblot test remains the most ratings for the other variables. Thematic Apperception Test (TAT)— famous—and infamous—projective Equally troubling, analyses of the may be as problematic as the psychological technique. An examiner hands Rorschach’s validity indicate that it is Rorschach. This method asks respon- 10 symmetrical inkblots, one at a time in a set poorly equipped to identify most psychi- dents to formulate a story based on order, to a respondent, who says what each atric conditions—with the notable excep- ambiguous scenes in drawings on blot resembles. A few blots include colored tions of schizophrenia and other distur- cards. Among the 31 cards available to shapes, but most are black and gray—like bances marked by disordered thoughts, psychologists are ones depicting a boy artist Andy Warhol’s rendering above (the such as bipolar disorder (manic-depres- contemplating a violin, a distraught actual blots cannot be published). sion). Despite claims by some Rorschach woman clutching an open door, and Responses to the inkblots purportedly proponents, the method does not consis- the man and woman who were men- reveal aspects of a person’s personality and tently detect depression, anxiety disorders tioned at the start of this article. One mental health. Advocates believe, for or psychopathic personality (a condition card, the epitome of ambiguity, is to- instance, that references to moving characterized by dishonesty, callousness tally blank. animals—such as the dinosaurs mentioned and lack of guilt). The TAT has been called “a clini- above—often indicate impulsiveness, Moreover, although psychologists cian’s delight and a statistician’s night- whereas allusions to a blot’s “blackness”—as frequently administer the Rorschach to mare,” in part because its administra- in a spaceship—often indicate depression. assess propensities toward violence, im- tion usually is not standardized: Swiss psychiatrist Hermann Rorschach pulsiveness and criminal behavior, most different clinicians present different probably got the idea of showing inkblots from research suggests it is not valid for these numbers and selections of cards to re- a European parlor game. The test debuted in purposes either. Similarly, no compelling spondents. Also, most clinicians inter- 1921 and reached high status by 1945. But a evidence supports its use for detecting pret the stories intuitively instead of critical backlash began taking shape in the sexual abuse in children. following a well-tested scoring proce- 1950s, as researchers found that Other problems have surfaced as dure. Indeed, a recent survey of nearly psychologists often interpreted the same well. Some evidence suggests that the 100 North American psychologists responses differently and that particular Rorschach norms meant to distinguish practicing in juvenile and family courts responses did not correlate well with specific mental health from mental illness are un- found that only 3 percent relied on a mental illnesses or personality traits. Today the methodological response to SCOTT O. LILIENFELD, JAMES M. WOOD and HOWARD N. GARB all conduct research on those weaknesses—the Comprehensive psychological assessment tools and recently collaborated on an extensive review of re- System (CS)—is used widely to score and search into projective instruments that was published by the American Psychological So- interpret Rorschach responses. But it has ciety (see “More to Explore,” on page 47). Lilienfeld and Wood are associate professors in been criticized on similar grounds. Moreover, the departments of psychology at Emory University and the University of Texas at El Paso, respectively. Garb is a clinical psychologist at the Pittsburgh Veterans Administra- several recent findings indicate that the THE AUTHORS tion Health Care system and the University of Pittsburgh and author of the book