This article was downloaded by: [University of Toledo] On: 14 April 2009 Access details: Access Details: [subscription number 908825564] Publisher Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Journal of Personality Assessment Publication details, including instructions for authors and subscription information: http://www.informaworld.com/smpp/title~content=t775653663

An Examination of the Construct Validity of the Rorschach Mutuality of Autonomy (MOA) Scale George Bombel a; Joni L. Mihura a; Gregory J. Meyer a a Department of Psychology, University of Toledo,

Online Publication Date: 01 May 2009

To cite this Article Bombel, George, Mihura, Joni L. and Meyer, Gregory J.(2009)'An Examination of the Construct Validity of the Rorschach Mutuality of Autonomy (MOA) Scale',Journal of Personality Assessment,91:3,227 — 237 To link to this Article: DOI: 10.1080/00223890902794267 URL: http://dx.doi.org/10.1080/00223890902794267

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf

This article may be used for research, teaching and private study purposes. Any substantial or systematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material. Journal of Personality Assessment, 91(3), 227–237, 2009 Copyright C Taylor & Francis Group, LLC ISSN: 0022-3891 print / 1532-7752 online DOI: 10.1080/00223890902794267

ARTICLES

An Examination of the Construct Validity of the Rorschach Mutuality of Autonomy (MOA) Scale

GEORGE BOMBEL,JONI L. MIHURA, AND GREGORY J. MEYER

Department of Psychology, University of Toledo

Using 100 clinical cases, we examined the construct validity of the Mutuality of Autonomy (MOA) Scale (Urist, 1977) using Westen and Rosenthal’s (2003) rcontrast - construct validity (CV) procedure for quantifying a pattern of convergent-discriminant relationships between a target measure and a set of criterion variables. Our 15 criterion variables included the Comprehensive System (CS; Exner, 2003) variables, a CS-based measure of ego strength (Resnick, 1994), and 3 subscales from the Social Cognition and Object Relations Scale (Westen, Lohr, Silk, Kerber, & Goodrich, 1990). We generated the rcontrast - CV coefficients to test 2 competing hypotheses: that the MOA Scale primarily measures object relations (OR) quality or that it primarily measures psychopathology. Results suggest that the MOA Scale is an equally potent measure of OR and psychopathology regardless of the MOA Scale index used.

Object relations (OR) constructs are important subjects of study mutually autonomous within relationships” (Urist, 1977, p. 4). in personality assessment research. Literature reviews dedicated Synopses of the scoring guidelines, for the healthiest to most to the subject have been published routinely (Fishler, Sperling, primitive levels, are the following: & Carr, 1990; Frank, 1995; Huprich & Greenberg, 2003; T. E. Smith, 1993; Stricker & Healey, 1990). One of the most popu- lar OR assessment instruments is the Rorschach Inkblot Method (Rorschach, 1921/1942), for which a number of OR scoring sys- 1. Figures are described as engaging in an activity or rela- tems have been developed. Some of these systems are rarely used tionship that conveys a sense of mutuality, with a reciprocal today (Coonerty, 1986; Kwawer, 1979; Mayman, 1967; Pruitt & acknowledgment of their respective individuality (e.g., “Two Spilka, 1964). Others continue to make appearances in the liter- griffons building a nest together.”). ature (Blatt, Brenneis, Schimek, & Glick, 1976; Burke, Fried- 2. Figures are described as engaging in a relationship or par- man, & Gorlitz, 1988; Cooper, Perry, Hoke, & Richman, 1985). allel activity without emphasis on reciprocity or mutuality, The scoring system by Urist (1977; Mutuality of Autonomy although the description does not necessarily need to com- [MOA] Scale) is one of the most well-known among Rorschach promise mutuality (e.g., “Two bears climbing up different OR assessment methods (Huprich & Greenberg, 2003; Stricker sides of the same mountain.”). Downloaded By: [University of Toledo] At: 00:58 14 April 2009 & Healey, 1990). The MOA Scale is an especially attractive 3. Figures are described as engaging in an activity in which measure because it is relatively simple to use; for example, they require each other, or one requires the other, that is, an scorers who are relatively unfamiliar with OR theory can score external source of support or direction is required (e.g., “A it reliably (Holaday & Sparks, 2001). Furthermore, unlike other clown hanging from a cliff.”). measures, Rorschach responses with inanimate, nonhuman-like 4. The described relationship conveys a sense that the defini- figures are scorable with the MOA Scale. tion or stability of one object necessarily requires the other Urist (1977) developed the MOA Scale as a measure of ob- because it is merely an extension or reflection of the other ject relational developmental maturity or the degree to which (e.g., “A woman scrutinizing her reflection in the mirror.”). individuation separation has been attained. The seven points, 5. The described relationship themes are characterized by se- or levels, of this scale represent specific developmental levels vere imbalances of mutuality in which figures are powerless along a dimension from empathic, reciprocal relatedness to de- and helpless versus omnipotent and controlling. Themes of structive envelopment and symbiotic fusion. Each level, more influence and controlling are present (e.g., “A sorcerer cast- specifically, corresponds to “developmentally significant grada- ing a spell on someone.”). tions in the individual’s capacity to experience self and other as 6. The figure(s) are described as engaging in activity that is clearly destructive or parasitic, and that compromises the autonomy or integrity of the victim (e.g., “A giant squashing Received July 19, 2007; Revised September 21, 2008. a dwarf.”). Editor’s Note: Radhika Krishnamurthy served as the Editor with full decision 7. The described relationship theme is envelopment and engulf- authority over this manuscript. ment in which one figure has devoured or consumed another, Address correspondence to George Bombel, UTHSCSA, Department of Psy- either explicitly or implicitly (e.g., “It’s an X-ray of a creature chiatry, 7703 Floyd Curl Drive, MC 7792, San Antonio, TX 78229–3900; Email: [email protected] that’s eaten another creature.”). 227 228 BOMBEL, MIHURA, MEYER

Importantly, Urist (Urist & Shill, 1982) made a distinction haviors, and/or self-reports of violence, marital problems, and between the phenomena he was attempting to measure—OR relationship functioning. development—and “a more general Rorschach health-sickness At the same time, some data suggest that the MOA Scale does factor” (p. 453). Indeed, in Urist and Shill’s second MOA not cleanly measure what it purports to measure. Tuber (1992) Scale publication, they took explicit methodological steps to noted that the scale does not correlate with chronological age, ensure that the latter would not influence raters’ efforts to which means that its scoring levels do not represent points on record the former in participants’ Rorschach responses. The- an OR developmental timeline but simply object representa- oretically, this distinction is a valid one. Object relational tions of varying qualities. Furthermore, the scale may also be a development is the maturational process of separation individ- potent measure of psychopathology. Harder, Greenwald, Wech- uation that can lead to healthy outcomes (Masterson, 1976). sler, and Ritzler (1984) found that higher (more pathological) “Health-sickness,” or psychopathology, does not refer to a de- MOA mean scores were significantly associated with diagno- velopmental concept or personality organization but rather to sis severity trichotomized into the categories of nonpsychotic, the Diagnostic and Statistical Manual of Mental Disorders (4th affective psychotic, and schizophrenic spectrum disorders (r = ed., text revision [DSM–IV–TR]; American Psychological As- .43, p<.001). sociation, 2000) Axis I psychopathology (e.g., dysphoria, pho- In a thorough study with long-term psychiatric inpatients, bic avoidance, thought disorder). The constructs, therefore, are Blatt et al. (1990) investigated the relationships between sev- not synonymous. Still, when developmental outcomes are un- eral MOA Scale scores (e.g., the mean, single healthiest score, healthy (e.g., borderline personality organization), vulnerabil- and the single most pathological score) and (a) estimates of so- ity to developing and expressing psychopathology presumably cial behavior and interpersonal relations derived from clinical increases (Berg, Packer, & Nunno, 1993). In these cases, ob- records, (b) neurotic and psychotic symptomatology as recorded ject representations of poor quality are likely to co-occur with in clinical records, and (c) Rorschach measures of reality test- symptomatology. ing (F + %) and thought disorder (e.g., severity of boundary The MOA Scale has respectable psychometric qualities. How- disturbance) developed earlier by Blatt and Ritzler (1974). Sur- ever, Urist’s (1977) initial interrater reliability values were not prisingly, although most adaptive scores related to a measure of impressive: Raters achieved exact agreement at the response interpersonal communication, MOA Scale mean scores corre- level only 52% of the time and were within 1 point of each lated significantly with symptom severity (r = .35, p<.0001), other 86% of the time. Holaday and Sparks (2001) calculated reality testing (r =−.41, p<.0001), and thought disorder the unweighted response-level exact agreement average in the (r = .49, p<.0001) but not with ratings of social behavior literature to be 74.3%. After expanding the MOA Scale scor- and interpersonal relations on the ward. MOA Scale indexes of ing guidelines, Holaday and Sparks obtained an exact agree- pathological object representations (i.e., the sum of responses ment rate of 80% at the response level using raters na¨ıve to scored 5, 6, or 7) evidenced a similar pattern of significant OR theory. Results from a recent meta-analysis of MOA Scale and nonsignificant results. Blatt et al. (1990) concluded that the interrater scoring reliability (Bombel, 2006) found weighted MOA Scale appears to measure psychopathology primarily and kappa and percent agreement coefficients of .83 and .81, respec- interpersonal relationship quality secondarily. tively, when these statistics were computed across individual Berg et al. (1993) investigated the relationship between responses, whereas intraclass correlations (ICCs) and Pearson’s thought disorder and object representation in psychiatric outpa- r s were .94 and .91, respectively, when computed at the proto- tients diagnosed with borderline personality disorder, narcissis- col level, that is, when raters were compared across aggregated tic personality disorder, and schizophrenia. A strong relationship protocol scores. These data indicate that the MOA Scale can be was found between MOA Scale pathology scores (sum of Levels reliably scored. 5, 6, and 7) and thought disorder (r = .58, p<.001) as measured Downloaded By: [University of Toledo] At: 00:58 14 April 2009 The MOA Scale has also demonstrated validity as a mea- by the Comprehensive System (CS; Exner, 1986) weighted sum sure of OR quality. In Urist’s (1977) initial explication of the of the six cognitive special scores (WSum6) variable. Further, measure, MOA Scale scores correlated significantly with (a) in- whereas the frequencies of MOA Scale Level 4 and 5 scores patients’ autobiographical material scored for object relational differentiated the patients with personality disorders from those themes (r = .63, p<.001) and (b) staff ratings of the quality with schizophrenia, the pathology composite did not success- of the inpatients’ relationships with hospital staff and other in- fully discriminate among the three patient groups. These data patients (r = .43, p<.001). Among an inpatient sample, Blatt, suggest that the MOA Scale is not making finer distinctions Tuber, and Auerbach (1990) found significant correlations (r = between poor object representations and thought disturbances .26, p<.001) between MOA Scale scores and the Concept of when both are present. the Object Scale (Blatt et al., 1976), which assigns OR scores to The results from these studies highlight concerns that have thematic and structural aspects of Rorschach responses. Ryan, been raised in literature reviews about the construct validity Avery, and Grolnick (1985), in an investigation of the conver- of the MOA Scale (Stricker & Healey, 1990; Tuber, 1992). It gent and discriminant validity of the MOA Scale, found that appears to be sensitive to relationship quality and social func- healthier (i.e., lower) MOA Scale responses were significantly tioning; however, it may be no less sensitive to severity of Axis related to social adjustment (r =−.24, p<.10), self-esteem (r I psychopathology. If this is the case, MOA Scale scores are =−.26, p<.05), and interpersonal cooperation (r =−.33, p< confounded and difficult to interpret. Given the popularity of .05) but not to teacher ratings of achievement and intelligence. the scale, a construct validity investigation exploring this issue Finally, a recent meta-analysis (Fowler, Addelson, & Clemence, is warranted. 2006) of the validity of the MOA Scale yielded small to medium Cronbach and Meehl (1955) noted that one could estimate the effect sizes (rs = .20–.34) across several indexes when criteria construct validity of a measure by providing evidence to show were correlates of OR phenomena, such as observer-rated be- that it interrelates with measures of other constructs in ways that MOA SCALE CONSTRUCT VALIDITY 229

would be expected if the underlying theory is true. Recently, the total number of scorable MOA Scale responses. This score Westen and Rosenthal (2003) developed a procedure to quan- is a ratio of developmentally healthy to primitive scores relative tify the pattern of relationships in this conceptual framework. to the number of MOA responses produced. Thus, it provides Based on analysis of variance contrast analysis, it involves gen- a more sophisticated summary of MOA Scale information than erating a set of predicted relationships (i.e., a pattern of contrast the other indexes. weights) between the target measure and the specified criterion variables and then calculating the degree to which this predic- tion matches the actual relationship pattern. In the true spirit of METHOD contrast analysis, the predicted relationship pattern between the Participants measure and the criterion variables constitutes the hypothesis to The participants in this study were drawn from a large archive be tested. The resulting effect size, rcontrast - construct validity (CV),is (N = 440) of individuals who underwent psychological assess- a Pearson’s product–moment correlation coefficient that is read- ment through a testing service associated with a large Midwest- ily interpretable and readily aggregated in a future meta-analysis ern university hospital. To be considered, participants’ case files (Westen & Rosenthal, 2003). needed to include a Rorschach protocol with between 14 and 50 To investigate the construct validity of the MOA Scale, responses (R). Limits were placed on R to help ensure results Westen and Rosenthal’s (2003) procedure was used in this would not be negatively affected by unreliably brief or exces- study to test competing hypotheses about what the scale ac- sively lengthy protocols (Dean, Viglione, Perry, & Meyer, 2007; tually measures. That is, Westen and Rosenthal’s procedure was Exner, 2003; Meyer, 1992). Also, case files needed to include conducted in two different circumstances resulting in two sepa- Thematic Apperception Test (TAT; H. A. Murray, 1943) stories rate rcontrast - CV values—the first assuming that the MOA Scale for cards 1, 2, 3BM, 4, 10, and 13MF. We chose the first 100 is a measure of OR and the second assuming the MOA Scale is case files meeting these criteria, starting from the most recently a measure of psychopathology. Therefore, first, we created a set created, for analyses here. The average number of Rorschach of predicted correlations between the MOA Scale and our crite- responses in the final sample was 22.71 (SD = 8.15). This group rion variables assuming that the scale is primarily a measure of was comprised of psychiatric inpatients (N = 30), psychiatric the quality of OR. For example, the MOA Scale should correlate outpatients (N = 36), general medical inpatients and outpatients more strongly with other measures of OR and less strongly with evaluated for psychiatric reasons (N = 33), and 1 incarcerated measures of thought disorder and Axis I symptom severity. We individual evaluated for psychiatric reasons. More males (N = then used the rcontrast - CV procedure to generate an effect size 58) than females (N = 42) were represented, and age of the par- representing the degree to which this MOA Scale as OR mea- ticipants ranged from 15 to 71 years (M = 34.5, SD = 12.99). sure prediction set matched the actual pattern of relationships Years of education ranged from 9 to 21 (M = 13.74, SD = between the MOA Scale and the 14 criterion variables chosen 2.55). At the time of testing, 28 of the individuals were mar- for this study. Next, we created a set of predicted correlations ried, 10 were divorced, 3 were widowed, and 59 had never been between the MOA Scale and the criterion variables assuming married. that the scale is primarily a measure of Axis I symptom sever- ity. This time, the rcontrast - CV procedure generated an effect size representing the degree to which this MOA Scale as pathology Measures measure prediction set matched the actual pattern of relation- The criterion variables used in this study were as follows: ships between the MOA Scale and the criterion variables. The competing prediction sets yield the following expectations: (a) 1. The Complexity of Human Representations (CHR) scale If the MOA Scale is primarily an OR measure, the rcontrast - CV ef- from the Social Cognition and Object Relations Scale Downloaded By: [University of Toledo] At: 00:58 14 April 2009 fect size for the MOA Scale as OR measure prediction set should (SCORS; Westen, Lohr, Silk, Kerber, & Goodrich, 1990). be larger than the rcontrast - CV effect size for the MOA Scale as 2. The Affect Tone (AT) scale from the SCORS. pathology measure prediction set; (b) contrariwise, if the MOA 3. The Emotional Investment in Human Relationships (EIR) Scale is primarily a measure of psychopathology severity, the scale from the SCORS-Global Method scale (SCORS-G; rcontrast - CV effect size for the MOA Scale as pathology measure Hilsenroth et al., 2004). prediction set should be larger than the rcontrast - CV effect size for the MOA Scale as OR measure prediction set. and the following variables from the CS (Exner, 2003): We tested this set of competing hypotheses four times in our study, once for each of our MOA Scale score indexes. The first 4. Good Human Representation (GHR). three are (a) MOA Scale mean score (MOAx), which is re- 5. Poor Human Representation (PHR). ported frequently in the literature (Stricker & Healey, 1990); (b) 6. Human Representational Variable (HRV). the single best or most adaptive score (MOAb); and (c) the sin- 7. X – %. gle worst or most pathological score (MOAp). These latter two 8. WDA%. scores have also been reported very frequently in the literature 9. WSum6. (e.g., Ackerman, Hilsenroth, Clemence, Weatherill, & Fowler, 10. Perceptual Thinking Index (PTI). 2000; Blais, Hilsenroth, Fowler, & Conboy, 1999; Cook, Blatt, 11. Suicide Constellation (S-CON). & Ford, 1995; Strauss & Ryan, 1987; Tuber, 1983). The final 12. Depression Index (DEPI). measure was a refined variable that makes use of the full spec- 13. Coping Deficit Index (CDI). trum of MOA scores defined here as (d) the relative Health index 14. A composite scale derived from CS scores called the Con- (MOAHI) and computed as the number of healthy Level 1 scores ceptual Ego Strength Index (CESI; Resnick, 1994; Resnick minus the most disturbed Level 5, 6, and 7 scores divided by & Meyer, 1995). 230 BOMBEL, MIHURA, MEYER

These variables were included because we hypothesized that sive, or malevolent representation or perceptions. Along these broad they would relate to the target constructs (OR or psychopathol- and heterogeneous dimensions, the HRV summarizes the overall qual- ogy) in varying degrees, and the rcontrast - CV procedure is most ity of human interpersonal perceptions and representations, that is, the sensitive to levels of agreement between the predicted and ob- implicit understanding of people and relationships. (p. 71) served correlations when the correlations include a wide range of values (Westen & Rosenthal, 2003). Further, we made an Research using the newer HRV measures has been limited to effort to include criterion variables that assess aspects of psy- just a few studies. However, Viglione et al. (2003) found sub- chopathology associated with both the more neurotic (e.g., stantial correlations between the HEV measures and their HRV DEPI) and more psychotic (e.g., WSum6) segments of the psy- counterparts (all rs ≥ .87), and the HEV has demonstrated con- choanalytic personality organization spectrum (Kernberg, 1986; struct validity. For example, Burns and Viglione (1996) found McWilliams, 1994). that HEV distinguished between high and low interpersonal re- latedness groups as identified with the Bell Object Relations SCORS. The SCORS, and its briefer version, the SCORS– Inventory (BORI; Bell, 1991), a version of the BORI modified G, are eight scale scoring systems designed to broadly assess to be a spouse-report instrument, and the Emotional Maturity affective processes, social information processing, and inter- Rating Form (Bessell, 1984). nalized representations of self and others. Interrater reliability The X – % variable represents the proportion of Rorschach re- of the SCORS has been well documented when used to rate sponses in which the individual disregarded the contours of the TAT stories (Ackerman et al., 2000; Fowler et al., 2004), early inkblots when formulating percepts. Exner’s (1993) reference memories (Fowler, Hilsenroth, & Handler, 1995), and narrative group of individuals with schizophrenia had higher X – % scores statements by patients during psychotherapy sessions (Peters, than the nonpatient normative group. Similarly, Dao and Pre- Hilsenroth, Eudell-Simmons, Blagys, & Handler, 2006). Fur- vatt (2006) found that depressed individuals without psychosis ther, the SCORS has demonstrated convergent and construct scored significantly lower (healthier) on the X – % variable than validity by relating significantly to other OR measures such as did individuals with a schizophrenia-spectrum disorder. WDA% the MOA Scale (Ackerman, Hilsenroth, Clemence, Weatherill, represents the extent to which a respondent is using common blot & Fowler, 2001) and the Concept of the Object Scale (Hibbard, contours in conventional ways. In the aforementioned study by Hilsenroth, Hibbard, & Nash, 1995). Dao and Prevatt, the depressed group scored significantly higher In this study, we used the CHR and ATscales from the SCORS (more conventional) than the schizophrenia-spectrum group. and the EIR scale from the SCORS–G as criterion variables be- The WSum6 indicates the presence of thinking disturbance cause they were expected to converge with the MOA Scale or a thought disorder (Exner, 2003). In Dao and Prevatt’s (2006) to differing degrees. The CHR scale assesses the capacity for sample, the schizophrenia-spectrum group exhibited signifi- seeing self and others as separate individuals with enduring dis- cantly higher WSum6 scores than did the depressed group. Fur- positions, complex motives, and unique subjective experiences. ther, it has differentiated children and adolescents with disturbed The AT scale measures the emotional tone of what an indi- thinking and behavior from those without such problems (S. R. vidual expects from others and the world, ranging from pain Smith, Baity, Knowles, & Hilsenroth, 2001). and malevolence to positive, enriching, and happy experiences. The S-CON is an actuarial index that helps identify individu- The EIR scale assesses the quality of relationship representa- als who may be at risk for suicide (Exner, 2003). Fowler, Piers, tions, ranging from profound selfishness to mutuality, respect, Hilsenroth, Holdwick, and Padawer (2001) found that S-CON and interdependence. We used Westen et al.’s (1990) 5-point scores ≥ 7 predicted near-lethal suicide attempts among a mixed CHR and AT scales here because they provide ample guidelines group of psychiatric patients and successfully discriminated be- for rater training. The EIR scale from the SCORS–G was cho- tween such attempts and parasuicidal behavior. Downloaded By: [University of Toledo] At: 00:58 14 April 2009 sen instead of the Emotional Investment in Relationships and Exner (2003) suggested that the DEPI is sensitive to some Moral Standards scale from the SCORS because it focuses more aspects of depression, although not necessarily a DSM ma- on relational phenomena. Typically, final ratings for each scale jor depressive illness. Still, in a review and meta-analysis by are generated by averaging scale scores across individual TAT Jørgenson, Andersen, and Dam (2000), the DEPI was more able stories (or early memories, etc.). to identify “true positives” when the diagnoses were nonpsy- chotic and unipolar depression than when target diagnoses were Rorschach CS. The Rorschach criterion variables used in psychotic and bipolar depression and when depression was as- this study are in the CS (Exner, 2003) or derived from its scores. sociated with borderline personality disorder. Jørgenson et al. The original Human Experience Variable (HEV), which in- noted that the DEPI may be more sensitive to unambiguous cluded the Good Human Experience (GHE) and Poor Human forms of depression; however, its overall effect size as a diag- Experience (PHE) variables, was recently modified to create the nostic measure of depression appears to be small. psychometrically superior trio of variables mentioned earlier: The CDI is comprised of a heterogeneous mix of variables the HRV, GHR, and PHR (Viglione, Perry, Jansak, Meyer, & related to coping resources, capacity for directing responses to Exner, 2003). Viglione et al. (2003) described the new variables environmental challenges, and interpersonal difficulties (Exner, as follows: 2003). Higher scorers tend to be socially inept or helpless and have impoverished and unfulfilling interpersonal relation- GHR responses are perceptions or representations of positive schema ships. Exner (2003) described an unpublished study wherein of self, other, and relationships manifested in accurate, realistic, logical, CDI scores differentiated outpatients who reported interpersonal intact, human responses and benign or cooperative interactions. PHR complaints from those that did not. are negative and problematic perceptions or representations as man- Resnick (1994) developed the CESI to measure the ego psy- ifested in distorted, unrealistic, damaged, confused, illogical, aggres- chological construct of ego strength. It is based on Kleiger’s MOA SCALE CONSTRUCT VALIDITY 231

(1992) notion that the EA variable from the CS can be inter- TABLE 1.—Interrater reliability coefficients and descriptive statistics for the preted as an indicator of ego-mediated coping resources, in- MOA Scale indexes and criterion variables. cluding the capacity to endure stress, anxiety, and other neg- Measure ICCa Mean SD Minimum Maximum ative affect states. The CESI is computed from CS scores for Form dominance, Form quality, cognitive Special Scores, and MOAx .91 2.87 0.78 1.00 5.00 primitive Content. Meyer and Resnick (1996; Resnick & Meyer, MOAb .75 1.57 0.67 1.00 4.00 1995) have found that even though the CESI has a strong cor- MOAp .88 4.62 1.59 1.00 7.00 − − relation (r = .72) with the Ego Impairment Index (EII; Perry MOAHI .78 0.05 0.25 0.71 1.00 CHR .59 2.94 0.60 2.00 4.67 & Viglione, 1991), it slightly outperformed the EII by corre- AT .71 2.92 0.49 1.50 4.08 lating more strongly with clinician-assigned DSM diagnostic EIR .69 3.38 0.60 1.83 5.67 codes along a theoretical continuum of ego impairment (rs = GHR .92 3.33 1.96 0.00 9.00 .40 vs. .35). It also correlated more strongly with the criterion PHR .96 4.55 4.04 0.00 18.00 HRV .90 −1.22 4.34 −15.00 7.00 than the Ego-Strength Scale (Barron, 1953) and Goldberg Index X-% .87 0.21 0.12 0.00 0.56 (Goldberg, 1965) from the Minnesota Multiphasic Personality WDA% .91 0.81 0.12 0.50 1.00 Inventory–2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaem- WSum6 .81 23.13 25.84 0.00 162.00 mer, 1989; rs of –.25 and .17, respectively). PTI .78 1.30 1.46 0.00 5.00 Importantly, there are no specific scoring confounds between S-CON .76 5.82 1.81 1.00 10.00 DEPI .83 4.53 1.37 1.00 7.00 the Rorschach criterion variables used here and the MOA Scale. CDI .89 2.96 1.31 0.00 5.00 All of the Rorschach criterion variables are derived primarily CESI .86 0.53 0.21 0.14 1.32 from structural, or quantitative, aspects of responses across pro- tocols. For example, although GHR, PHR, the CDI, and the Note. Descriptive data based on N = 100. MOA = Mutuality of Autonomy Scale; ICC = intraclass correlation: 2-way random effects model absolute agreement ICC (A,1; CESI are sensitive in varying degrees to content themes of ag- McGraw & Wong, 1996) computed at the summary-score level; MOAx = MOA Scale gression and/or cooperation (like the MOA Scale), they are mean; MOAb = MOA Scale best score index; MOAp = MOA Scale Pathology index; MOAHI = MOA Scale Health index; SCORS = Social Cognition and Object Relations calculated largely from specific structural summary elements Scale; CHR = Complexity of Human Representation (CHR) SCORS scale; AT = Affect (e.g., FQ, Special Scores, noninterpersonal theme Content) in Tone (AT) SCORS scale; EIR = Emotional Investment in Human Relationships (EIR) the protocol (Exner, 2003). In contrast, MOA Scale scores are SCORS scale; PTI = Perceptual Thinking index; S-CON = Suicide Constellation; DEPI = Depression index; CDI = Coping Deficit index; CESI = Conceptual Ego Strength index. assigned based only on molar, interpersonally oriented thematic a CHR, AT, and EIR based on N = 20; all other variables based on N = 23. elements; their values do not represent formal aspects of per- cepts but rather OR developmental levels (Urist, 1977). ticipants’ scores on other SCORS scales, MOA Scale scores, and patient diagnosis. For each participant, final scores for each Procedures scale were simple averages across the six stories. Another doc- Scoring and reliability. All Rorschach tests in the original toral student in clinical psychology randomly selected and in- database were administered and scored by G. J. Meyer or one dependently scored 20 protocols for the three SCORS scales. of several clinical psychology doctoral students in training with G. Bombel and this student trained to the “gold standard” story him who had received course work and supervised training in examples in the SCORS and SCORS–G manuals. Interrater re- the CS before data collection began. Protocols were scored with liability ICCs (A,1) were computed on the final scores for each the CS at the time of testing. To determine interrater reliability, scale and were in the fair to good range (Cicchetti, 1994; see a batch of randomly selected protocols (N = 23) was indepen- Table 1). dently scored by another doctoral student in clinical psychology. Downloaded By: [University of Toledo] At: 00:58 14 April 2009 Two-way, random effects, absolute agreement ICCs (A,1; Mc- The rcontrast - CV procedure. In this study, the initial step in Graw & Wong, 1996) were excellent (see Table 1). Westen and Rosenthal’s (2003) procedure was to create pre- Holaday and Sparks (2001) developed a set of comprehen- dicted correlations between the MOA Scale and the criterion sive scoring guidelines for the MOA that largely follow Urist’s variables. Each of us first read brief descriptions of the MOA (1977) initial article but expand on scoring guidelines and ex- Scale, the criterion variables, and the concepts of psychopathol- amples for each scoring level. Holaday and Sparks permitted ogy and object relations theory. Next, we independently pre- reliable scoring even among raters na¨ıve to OR theory. Using dicted what the magnitude and direction of the observed corre- these guidelines, G. Bombel scored all Rorschach protocols and lation might be between the MOA Scale and each of the crite- calculated the four index scores (MOAx, MOAb, MOAp, and rion variables while taking into account method variance. We MOAHI). The MOA Scale was scored blind to the criterion did this twice. First, we made ratings assuming that the MOA variables in this study (i.e., the SCORS and other Rorschach Scale is primarily a measure of OR. For example, a moder- variables) and patient diagnosis. To determine interrater relia- ate negative correlation might be predicted between the MOA bility for the MOA Scale, we randomly selected 20 protocols Scale and CHR because they would appear to measure similar and J. L. Milhura independently scored them. Agreement was constructs given our assumption about the MOA Scale. The pre- excellent (κ = .94) regarding whether a Rorschach response dicted correlation would be negative because lower MOA Scale was scorable with the MOA Scale (Cicchetti, 1994). Interrater scores, and higher CHR scores, represent healthier representa- reliability ICCs (A,1) for the MOA Scale indexes were also in tions. We obtained the final MOA Scale as OR prediction set the excellent range (see Table 1). of correlations by averaging the predicted correlations across G. Bombel scored all TAT protocols (cards 1, 2, 3BM, 4, 10, the three raters for each criterion variable. The average mea- and 13MF) with the CHR, AT, and EIR scales. When scoring sures ICC (A,3; McGraw & Wong, 1996) among us was .96. with a particular SCORS scale, G. Bombel was unaware of par- Subsequently, we repeated these steps assuming that the MOA 232 BOMBEL, MIHURA, MEYER

Scale is primarily a measure of psychopathology. We obtained TABLE 2.—Predicted and observed correlations between the MOA Scale and the final MOA Scale as pathology prediction set of correlations criterion variables. as before, and agreement among the raters was again excellent Predicted Correlationsa Observed Correlations (ICC [A,3] = .94; Cicchetti, 1994). Criterion Next, for the MOA Scale as OR prediction set, we generated Variable OR Pathology MOAxb MOAbb MOApb MOAHIc exact lambda weights by subtracting the mean of the predicted ∗∗ ∗∗ ∗∗ correlations from each individual correlation and then squaring CHRc −.43 −.15 .32 .08 .35 −.27 c − − − − ∗ and summing these values. We obtained actual correlations be- AT .32 .20 .05 .18 .21 .12 EIRc −.40 −.15 .02 −.06 .04 .06 tween one of the MOA scales and the criterion variables, trans- GHRc −.38 −.22 −.13 −.33∗∗ .12 .17 formed them into Fisher’s Z scores, and then multiplied them PHRb .46 .30 .28∗∗ −.15 .42∗∗ −.33∗∗ ∗∗ ∗∗ ∗∗ by the squared lambda weights for
the predicted correlations. HRVc −.33 −.22 −.32 −.01 −.34 .38 We used the sum of this column ( λZ ), along with the me- X-%b .15 .32 .10 .10 .04 −.16 r c − − − − dian intercorrelation (r ) among the criterion variables, and the WDA% .17 .30 .09 .13 .00 .11 x WSum6b .23 .45 .25∗ −.19 .33∗∗ −.19 average squared observed correlation to compute Zcontrast(see PTIb .18 .45 .19 −.04 .25∗ −.21∗ Westen & Rosenthal, 2003, for more detailed procedures). We S-CONb .15 .43 .32∗∗ −.02 .29∗∗ −.33∗∗ ∗ ∗ ∗∗ then obtained the p and t values associated with the Zcontrast with DEPIb .15 .42 .21 .12 .20 −.32 the Distribution function in SPSS (SPSS Inc., Chicago, IL), and CDIb .18 .37 .05 .34∗∗ −.22∗ −.09 b ∗∗ − ∗∗ − ∗ we computed rcontrast - CV as the square root of the squared tscore CESI .18 .45 .30 .13 .26 .25 divided by the sum of the squared tscore and the degrees of free- Note. All analyses, N = 100. MOA = Mutuality of Autonomy Scale; OR = object dom. We subsequently repeated the procedure using the MOA relations: predicted correlations for the MOA Scale as an OR measure; Pathology = Scale as psychopathology severity prediction set. We then re- predicted correlations for the MOA Scale as a measure of psychopathology; MOAx = peated the whole process for each of the MOA Scale indexes. MOA Scale mean; MOAb = MOA Scale best score index; MOAp = Moa Scale pathology index; MOAHI = MOA Scale Health index; SCORS = Social Cognition and Object Thus, for each MOA index, we generated two rcontrast - CV ef- Relations Scale; CHR = Complexity of Human Representation SCORS scale; AT = Affect fect sizes, one assuming that the MOA Scale measures OR Tone SCORS scale; EIR = Emotional Investment in Human Relationships SCORS scale; PTI = Perceptual Thinking index; S-CON = Suicide Constellation; DEPI = Depression and one assuming that it measures psychopathology severity. index; CDI = Coping Deficit index; CESI = Conceptual Ego Strength index. Subsequently, we tested the differences between the OR and a Signs of these correlations should be reversed when considering the MOAHI, which psychopathology effect sizes for each index for significance is reverse scored relative to the other MOA scales. bLower scores are healthier. cHigher scores are healthier. using a procedure for comparing two sets of contrasts (i.e., the- *p<.05, two-tailed. ∗∗p<.01, two-tailed. ories; Rosenthal, Rosnow, & Rubin, 2000). Finally, to identify which index was the better measure of OR and which was the better measure of psychopathology, we examined rcontrast - CV not overlap the MOAHI and MOAx CIs. This suggests that these effect sizes and their 95% confidence intervals across indexes latter indexes are more sensitive as overall OR and pathology for each prediction set. Procedures have not been developed to measures√ than MOAb is. CIs spanned about .30 points (Zr ± compare such effect sizes statistically (R. Rosenthal, personal 1.96/ N − 3), regardless of prediction set, and all other CIs communication, April 13, 2008). overlapped. However, overlapping CIs do not necessarily pre- vent mean differences from being significant (Wolfe & Hanley, 2002). As noted, the difference between MOAx OR and MOAx RESULTS pathology rcontrast - CV effect sizes was significant (p = .03) de- Table 1 presents descriptive statistics for the MOA Scale in- spite an absolute value of .07 and overlapping CIs. MOAHI OR dexes and criterion variables. The distribution of the WSum6 and pathology effect sizes differed from their MOAx and MOAp Downloaded By: [University of Toledo] At: 00:58 14 April 2009 variable was skewed (2.72) and kurtotic (9.96) because 1 partic- counterparts by .10 to .18 points, whereas MOAp effect sizes ipant had an extremely high value; it was therefore transformed were about .30 points higher than corresponding MOAb effect with a square root function (revised M = 4.22; SD = 2.32; skew sizes. = 0.913; kurtosis = 1.55). The observed correlations between A surprising finding was that the effect sizes for the MOAb the MOA Scale indexes and the criterion variables, as well as were negligible. In this sample, most participants had a MOAb the set of competing predicted correlations, are in Table 2. The primary rcontrast - CV procedure data are presented in Table 3. As can be seen from the first row in Table 3, for all four indexes, TABLE 3.—MOA Scale rcontrast - construct validity (CV) effect size statistics for the MOA Scale as pathology measure effect size was larger competing prediction sets across indexes. than the MOA Scale as OR measure effect size. The difference MOAx MOAb MOAp MOAHI approached significance for MOAHI (p = .07), and reached Quantity OR Pathology OR Pathology OR Pathology OR Pathology significance for MOAx (p = .03). This latter index appears to be more sensitive to pathology than to OR, whereas the other rcontrast - CV .42 .49 .03 .10 .39 .41 .54 .59 95% CI indexes appear to be equally sensitive to both constructs. Fol- From: 0.25 0.32 −0.17 −0.01 0.21 0.24 0.38 0.45 lowing Cohen’s (1992) guidelines, for both the MOA Scale as To: 0.57 0.63 0.23 0.30 0.54 0.56 0.66 0.71 − − OR measure and MOA Scale as pathology measure, rcontrast - CV Zcontrast 4.40 5.19 0.28 .97 4.01 4.21 5.79 6.50 effect sizes were large (>.50) for the MOAHI, medium to large tcontrast 4.64 5.58 0.28 .97 4.19 4.42 6.34 7.29 pcontrast <.001 <.001 0.390 .166 <.001 <.001 <.001 <.001 (.39–.49) for MOAx and MOAp, and small or negligible (≤.10) for MOAb. Note.MOA= Mutuality of Autonomy Scale; OR = object relations: MOA Scale as OR measure; Pathology = MOA Scale as measure of psychopathology; MOAx = MOA Scale For each of the prediction sets (OR, pathology), the 95% con- mean; MOAb = MOA Scale best score index; MOAp = MOA Scale pathology index; fidence interval (CI) around MOAb’s rcontrast - CV effect size did MOAHI = MOA Scale health index; CI = confidence interval. MOA SCALE CONSTRUCT VALIDITY 233

score of 1 (n = 52) or 2 (n = 42); only 6 of them scored above a the new and initial effect sizes were negligible (.01–.04) and did 2. Here, MOAb simply did not covary much with other variables, not favor one set of criteria systematically. Therefore, we do not so relationships were small. report these data in a table. A closer examination of our SCORS data revealed some puz- zling relationships between the SCORS variables and the MOA Scale indexes. The observed correlations between all four MOA DISCUSSION indexes and the CHR scale were not in the expected direction; In this study, we examined the construct validity of the MOA the same was true for the MOAx and MOAp scores with the Scale using the rcontrast - CV procedure developed by Westen and EIR scale and for MOAb with the AT scale. This occurred de- Rosenthal (2003). We generated competing hypotheses—that spite the fact that the MOA scores showed an expected pattern the MOA Scale primarily measures OR and that it primarily of relationship with other criteria. These discrepancies led us to measures psychopathology. Each hypothesis was represented as consider errors when scoring the SCORS scales and/or entering a pattern of predicted correlations between the MOA Scale and a them into our database. To test this, we generated scatter plots set of criterion variables, and each was tested against the actual between MOAx and the SCORS variables, identified outliers, correlations in our sample. We then generated the rcontrast - CV and reexamined their scores on MOA and SCORS variables. effect sizes representing the degree of correspondence between Further, 20 participants’ MOA and SCORS scores were chosen each of these prediction sets (OR, psychopathology) and the randomly and checked for data entry error. No scoring or data actual relationships. In light of previous research findings, our entry errors were identified. However, given the unexpected rela- initial round of analyses produced some puzzling relationships tionships, we reanalyzed the data without the SCORS variables among the SCORS variables and between SCORS variables and (See Table 4). some MOA Scale indexes. We therefore decided to repeat the The rcontrast - CV effect sizes for most indexes increased across analyses without the SCORS data. Magnitudes of the resulting both prediction sets when the SCORS variables were excluded; rcontrast - CV effect sizes were large for the MOAHI, MOAx, and however, the increases were most pronounced for the OR predic- MOAp indexes and small for the MOAb index, although all tions. The indexes demonstrated the same orders of magnitude were significant (p<.05). The differences between the MOA described earlier, descending from MOAHI, MOAx, MOAp, Scale as OR measure and MOA Scale as pathology measure to MOAb. This time, both MOAHI and MOAx effect sizes rcontrast - CV effect sizes were not significant for any index; indeed, were large. MOAp and MOAb remained medium large and none of the effect sizes differed by more than .07 points. Thus, small, respectively, although both MOAb effect sizes reached neither of the competing hypotheses was supported. The MOA significance in these analyses (p ≤ .03). MOA Scale OR ef- Scale appears to be a good measure of OR quality as well as fect sizes were higher than pathology effect sizes for MOAHI psychopathology severity, but it does not appear to discriminate and MOAp, and the reverse was true for MOAb; but ulti- between the two constructs. mately, none of the OR–pathology effect size differences were Taken separately, our observed OR and pathology effect sizes significant. are therefore consistent with studies that have supported the con- We utilized Holaday and Sparks’ (2001) revised MOA Scale struct validity of the MOA Scale as a measure of object relations scoring guidelines instead of Urist’s (1977) original criteria. (Ackerman et al., 2001; Blatt et al., 1990; Fowler, Hilsenroth, The reviewers of the initial version of this manuscript noted & Handler, 1996; Urist, 1977) as well as studies in which MOA that these two sets of guidelines differ in how a response that Scale scores have converged with indicators of psychopathol- portrays “Fighting between equals” would be scored. The for- ogy, such as diagnosis severity and lifetime psychosis severity mer would assign a Level 2 based on the portrayed balance of (Harder et al., 1984). Furthermore, our data are consistent with power, whereas the latter would assign a Level 5 for fighting a number of other studies that have shown that the MOA Scale Downloaded By: [University of Toledo] At: 00:58 14 April 2009 (lower scores are healthier). To address this issue, all 18 “Fight- is a significant predictor of behavior and an effective tool to ing between equals” responses were rescored as Level 5 (as per discriminate among a variety of groups (Ackerman et al., 2000; Urist, 1977), and the analyses were re-run. Differences between Brown-Cheatham, 1993; Cook et al., 1995; Fowler et al., 2004; Fowler, Hilsenroth, & Nolan, 2000; Goddard & Tuber, 1989; Kavanagh, 1985; Leichsenring, 2004; Leifer, Shapiro, Martone, & Kassem, 1991; Sayler, Holmstrom, & Noshpitz, 1991; Strauss TABLE 4.—Mutuality of Autonomy (MOA) Scale r contrast - construct & Ryan, 1987; Tuber, 1983; Tuber, Frank, & Santostefano, validity (CV) effect size statistics for competing prediction sets across indexes without Social Cognition and Object Relations Scale data. 1989). Although preliminary, our findings raise some practical is- MOAx MOAb MOAp MOAHI sues. Classical psychoanalytic drive/structure theory holds that internal representations of self and others develop from the in- Quantity OR Pathology OR Pathology OR Pathology OR Pathology terplay of instincts, social contact with caregivers, and inherited rcontrast - CV .56 .56 .18 .22 .47 .40 .62 .61 characteristics such as temperament (Berg et al., 1993; Freud, 95% CI From: 0.41 0.41 −0.02 0.02 0.30 0.22 0.48 0.47 1949). In contrast, OR theory states that self-representations To: 0.68 0.68 0.36 0.40 0.61 0.56 0.73 0.72 and other representations develop concomitant with, or even Zcontrast 6.04 6.07 1.77 2.21 4.89 4.17 −6.90 −6.78 developmentally precede, other aspects of psychological struc- tcontrast 6.67 6.71 1.79 2.24 5.22 4.37 7.85 7.68 pcontrast <.001 <.001 .038 .014 <.001 <.001 <.001 <.001 ture such as the instincts and ego functions (Christopher, Bick- hard, & Lambeth, 2001; Kernberg, 1986; Masterson, 1981). Note.OR= object relations: MOA Scale as OR measure; Pathology = MOA Scale In either theory, the development of object representations as measure of psychopathology; MOAx = MOA Scale mean; MOAb = MOA Scale best score index; MOAp = MOA Scale Pathology index; MOAHI = MOA Scale Health index; necessarily covaries with the development of the personality, CI = confidence interval. including into pathological outcomes. Given this theoretical 234 BOMBEL, MIHURA, MEYER

relationship, OR quality and psychopathology should be fun- nomena and psychopathology at the aggregated index score damentally intertwined and correlated constructs. Indeed, the level of analysis. Indeed, OR quality may be too complicated to relationship between our OR and pathology prediction sets in assess adequately in a single Rorschach modality. Measures that the second and third column of Table 2 reflected this (r = .89; focus almost exclusively on theme (Coonerty, 1986; Kwawer, ICC [A,1] = .78). 1979; Mayman, 1967; Urist, 1977) or structural aspects of re- However, there has been an ongoing interest in developing as- sponses (Blatt et al., 1976; Pruitt & Spilka, 1964) may not be sessment approaches or instruments that can home in on specific sampling the full breadth of how this construct is expressed aspects of dynamic personality structure and functioning, for on the Rorschach. Here, it could be that OR phenomena and example, primary process phenomena (Holt, 2002); ego impair- psychopathology are distinct enough to assess more purely as ment (Perry & Viglione, 1991); defensive functioning (Lerner, separate constructs but that Rorschach measures are only sen- 1990); core conflict (Luborsky & Crits-Cristoph, 1990); attach- sitive to a fraction of the actual phenomena that would dis- ment (Main, Kaplan, & Cassidy, 1985); and of course, object criminate between the two. Alternately, the two constructs are representations (Urist, 1977), to name a few. When such mea- simply not distinguishable enough to measure more purely than sures are designed for use with the Rorschach, scoring guide- we presently do. Our data may support the latter hypothesis. lines focus on thematic and/or structural elements of response As noted earlier, based on the two sets of predicted correla- contents that relate theoretically to the constructs being assessed. tions, the independent judges expected that the MOA Scale as For example, an MOA Scale score (level) for a particular re- a pure measure of OR would behave much like the MOA Scale sponse is assigned based on the OR developmental theme of as a pure measure of pathology (r = .89; ICC [A,1] = .78). any interaction portrayed. This scoring focus is an attempt to Importantly, these judgments were reliable, with average mea- make the measure more sensitive to the target construct than to sures ICCs across judges for the OR and pathology prediction all other constructs (i.e., discrimination), even ones that might sets of .96 and .94, respectively. Still, to further clarify the is- overlap theoretically. If the guidelines can accomplish this, the sue, it would be helpful to use Westen and Rosenthal’s (2003) final score communicates information primarily about the target rcontrast - CV procedure to examine implicit and performance OR construct. If not, the score will communicate information about measures with samples that are representative of the normal the target construct as well as other constructs the guidelines population. cannot “tune out.” Here, the aggregated score would represent In our sample (N = 100; Total r = 2,271), 18 responses some unknowable proportion of information about target and across 13 protocols portrayed “Fighting between equals.” Scor- nontarget constructs. ing these responses as Level 5 (Urist, 1977) or Level 2 (Ho- In this study, we examined precisely this capacity in the laday & Sparks, 2001) ultimately made little difference. The MOA Scale and found that it did not discriminate OR phenom- rcontrast - CV effect sizes generated from the Urist (1977) and Ho- ena from general psychopathology. Consequently, interpreting laday and Sparks (2001) scoring guidelines were essentially MOA Scale index scores as relatively pure indicators of internal- equivalent. This suggests that the “Fighting between equals” ized object representation quality that are distinct from overall response occurs infrequently enough that Holaday and Sparks’ psychopathology would not be sound. At a more molecular level (2001) MOA Scale scoring guidelines can be used in lieu of of interpretation, however, some authors have suggested that lev- Urist’s (1977) in research and clinical settings. This is fortunate els at the healthy end of the scale are generally more sensitive to because the Holaday and Sparks criteria are more comprehen- OR phenomena than to psychopathology, whereas Levels 5, 6, sive and provide a great deal more guidance about scoring de- and 7 are more sensitive to psychopathology (Berg et al., 1993; cisions than the Urist guidelines. Furthermore, as noted, Urist’s Blatt et al., 1990). Our data are consistent with this. Only two (1977; Urist & Shill, 1982) published descriptions of MOA criterion variables correlated significantly with MOAb: GHR Scale are inconsistent across Levels 2 through 6, which can Downloaded By: [University of Toledo] At: 00:58 14 April 2009 (r =−.33, p<.01) and the CDI (r = .34, p<.01). MOAp create scoring confusion. correlated significantly with S-CON (r = .29, p<.01) and MOAx is one of the most reported MOA indexes in the lit- DEPI (r = .20, p<.05), whereas WSum6 (r = .33, p<.01) erature (Ackerman et al., 2000; Blatt et al., 1990; Fowler et and PTI (r = .25, p<.05) correlated better with MOAp than al., 2004; Goddard & Tuber, 1989; Leifer et al., 1991; Mazor, with any other index. PHR, which is scored from structural ele- Alfa, & Gampel, 1993; Ryan et al., 1985; Spear & Sugarman, ments of responses, related better to MOAp (r = .42, p<.01) 1984; Strauss & Ryan, 1987; Tuber, 1989; Tuber & Coates, than did either of these two thought disorder variables. 1989), and it has frequently been used as the sole MOA Scale The MOA Scale may be most clinically useful when interpre- index (Canetto, Feldman, & Lupei, 1989; Donahue & Tuber, tation begins with a careful review of response content and con- 1993; Fowler et al., 1995; Harder et al., 1984; Hart & Hilton, siders a range of additional scores to refine interpretation. This 1988; J. F. Murray, 1985; Tuber et al., 1989). Results of this process can reveal the overall tone of the object representations study, however, suggest that it may not be the best MOA Scale in the protocol and clarify how OR and pathology phenomena index for assessing its target constructs, OR quality and psy- contributed to final index scores (Berg et al., 1993). For exam- chopathology. MOAHI, which is computed as a ratio of healthy ple, an MOAx score of 5.32 might reflect OR quality more than to primitive scores relative to the number of scorable MOA re- psychopathology if X – % = .15, PTI = 2, and S-CON = 4. On sponses, appears to be the most sensitive MOA Scale index. the other hand, psychopathology would likely contribute more MOAHI rcontrast - CV effect sizes were the largest of any index to this same MOAx score if it were accompanied by X – % = regardless of prediction set (OR or pathology) and whether the .35, PTI = 4, and S-CON = 7. SCORS variables were included in the analyses. One possible broader implication of these results is that A limitation of this study was the uncertainty regarding theme-based Rorschach approaches to assessing OR phenom- the validity of the CHR, AT, and EIR scales. The interrater ena, in general, are inherently unable to discriminate OR phe- reliability values ranged from fair to good and suggest that MOA SCALE CONSTRUCT VALIDITY 235

these scales were scored with an acceptable degree of reliability Bombel, G. A. (2006). A meta-analysis of interrater scoring reliability for the by raters who previously documented reliable scoring relative Rorschach Mutuality of Autonomy (MOA) Scale. Unpublished master’s thesis, to gold standard protocols; however, the correlations among University of Toledo, Toledo, Ohio. these scales, as well as the relationships between these scales Brown-Cheatham, M. (1993). The Rorschach Mutuality of Autonomy Scale in and some MOA Scale indexes, did not correspond well to pre- the assessment of Black father-absent male children. Journal of Personality viously published data. For example, Ackerman et al. (2001) Assessment, 61, 524–530. Burns, B., & Viglione, D. J. (1996). The Rorschach Human Experience variable, obtained expected significant correlations between MOAb and interpersonal relatedness, and object representations in nonpatients. Psycho- two of the SCORS scales, AT and EIR, whereas we found small logical Assessment, 8, 92–99. or negligible relationships between MOAb and all three SCORS Burke, W. F., Friedman, G., & Gorlitz, P. (1988). The psychoanalytic Rorschach scales. Part of this, as noted, may be due to the properties of the profile: An integration of drive, ego, and object relations perspectives. Psy- MOAb index score distribution in our sample. Our results also choanalytic Psychology, 5, 193–212. differed from Ackerman et al.’s (2001) in that we obtained sig- Butcher, J. N., Dahlstrom, W. G., Graham, J. R., Tellegen, A., & Kaemmer, B. nificant relationships between MOAp and two SCORS scales, (1989). MMPI–2: Minnesota Multiphasic Personality Inventory–2: Manual CHR and AT. The significant relationships between MOAp and for administration and scoring. Minneapolis: University of Minnesota Press. CHR, and MOAHI and CHR, were in unexpected directions, Canetto, S. S., Feldman, L. B., & Lupei, R. L. (1989). Suicidal persons and although MOAp and MOAHI generally behaved as expected their partners—Individual and interpersonal dynamics. Suicide and Life- Threatening Behavior, 19, 237–248. with other variables. Christopher, J. C., Bickhard, M. H., & Lambeth, G. S. (2001). Otto Kernberg’s A second limitation of this study was the limited number of object relations theory: A metapsychological critique. Theory & Psychology, assessment methods used to obtain personality data. We used 11, 687–711. only implicit techniques. Arguably, however, the Rorschach and Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating TAT tasks are different enough that placing them in the same normed and standardized assessment instruments in psychology. Psycholog- category of assessment method is challenging; indeed, it may ical Assessment, 6, 284–290. be inappropriate (Meyer & Kurtz, 2006). Still, the design of Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159. any future replication of this project would do well to include Cook, B., Blatt, S., & Ford, R. (1995). The prediction of therapeutic response to behavioral measures among the criterion variables. long-term intensive treatment of seriously disturbed young adult inpatients. Psychotherapy Research, 5, 218–230. Coonerty, S. (1986). An exploration of separation-individuation themes in the borderline personality disorder. Journal of Personality Assessment, 50, 501– ACKNOWLEDGMENTS 511. We thank the anonymous reviewers for their helpful com- Cooper, S. H., Perry, J. C., Hoke, L., & Richman, R. (1985). Transitional ments on earlier versions of this article. We also thank Prachi relatedness and borderline personality disorder. Psychoanalytic Psychology, Kene, Margaret Chan, Nick Katko, and Aaron Upton for their 2, 115–128. help with interrater reliability scoring. Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302. Dean, K. L., Viglione, D. J., Perry, W., & Meyer, G. J. (2007). A method to REFERENCES optimize the response range while maintaining Rorschach Comprehensive Ackerman, S. J., Hilsenroth, M. J., Clemence, A. J., Weatherill, R., & Fowler, System validity. Journal of Personality Assessment, 89, 149–161. J. C. (2000). The effects of social cognition and object representation on Dao, T. K., & Prevatt, F. (2006). A psychometric evaluation of the Rorschach psychotherapy continuation. Bulletin of the Menninger Clinic, 64, 386–408. Comprehensive System’s Perceptual Thinking Index. Journal of Personality Ackerman, S. J., Hilsenroth, M. J., Clemence, A. J., Weatherill, R., & Fowler, Assessment, 86, 180–189. J. C. (2001). Convergent validity of Rorschach and TAT scales of object Donahue, P. J., & Tuber, S. B. (1993). Rorschach adaptive fantasy images and relations. Journal of Personality Assessment, 77, 295–306. coping in children under severe environmental stress. Journal of Personality Downloaded By: [University of Toledo] At: 00:58 14 April 2009 American Psychiatric Association (2000). Diagnostic and statistical manual of Assessment, 60, 421–434. mental disorders (4th ed., text revision). Washington, DC: Author. Exner, J. E. (1986). The Rorschach: A Comprehensive System: Vol. 1. Basic Barron, F. (1953). An ego-strength scale which predicts response to psychother- foundations (2nd ed.). New York: Wiley. apy. Journal of Consulting Psychology, 17, 327–333. Exner, J. E. (1993). The Rorschach: A Comprehensive System: Vol. 1. Basic Bell, M. (1991). An introduction to the Bell Object Relations Reality Testing foundations (3rd ed.). Oxford, England: Wiley. Inventory. (Available from Morris Bell, Psychology Service, 116B, Veterans Exner, J. E. (2003). The Rorschach: A Comprehensive System: Vol. 1. Basic Affairs Medical Center, West Haven, CT 06516). foundations and principles of interpretation (4th ed.). Hoboken, NJ: Wiley. Berg, J. L., Packer, A., & Nunno, V. J. (1993). A Rorschach analysis: Parallel Fisher, P. H., Sperling, M. B., & Carr, A. C. (1990). Assessment of adult relat- disturbance in thought and in self/object representation. Journal of Personality edness: A review of empirical findings from object relations and attachment Assessment, 61, 311–323. theories. Journal of Personality Assessment, 55, 499–520. Bessell, H. (1984). Thelovetest. New York: William Morrow. Fowler, J. C., Ackerman, S. J., Speanburg, S., Bailey, A., Blagys, M., & Con- Blais, M. A., Hilsenroth, M. J., Fowler, J. C., & Conboy, C. A. (1999). A klin, A. C. (2004). Personality and symptom change in treatment-refractory Rorschach exploration of the DSM–IV borderline personality disorder. Jour- inpatients: Evaluation of the phase model of change using Rorschach, TAT, nal of Clinical Psychology, 55, 563–572. and DSM–IV Axis V. Journal of Personality Assessment, 83, 306–322. Blatt, S. J., Brenneis, C. B., Schimek, J. G., & Glick, M. (1976). Normal Fowler, J. C., Addelson, B., & Clemence, A. J. (2006). (2006, March). A meta- development and psychopathological impairment of the concept of the object analysis of the Mutuality of Autonomy Scale. Paper presented at the Annual on the Rorschach. Journal of Abnormal Psychology, 85, 364–373. Meeting of the Society for Personality Assessment, Washington, DC. Blatt, S. J., & Ritzler, B. A. (1974). Thought disturbance and boundary dis- Fowler, C., Hilsenroth, M. J., & Handler, L. (1995). Early memories: An explo- turbances in psychosis. Journal of Consulting and Clinical Psychology, 42, ration of theoretically derived queries and their clinical utility. Bulletin of the 370–381. Menninger Clinic, 59, 79–98. Blatt, S. J., Tuber, S. B., & Auerbach, J. S. (1990). Representation of interper- Fowler, J. C., Hilsenroth, M. J., & Handler, L. (1996). A multimethod approach sonal interactions on the Rorschach and level of psychopathology. Journal of to assessing dependency: The early memory dependency probe. Journal of Personality Assessment, 54, 711–728. Personality Assessment, 67, 399–413. 236 BOMBEL, MIHURA, MEYER

Fowler, J. C., Hilsenroth, M. J., & Nolan, E. (2000). Exploring the inner world Masterson, J. F. (1981). The narcissistic and borderline disorders: An integrated of self-mutilating borderline patients: A Rorschach investigation. Bulletin of developmental approach. New York: Brunner/Mazel. the Menninger Clinic, 64, 365–385. Mayman, M. (1967). Object representations and object relationships in Fowler, J. C., Piers, C., Hilsenroth, M. J., Holdwick, D. J., & Padawer, Rorschach responses. Journal of Projective Techniques & Personality As- J. R. (2001). The Rorschach Suicide Constellation: Assessing vari- sessment, 31, 17–24. ous degrees of lethality. Journal of Personality Assessment, 76, 333– Mazor, A., Alfa, A., & Gampel, Y. (1993). On the thin line between connection 351. and separation: The individuation process, from cognitive and object relations Frank, G. (1995). On the assessment of self and object representations from the perspectives, in Kibbutz adolescents. Journal of Youth and Adolescents, 22, Rorschach: A review of the research and commentary. Psychological Reports, 641–669. 76, 659–671. McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass Freud, S. (1949). An outline of psychoanalysis. New York: Norton. correlation coefficients. Psychological Methods, 1, 30–46. Goddard, R., & Tuber, S. (1989). Boyhood separation anxiety disorder: thought McWilliams, N. (1994). Psychoanalytic diagnosis: Understanding personality disorder and object relations psychopathology as manifested in Rorschach structure in the clinical process. New York: Guilford. imagery. Journal of Personality Assessment, 53, 239–252. Meyer, G. J. (1992). Response frequency problems in the Rorschach: Clinical Goldberg, L. R. (1965). Diagnosticians vs. diagnostic signs: The diagnosis of and research implications with suggestions for the future. Journal of Person- psychosis vs. neurosis from the MMPI–2. Psychological Monographs, 79(9, ality Assessment, 58, 231–244. Whole No. 602). Meyer, G. J., & Kurtz, J. E. (2006). Advancing personality assessment terminol- Harder, D. W., Greenwald, D. F., Wechsler, S., & Ritzler, B. A. (1984). The Urist ogy: Time to retire “objective” and “projective” as personality test descriptors. Rorschach Mutuality of Autonomy Scale as an indicator of psychopathology. Journal of Personality Assessment, 87, 223–225. Journal of Clinical Psychology, 40, 1078–1083. Meyer, G. J., & Resnick, J. (1996). Assessing ego impairment: Do scoring pro- Hart, B., & Hilton, I. (1988). Dimensions of personality organization as pre- cedures make a difference? Paper presented at the XV International Congress dictors of teenage pregnancy risk. Journal of Personality Assessment, 52, of Rorschach and Projective Methods, Boston, MA. 116–132. Murray, H. A. (1943). Thematic Apperception Test manual. Cambridge, MA: Hibbard, S., Hilsenroth, M. J., Hibbard, J. K., & Nash, M. R. (1995). A va- Harvard University Press. lidity study of two projective object representations measures. Psychological Murray, J. F. (1985). Borderline manifestations in the Rorschachs of male trans- Assessment, 7, 432–439. sexuals. Journal of Personality Assessment, 49, 454–466. Hilsenroth, M., Stein, M., & Pinsker, J. (2004). Social cognition and object Perry, W., & Viglione, D. J. (1991). The Ego Impairment Index as a predictor of relations scale: Global rating method. Unpublished manuscript, The Derner outcome in melancholic depressed patients treated with tricyclic antidepres- Institute of Advanced Psychological Studies, Adelphi University, Garden sants. Journal of Personality Assessment, 56, 487–501. City, NJ. Peters, E. J., Hilsenroth, M. J., Eudell-Simmons, E. M., Blagys, M. D., & Holaday, M., & Sparks, C. L. (2001). Revised guidelines for Urist’s Mutuality Handler, L. (2006). Reliability and validity of the Social Cognition and of Autonomy Scale (MOA). Assessment, 8, 145–154. Object Relations Scale in clinical use. Psychotherapy Research, 16, 617– Holt, R. R. (2002). Quantitative research on the primary process: method and 626. findings. Journal of the American Psychoanalytic Association, 50, 457– Pruitt, W. A., & Spilka, B. (1964). Rorschach empathy-object relationship 482. scale. Journal of Projective Techniques & Personality Assessment, 28, 311– Huprich, S. K., & Greenberg, R. P. (2003). Advances in the assessment of object 336. relations in the 1990s. Clinical Psychology Review, 23, 665–698. Resnick, J. (1994). Rorschach assessment of ego functioning: A comparison of Jørgenson, K., Andersen, T. J., & Dam, H. (2000). The diagnostic efficiency the EII and CESI. Unpublished master’s thesis, Loyola University of Chicago, of the Rorschach Depression Index and the Schizophrenia Index: A review. Chicago, Illinois. Assessment, 7, 259–280. Resnick, J., & Meyer, G. J. (1995, March). Rorschach assessment of ego func- Kavanagh, G. G. (1985). Changes in patients’ object representations during tioning: A comparison of the EII and the CESI. Paper presented at the annual psychoanalysis and psychoanalytic psychotherapy. Bulletin of the Menninger meeting of the Society for Personality Assessment, Atlanta, GA. Clinic, 49, 546–564. Rorschach, H. (1942). Psychodiagnostik [Psychodiagnostics] (Han Huber, Kernberg, O. K. (1986). Severe personality disorders: Psychotherapeutic strate- Trans.). Bern, Switzerland: Verlag. (Original work published 1921) Downloaded By: [University of Toledo] At: 00:58 14 April 2009 gies. New Haven, CT: Yale University Press. Rosenthal, R., Rosnow, R. L., & Rubin, D. B. (2000). Contrasts and effect sizes Kleiger, J. H. (1992). A conceptual critique of the EA:es comparison in in behavioral research: A correlational approach. New York: Cambridge the Comprehensive Rorschach System. Psychological Assessment, 4, 288– University Press. 296. Ryan, R. M., Avery, R. R., & Grolnick, W. S. (1985). A Rorschach assessment Kwawer, J. S. (1979). Borderline phenomena, interpersonal relations, and the of children’s mutuality of autonomy. Journal of Personality Assessment, 49, Rorschach test. Bulletin of the Menninger Clinic, 43, 515–524. 6–12. Leichsenring, F. (2004). Quality of depressive experiences in borderline per- Salyer, K. M., Holmstrom, R. W., & Noshpitz, J. D. (1991). Learning disabilities sonality disorders: Differences between patients with borderline personality as a childhood manifestation of severe psychopathology. American Journal disorder and patients with higher levels of personality organization. Bulletin of Orthopsychiatry, 61, 230–240. of the Menninger Clinic, 68, 9–22. Smith, T. E. (1993). Measurement of object relations: A review. Journal of Leifer, M., Shapiro, J. P., Martone, M. W., & Kassem, L. (1991). Rorschach Psychotherapy Practice & Research, 2, 19–37. assessment of psychological functioning in sexually abused girls. Journal of Smith, S. R., Baity, M. R., Knowles, E. S., & Hilsenroth, M. J. (2001). Assess- Personality Assessment, 56, 14–28. ment of disordered thinking in children and adolescents: The Rorschach Lerner, P. M. (1990). Rorschach assessment of primitive defenses: A review. Perceptual-Thinking Index. Journal of Personality Assessment, 77, 447– Journal of Personality Assessment, 54, 30–46. 463. Luborsky, L., & Crits-Cristoph, P. (1990). Understanding transference: The Spear, W. E., & Sugarman, A. (1984). Dimensions of internalized object rela- core conflictual relationship theme method. New York: Basic Books. tions in borderline and schizophrenic patients. Psychoanalytic Psychology, 1, Main, M., Kaplan, N., & Cassidy, J. (1985). Security in infancy, childhood, and 113–129. adulthood: A move to the level of representation. In I. Bretherton & E. Waters Strauss, J., & Ryan, R. M. (1987). Autonomy disturbances in subtypes of (Eds.), Monographs of the Society for Research in Child development, 50, anorexia-nervosa. Journal of Abnormal Psychology, 96, 254–258. 66–104. Stricker, G., & Healey, B. J. (1990). Projective assessment of object relations: Masterson, J. F. (1976). Psychotherapy of the borderline adult.NewYork: A review of the empirical literature. Psychological Assessment, 2, 219– Brunner/Mazel. 230. MOA SCALE CONSTRUCT VALIDITY 237

Tuber, S. B. (1983). Children’s Rorschach scores as predictors of later adjust- Urist, J., & Shill, M. (1982). Validity of the Rorschach Mutuality of Auton- ment. Journal of Consulting & Clinical Psychology, 51, 379–385. omy Scale: A replication using excerpted responses. Journal of Personality Tuber, S. B. (1989). Assessment of children’s object-representations with the Assessment, 46, 450–454. Rorschach. Bulletin of the Meninger Clinic, 53, 432–441. Viglione, D. J., Perry, W., Jansak, D., Meyer, G., & Exner, J. E. (2003). Modi- Tuber, S. (1992). Empirical and clinical assessments of children’s object re- fying the Rorschach Human Experience variable to create the Human Repre- lations and object representations. Journal of Personality Assessment, 58, sentational variable. Journal of Personality Assessment, 81, 64–73. 179–197. Westen, D., Lohr, N., Silk, K., Kerber, K., & Goodrich, S. (1990). Social cog- Tuber, S., & Coates, S. (1989). Indices of psychopathology in the Rorschachs of nition and object relations scale (SCORS): Manual for coding TAT data (5th boys with severe gender identity disorder: A comparison with normal control ed.). Unpublished manuscript, University of Michigan, Ann Arbor. subjects. Journal of Personality Assessment, 53, 100–112. Westen, D., & Rosenthal, R. (2003). Quantifying construct validity: Two Tuber, S. B., Frank, M. A., & Santostefano, S. (1989). Children’s anticipation simple measures. Journal of Personality and Social Psychology, 84, 608– of impending surgery: Shifts in object-representational paradigms. Bulletin 618. of the Menninger Clinic, 53, 501–511. Wolfe, R., & Hanley, J. (2002). If we’re so different, why do we keep overlap- Urist, J. (1977). The Rorschach Test and the assessment of object relations. ping? When 1 plus 1 doesn’t make 2. Canadian Medical Association Journal, Journal of Personality Assessment, 41, 3–9. 166, 65–66. Downloaded By: [University of Toledo] At: 00:58 14 April 2009