Intelligence Test Interpretation Research
Total Page:16
File Type:pdf, Size:1020Kb
APPENDIX A Intelligence Test Interpretation Research SCIENTIFIC CONSIDERATIONS Sample Composition AND ASSUMPTIONS I selected both exceptional and nationally repre- ( SEE CHAPTER 17) sentative samples for inclusion. Differing Interpretations Study Selection ofthe Same Results The majority ofstudies included in Appendix A I favored a "preponderance-of-evidence" crite- were aimed at identifying the latent traits un- rion when alternative interpretations needed to derlying test scores, and profiles are typically be reconciled. In his landmark factor analytic factor analytic. Therefore, criteria for identify- study of nearly 500 data sets, Carroll (1993) ing technically adequate factor analyses had to found that several Stratum I reading, spelling, be addressed. The considerations that I used for and writing abilities loaded with the "gc" (crys- including factor analytic studies were: tallized ability) factor for several data sets. In one large-scale factor analytic investigation, McGrew Sample Size (1997) found a reading/writing factor that dif- I chose primarily large, national standardization fered from a "gc'' factor. I think that Carroll's samples where N equaled approximately 200 or findings represent a preponderance of evidence greater. in comparison to the single study of McGrew 603 604 APPENDIX A INTELLIGENCE TEST INTERPRETATION RESEARCH (1997). Consequently, I subsume reading and tions when disagreements arise between studies. writing abilities under the Stratum II "gc" ability. When referring to identification of the latent trait assessed by the WISC-III third factor again, Differing Results Given the Same for example, Carroll (1994) asserts that "the Interpretations WISC-III was not designed for factor analysis because the various factors that it may measure- An example of this scenario is as follows. On one at least beyond the Verbal and Performance fac- hand, Cohen (1959) labels the third WISC factor tors-are not represented adequately by the as Freedom from Distractibility; he characterizes multiple measures of those factors" (p. 138). I it so "primarily due to the loadings of subtests agree that the measure ofinattention/distractibility which clearly do not involve memory (Mazes, via the third factor is unsupportable. Drawing con- Picture Arrangement, Object Assembly), but clusions, however, is clouded by scenarios where which it seems reasonable to suppose are quite both positive and negative findings exist. In this vulnerable to the effects ofdistractibility" (p. 288). case I make an arbitrary decision. On the other hand, Blaha and Wallbrown (1996) assign the same label as Cohen to the third factor Exploratory versus Confirmatory Methods with significant loadings only for the Arithmetic and Digit Span subtests. Again, I invoke the pre- Generally speaking, confirmatory factor analytic ponderance-of-evidence rule to make a decision methods have gained considerable popularity regarding the conclusion to be drawn. Most stud- over exploratory methods in modem factor analy- ies have found the Arithmetic and Digit Spans sis. I think that confirmatory methods may also subtests to load on this factor. They have also, result in more scientifically useful tests of factor however, found little evidence that inattention/ structure (see Kamphaus, Benson, Hutchinson, & distractibility is likely to be the central latent trait Platt, 1994, for an example). Therefore, I prefer assessed by this factor (Kamphaus, 1998). such methods over traditional factor analysis. Types ofEvidence Myopic Research Evidence There is not a clear consensus on the issue of Carroll (1993) makes a compelling argument weighting evidence differentially. Although it that factor analyses oftests such as the WISC-III may seem that factor analytic studies have been are likely to produce noncontributory results be- reified to a special status in the evaluation of cause of the nature of the scale. I try, whenever intelligence test validity, other forms of validity possible, to draw conclusions regarding research may be equally ifnot more important depending that is consistent with findings of other disci- on the issue under study. If, for example, the plines (e.g., cognitive psychology). research question deals with the issue of using an intelligence test for differential diagnosis, Lack ofEvidence then I would weigh research on various diag- nostic groups more heavily than factor analytic I appreciate that some intelligence test interpre- findings. tations are not necessarily invalid in the absence ofresearch. They could be found valid ifthey are studied. Clearly, psychology is no different from The Preponderance ofEvidence medicine and other professions where profes- This premise means that Carroll's multisample sionals have to take action or make nonscientifi- factor analytic work will be proportionally more cally based interpretations in unusual cases and influential than single factor analytic investiga- circumstances. A lack of evidence should not be APPENDIX A INTELLIGENCE TEST INTERPRETATION RESEARCH 605 allowed to limit experimentation. I have and will assume incompleteness and build on the research draw interpretations that are case specific and summarized here with continuing education ac- untested. tivities. Having presented this caveat, it is also clear that some research findings have stood the test of time. One would be hard-pressed, for Theoretical Evidence example, to find new evidence that vocabu- Some test interpretations may be untested, and lary measures are, in fact, measures of spatial yet they may be based on a theory that has some abilities. empirical support. McGrew and Flanagan (1998), for example, provide a comprehensive Applicability ofResearch from test interpretation manual that, of necessity, Previous Editions often theorizes about the fit of intelligence test Fortunately, even the venerable Wechsler scales subtests and scales with Horn's version of"grgc'' are evolving in a manner that precludes auto- theory. They posit, for example, about the Stra- matic generalization ofprevious findings to cur- tum II and Stratum III abilities measured by nu- rent editions. I think that some findings, such as merous tests, including the WAIS-III and the futility of using the PA subtest to measure WPPSI-R. Their speculation about the abilities social judgment, are applicable to the WISC-III measured by the WAIS-III, for example, is es- and WAIS-III. The addition of the Symbol sentially theoretical since the WAIS-III has not Search subtest, however, may make factor ana- been factor analyzed jointly with well-validated lytic evidence for the WISC-R less relevant for measures of Horn's theory. Such theoretical understanding the WISC-III third and fourth propositions are of primary assistance to re- factors. searchers and ofpotential assistance to practition- ers. In fact, speculation based on considerable Definition ofTerms prior research and explicit theory is likely better than speculation based on idiosyncratic theories Intelligence testing research is characterized by or from a poorly articulated theory that is untest- similar sounding terms that are offered without able. Nevertheless, I give theoretical evidence operational definitions. For instance, I do not less weight than the results ofresearch evidence. know the extent to which terms such as Spa- tial:Mechanical, Visualization, Perceptual Organi- zation, and Spatial Organization without Essential Incorporating New Evidence Motor Activity are interchangeable. Moreover, I My summary of extant interpretive research is admit to not having a sound working definition outdated on its date ofpublication. We have the of terms such as integrated brain functioning, good fortune now to be part of the Internet, which precludes me from using such a term for which provides for quicker dissemination and interpretation. Do not make interpretations for updates of findings. Readers of this text should which you do not have a working definition. Summary Table of Intelligence Test Research Findings °'0 Test Solutions Summary Conclusion °' WISC-III Two factor The consistency ofthe WISC-III and the This study extended the examination offactor WPPSI-R Verbal Comprehension and invariance to a larger number of age groups Perceptual Organization factors was within the WISC-III and the WPPSI-R, as well investigated using cross-validation of covariance as utilized contemporary, advanced > structure models applied to the data from the methodology to this end. In summary, the two- 'd 'd respective normative samples of the tests (Allen factor structure consisting ofVerbal t'1 & Thorndike, 1995b). Comprehension and Perceptual Organization was confirmed (Allen & Thorndike, 1995). .. Three factor The WISC-III was factor analyzed for the With the Symbol Search subtest deleted from > national standardization sample without the analyses, the third factor, Freedom from including the Symbol Search subtest (Reynolds Distractibility, appeared to be as stable on the & Ford, 1994). WISC-Ill as on the WISC-R. A three-factor t"' solution is therefore supported, with the third 6 factor being comprised ofArithmetic, Digit Span, and Coding (Reynolds & Ford, 1994). t?l ., Four factor The WISC-III was factor analyzed for the The four-factor (corresponding to the index t?l national standardization sample (Wechsler, scores) solution was deemed most appropriate 1991). based on a variety ofcriteria, including confirmatory factor analytic fit statistics (Wechsler, 1991). WISC-III vs. WISC-R The WISC-III was administered to 257 The data