Frey & Detterman 2004
Total Page:16
File Type:pdf, Size:1020Kb
PSYCHOLOGICAL SCIENCE Research Article Scholastic Assessment or g? The Relationship Between the Scholastic Assessment Test and General Cognitive Ability Meredith C. Frey and Douglas K. Detterman Case Western Reserve University ABSTRACT—There is little evidence showing the relationship be- Testing Service (ETS), the group that administers the SAT along with tween the Scholastic Assessment Test (SAT) and g (general in- countless other standardized tests for the College Board, began a telligence). This research established the relationship between program of validity research with the SAT, known as the Validity Study SAT and g, as well as the appropriateness of the SAT as a Service (VSS). This service, offered free to interested colleges, com- measure of g, and examined the SAT as a premorbid measure of putes the average multiple correlation between three predictor vari- intelligence. In Study 1, we used the National Longitudinal ables (SAT math score, SAT verbal score, and high school record) and Survey of Youth 1979. Measures of g were extracted from the grade point average (GPA) among college freshmen. Between the years Armed Services Vocational Aptitude Battery and correlated with 1964 and 1985, the average correlation between SAT score and SAT scores of 917 participants. The resulting correlation was freshman GPA was in the range from .38 to .46 (Morgan, 1990). De- .82 (.86 corrected for nonlinearity). Study 2 investigated the spite this history of analysis, and the knowledge that g is one of the correlation between revised and recentered SAT scores and most reliable predictors of academic achievement, ETS refuses to cite scores on the Raven’s Advanced Progressive Matrices among any SAT-IQ correlations (Seligman, 1994). The College Board presi- 104 undergraduates. The resulting correlation was .483 (.72 dent, Gaston Caperton, was recently quoted in an on-line publication corrected for restricted range). These studies indicate that the of U.S. News & World Report as admitting that ‘‘in its original form [the SAT is mainly a test of g. We provide equations for converting SAT] was an IQ test’’ (Barnes, 2002). However, the people responsible SAT scores to estimated IQs; such conversion could be useful for for developing and administering the SAT insist that, once logic items estimating premorbid IQ or conducting individual difference were replaced by reading comprehension in 1946, the SAT ceased to research with college students. measure intelligence (Barnes, 2002). Rather, the College Board cur- rently asserts that the SAT measures reasoning ability, which they hold as different from intelligence. The relationship between academic achievement and general intelli- The unusual distinction made by ETS and the College Board not- gence, or g, is well established in the field of intelligence (Jensen, withstanding, some researchers have voiced support for the SAT as a 1998). One particular test of academic achievement, however, is de facto intelligence test. In his 1994 book, A Question of Intelligence, conspicuously absent from these studies. The SAT (once called the Seligman noted, ‘‘Henry Chauncey, a former president of ETS has Scholastic Aptitude Test, and now the Scholastic Assessment Test), an been quoted (in Klitgaard, Choosing Elites, 1985, p. 92) as stating that exam that generations of high school juniors and seniors have ago- the SAT is essentially an intelligence test’’ (pp. 206–207). In fact, the nized over and that has spawned innumerable preparatory books and author of the original SAT, Carl Campbell Brigham, developed his test classes, has not, surprisingly enough, been frequently analyzed in from Army intelligence tests to more accurately differentiate among conjunction with measures of general cognitive ability. individuals at the upper end of the scale (Lemann, 1999). Although That is not to say that there is a lack of research in general on the the College Board asserts that the test is substantially different from SAT. In fact, to the contrary, the SAT’s relation to college admissions its original form, Lemann asserted that, other than having its language and achievement has been the focus of much research. A study of updated, the SAT has changed little since its original version. 25,000 applicants at nine private colleges yielded unsurprising re- Furthermore, empirical evidence suggests a substantial relationship sults: Two thirds of the variance in predicting admissions decisions between the SAT and g. In a study of 339 undergraduates, Brodnick could be accounted for by academic factors, split equally between and Ree (1995) used covariance structure modeling to examine the high school record and admissions test scores, including SAT scores relationship between psychometric g, socioeconomic variables, and (Willingham & Breland, 1982). Furthermore, in 1964, the Educational achievement-test scores. They found substantial general-factor load- ings on both the math (.698) and the verbal (.804) SAT subtests. Address correspondence to D.K. Detterman, Department of Psy- Because they used the SAT to develop their measure of g, it is not chology, Case Western Reserve University, Cleveland, OH 44106; clear if this general factor is the same as that obtained from standard e-mail: [email protected]. intelligence tests. If the general factors are indeed the same, then the Volume 15—Number 6 Copyright r 2004 American Psychological Society 373 Scholastic Assessment Test and g SAT may have been overlooked as a potentially useful measure of TABLE 1 general cognitive functioning. Factor Loadings From the Factor Analysis of the Armed It would be of considerable benefit for researchers and clinicians to Services Vocational Aptitude Battery have an equation for estimating an individual’s IQ from an objective measure such as the SAT. In cases of suspected cognitive decline due Subtest Factor 1 Factor 2 to injury or illness, clinicians currently use a series of demographic General Science .881 À.116 variables to predict their clients’ premorbid IQ. For many of these Arithmetic Reasoning .858 .045 individuals, the SAT may be the only objective measure of premorbid Word Knowledge .885 .071 functioning. The method of predicting IQ with demographic variables Paragraph Comprehension .825 .175 carries with it a standard error of prediction (SEp) of 11.4 (Karzmark, Numerical Operations .724 .426 Heaton, Grant, & Matthews, 1985). It appears that there is ample room Coding Speed .657 .417 for improving the accuracy with which clinicians currently estimate Auto and Shop Information .727 À.425 premorbid IQ. Mathematics Knowledge .800 .129 À In addition to the obvious clinical application, there is a substantial Mechanical Comprehension .794 .320 À benefit to researchers’ use of a regression equation to estimate IQ. Electronics Information .829 .320 Much research in the field of psychology is carried out on under- graduate volunteers. Although there has been a recent push by the the ASVAB are shown in Table 1. They indicate a substantial loading University of California system to discontinue using SAT scores as an of all subtests of the ASVAB on a first factor, g. admission criterion, many colleges and universities continue to re- ASVAB first-factor scores were transformed to an IQ scale using the quire the SAT for admission (Barnes, 2002). Consequently, there is a following equation: IQ 5 (z à 15)1100. Finally, the IQ scores derived vast pool of potential subjects who already have a good measure of from the ASVAB were correlated with SAT scores for the 917 re- their general cognitive functioning in their transcripts. If a researcher spondents who had scores on both measures. Simple correlations could estimate IQ with sufficient accuracy from the scores on record, it between both SAT scores and ASVAB IQ scores and scores on the would eliminate hours of additional testing in cases in which measures small-N intelligence tests were also analyzed. of general intelligence are required. To these ends, we examined the relationship between the SAT and general cognitive ability in two studies. We also developed regression equations to estimate IQ from Results the SAT. Of the 11,878 subjects who had scores on all ASVAB subtests, 917 also took the SAT and were the primary focus in this study. There was a significant correlation between SAT and ASVAB IQ (r 5 .820, STUDY 1 p < .001). However, when a scatter plot of the relationship was examined (see Fig. 1a), a nonlinear relationship was evident. To correct for Method nonlinearity, we added the squared SAT score (R5.857, p < .001) to Participants the regression. A cubic component of SAT also added significantly to 5 Data were extracted from the National Longitudinal Survey of Youth the prediction of IQ (p .011), but the added variance was so small 1979 (NLSY79) data set. The NLSY79 contained a national proba- that we decided to not include it in the regression equation. Figure 1b bility sample of civilian and military subjects aged 14 to 21, with an presents a scatter plot of IQ predicted from a regression equation with overrepresentation of certain minority groups. A total of 12,686 men the squared and cubic components of SAT versus IQ obtained from the and women participated in the NLSY79. first factor of the ASVAB. Entering math and verbal subtest scores into the regression separately, as opposed to using a composite score, did not improve the prediction of IQ. Procedure The simple correlations between SAT and IQ (or whatever measure From the NLSY79 data set, we extracted measures of performance on was reported for the IQ test) for small-N intelligence tests ranged from the Armed Services Vocational Aptitude Battery (ASVAB) and the .53 to .82 for participants who had taken the SAT and also had another SAT (known as the Scholastic Aptitude Test at that time), as well as a IQ test score reported in their school records. All simple correlations number of intelligence tests that did not have scores available for as between tests are summarized in Table 2.