<p> CareerScope® Research Brief #1: Concurrent Validity of the CareerScope Aptitude Battery</p><p>Introduction Procedures CareerScope is a self-administered computer- Participation in the study was voluntary, based interest and aptitude assessment system. The however procedures were designed to maximize aptitude tests measure six constructs identified by the participation and to encourage participants to perform U.S. Department of Labor in its development of the to the best of their ability on all tasks, as well as to General Aptitude Test Battery (GATB). These are: ensure data quality. Packets of information were General Learning Ability, Verbal Aptitude, distributed to employees at two JEVS worksites. To Numerical Aptitude, Spatial Aptitude, Form ensure anonymity, participants were not identified by Perception, and Clerical Perception. Using the six name in the recruitment or assessment process, but corresponding tests of the GATB as criterion tests, instead were identified by ID numbers assigned to we have assessed the concurrent validity of the six each informational packet. The ID numbers of CareerScope aptitude tests in a sample of employed participants who completed both tests were entered adults. into a lottery, with the winner receiving $100. Unlike the GATB, which is a paper and pencil Highest scores on each of the CareerScope and assessment administered by proctor, CareerScope is GATB subtests were awarded $10 each. Sealed self-administered via computer. CareerScope results of both tests were made available to requires a fourth grade reading level as well as participants after conclusion of the study, again using facility with a computer mouse. Moreover, the ID numbers only. Staff monitoring the assessments CareerScope subtests are shorter in length than the were generally unknown to the participants. Fifty- GATB subtests. nine percent of employees who received packets A meta-analysis of 159 studies of the effects of participated in the study. medium of administration found a substantial All participants were administered both the medium effect for speeded tests, but none for power GATB (Parts 1-8) and CareerScope batteries, with tests (Mead & Drasgow, 1993). This suggests that half assigned to taking the GATB first, and half to the the more highly speeded tests (Clerical Perception, CareerScope-first group. Aptitudes assessed by the Form Perception) are likely to have lower validity GATB and CareerScope are General Learning coefficients than the power tests (e.g., Numerical Ability (G,GA), Verbal Aptitude (V,VA), Numerical Reasoning, Pattern Visualization, Word Meanings). Aptitude (N, NA), Spatial Aptitude (S, SA), Form Perception (P, PA), and Clerical Perception (Q, QA). Participants Demographic information was collected using a one- Participants in this study were 115 employees of page questionnaire. the Philadelphia Jewish Employment and Vocational Service (JEVS). Participants ranged in age from 18 Data Analysis to 65 (Mean = 44.0, SD = 10.7, Median = 46). Most Pearson product moment correlations were were female (74 %) and Caucasian (79 %), with 16% computed between standard scores of each black. All had earned a high school diploma or its CareerScope aptitude test and each GATB test. The equivalent, and 60% had earned a bachelor’s degree. correlations of each CareerScope test with that of its A range of occupations were represented by corresponding GATB test are the validity participants. These included maintenance, coefficients, while correlations with other GATB accounting, human resources, executive, secretarial, tests provide evidence of construct (and differential) clerical, counseling, employment training/placement, validity. In addition, the intercorrelations of the production workshop staff, and home health aides. CareerScope aptitude tests and intercorrelations of Although CareerScope requires a 4th grade the GATB aptitude tests were computed. English reading level and ability to use a computer mouse, employees were not screened at recruitment Results for these characteristics. However, a screening Table 1 shows the correlations of each mechanism built in to normal CareerScope CareerScope test with each GATB test. The validity administration identified those who were unable to coefficients appear bolded on the diagonal. Table 2 complete the assessments independently due to shows the intercorrelations of the CareerScope tests, inability to read the instructions or use the computer and Table 3 shows the intercorrelations of the GATB mouse effectively. Results from these 18 subjects are tests. not included in these analyses.</p><p>© 1999 Vocational Research Institute Page 1 Philadelphia, PA ______Table 1 Correlations of CareerScope Aptitude Tests with GATB Aptitude Tests</p><p>GATB Aptitudes G V N S P Q CareerScope GA .81‡ .72‡ .65‡ .53‡ .34‡ .43‡ Aptitudes VA .56‡ .74‡ .34‡ .19* .18 .33‡ NA .72‡ .58‡ .82‡ .28‡ .31‡ .48‡ SA .40‡ .19* .16 .71‡ .39‡ .06 PA .47‡ .36‡ .30‡ .52‡ .59‡ .42‡ QA .49‡ .43‡ .46‡ .22‡ .48‡ .52‡ ______‡ Correlation is significant at the 0.01 level (2-tailed). * Correlation is significant at the 0.05 level (2-tailed).</p><p>______Table 2 Intercorrelations of CareerScope Aptitude Tests</p><p>CareerScope Aptitudes GA VA NA SA PA QA CareerScope GA 1.00 Aptitudes VA .75‡ 1.00 NA .74‡ .40‡ 1.00 SA .57‡ .18 .24‡ 1.00 PA .50‡ .37‡ .35‡ .45‡ 1.00 QA .58‡ .55‡ .50‡ .23* .65‡ 1.00 ______‡ Correlation is significant at the 0.01 level (2-tailed). * Correlation is significant at the 0.05 level (2-tailed).</p><p>Table 3 Intercorrelations of GATB Aptitude Tests</p><p>GATB Aptitudes G V N S P Q GATB G 1.000 Aptitudes V .82‡ 1.000 N .82‡ .59‡ 1.000 S .57‡ .29‡ .25‡ 1.000 P .41‡ .26‡ .37‡ .53‡ 1.000 Q .52‡ .45‡ .56‡ .24* .59‡ 1.000 ______‡ Correlation is significant at the 0.01 level (2-tailed). * Correlation is significant at the 0.05 level (2-tailed).</p><p>© 1999 Vocational Research Institute Page 1 Philadelphia, PA Discussion The six aptitudes measured by CareerScope all were found to correlate at least moderately with the corresponding criterion aptitudes as measured by the GATB. Two (General Learning Ability and Numerical Ability) were high, exceeding .80. As expected, the lowest validity coefficients were found for the two most speeded tests, Form Perception (.59) and Clerical Perception (.52). In addition, the pattern of intercorrelations among the CareerScope aptitude tests was similar to that of the GATB tests.</p><p>References</p><p>Mead, Alan D., & Drasgow, Fritz (1993). Equivalence of computerized and paper-and- pencil cognitive ability tests: A meta-analysis. Psychological Bulletin, 114, 449-458. U.S. Department of Labor (1982). Manual for General Aptitude Test Battery, section 1: Administration and scoring. Washington, DC: U.S. Government Printing Office. Vocational Research Institute (1998). User’s Guide: CareerScope. Philadelphia: Author.</p><p>© 1999 Vocational Research Institute Page 1 Philadelphia, PA</p>