REPORTNO CB-R-81-6; ETS-Rg-81-21

DOCUMENT "RESUME

ED 209 329 TM 810 871

AUTHOR Breland, HunterM.; Griswold,, Philip A. TITLE Group Comparisons for Basic Skills Measures. INSTITUTION College Entrance Examination Board,.New York,N.Y.;' - t Educational Testing Service, Princeton, N.J. 'a- REPORTNO CB-R-81-6; ETS-Rg-81-21 . PUB RATE' 81 NOTE 35p.

: . EDRS PRiCE NW Plus Postage. PC Not Available from EDR§. DESCRIPTORS *Achievement Tests; Asian Americans;'*Basic Skills; Blacks; *Comparative Analysis; Correlation; Higher Education;. Hispanic Americans; Minority Groups; *Predictive Measurement; *Racial Differences; Regression (Statistics); *Sex Differences; Whites IDENTIFIERS English Placement Test; Scholastic Aptitude Test; Test of Standard Written English

,ABSTRACT D Correlation, regression, and score interval analyses . , were conducted for academic tests including theScholastic Aptitude Test (SAT), Test of Standard Written English (TSWE) ,, and foug subtests of the English Placement Test (EPT) for sixdiffetent' groups. The groups were' men, women, Asians,blacks, Hispanics, and ' ,whites. The use of grade point average or individual coursegrades was avoided because these are subjective, often bi&sed, measures.The EPT-Essay score based on a forty-five minutes*ting sample was considered a good standard for comparing other b. sic skills measures available. The measures were evaluated as predic ors ofessay writing and overall performance. The correlational comparisonsshowed few differences across groups, except that correlations tended to be lower for the white sample because of var.ance restrictions. The regression comparisons agreed with previous studies showing blacks and Hispanics,to be generally overpredicted. FalAe negatives,those people who score low on a predictor but who excel on a criterion, occurred least in the,black and Hispanic groups. (Author/DWH)

p 1

*********************************************************************** Reproductions supplied by EDRS are the best that can be made *' from the original document. ***!!*********************************************************** ********

4 US DEPARTMENT OF EDUCATION NATIONAL INSTITUTE OF EDUCATION EDUCATIONAL...RESOURCES INFORMATION College Board CENTER ,ERICI Th.sCO-urne.nt his n rp'o1u en from th pwsr orronlz4, Report On9,nd!rn,4 Morn,' 0,J,yes t11. tA-Po "nip

Po nt , 3 -J No. 81-6 05YY ^n'Int f63 y NIE 009' /

Group CComparisons for Basic Skills Measures Hunter M. Breland Philip A. Griswold

PERMISSION TO REPRODUCE THIS MATERIALIN MICROFICHE ONLY HAS BEEN GRANTED BY,

, P, K

TO TFIE EDUCATIONAL RESOURUES MATION CENTER fEfiqr,

tI=.=41- . 4...... "....0

Comparisons for Basic Skills Measures Hunter M. Breland. Philip JA. Griswold Educational Testing Service I..

College Board Report No. 81 -6 ETS RR No. 81-21

College Entrance Examination Board, New York, 1981

3 Hunter M. Breland andPhilip A. Griswold are staff members of Educational Testing Service,

Princeton, New Jersey. .

Researchers are encouraged to expfess freely their professional judgment. Therefore, points of view or opinions stated in College Board Reports do not necessarily represent official College Board position or )blicy.

The College Board is a nonprofit membership organization that provides tests andother educational services for students, schools, and colleges.The membership is composed oft more than 2,504 colleges, schools, school systems, and educationassociations. Represen- tatives of the members serve on the Board of Trustees and advisory councils and committees that consider the programs of the College Board and participate in the determinationof its policies and activities.

Additional copies of this report may be obtained from College Board Publication Orders, Box 2815, Princeton, New Jersey 08541.' The price is $4.

Abstract iv

Introduction 1

Instruments 2

Procedures

Correlational Analyses Awe 4

Regression Analyses 8

8 . Analyses by Score Levels

Summary 17

References 18

Appendixes

A. Intercorrelation Matrices

B. Regression Analyses: Procedures and Tables 27 1

TABLES

5 1. CorrelatiOns with EPT-Essay Score

5 2. Correlations between the EPT-Reading Score and ATP Scores

6 3. Correlations between the EPT-Sentence Construction Score and ATP Scores

6 4. Correlations between the EPT-Logic and Organization Score and ATP Scores

7 5. Correlations between EPT-Total Score and ATP Scores

6.-Predicting the EPT-Essay. Score and EPT-Total Score from Multiple EPT and ATP Scores 7

7. CoX variance Analyses

10 8. Essay Writing Performance by Group and TSWE Scpre Level

9. Essay Writing Performance by Group and SAT-V Score Level 12

13 10. Esay Writing Performance by Group and EPT-Reading Score

14 11. Essay Writing Performance by Group and EPT-Sentence Construction Score

12. Essay Writing Performance by Group and)OPT-Organization and Logic Score

16 13. EPT-Total Score Performance by Group and TSWE Score Level

ii1

5 I ABSTRACT

Correlation, regression, end score interval analyses were conducted for six academic measures as predictors, of essay writing and overall performance.,,,( Comparisons for all analyses were made for men, women, Asians, blacks, Hispanics, and whites. Thecor- relatiOnal comparisons showed few differences across giOups, except that correlations tended to be lower for the white sample because of variance restrictions.The regression comparisons agreed with previous studies showing blacks and H4panics to bt generally overpredicted. Ori essay-writing. performance, meh were also overpredicted by conventional basic skills measures. In contrast, women tended to write better essays than would have beep predicted by conventional basic skillsmeasures. False negatives, those people who score ,low on a predictor but who excel on a criterion,

occurred least in the'black and Hispanic groups. _ A

6 iv INTRODUCTION a 11

Because of contemporary concerns about fairness to sex and ethnic groups,it has become customary to conduct analyses of academic tests in use orproposed for use to determine if any unfairness may result from such use. A previous study (Breland, 1977b) reported a number of such analyses for the Test of Standard WrittenEnglish' (TSWE), but this report was limited by the available data. Because of the relatively small number of minoiity students in the.sample, it was necessary to combine all minorities into one group for analysis.The present report does not have that limitation.

The most common approach to the examination of fairness is to compare correlational and regression analyses for,,the various eroups of interest.However, these kinds of analyses do not reveal certain kinds of potential unfairness. For example, they do not examine specific groups of people who may score low on a predictor but who perform satisfactorily if allowed to enter an educational program. Such people have been termed "false negatives." Conversely, there exist groups of students who score high on a predictor but who do not perform satisfactorily later. These have been called "false popitives."When false positives and false negatives are summed, they represent "misses" people for whom an erroneous prediction was made. Cor- ,p relational and regression analyses may not reveal the actual consequences represented in false negyives and false positives. For that reason it is important to examine the bivariate distributions of predictor scores and actual performance measures to.understand better the pytential for unfairness in the use of tests.

A second issue in fairness investigations is. whether the outcome being predicted is reliable and unbiased. This concern is commonly referred to as the "criterion problem." The criterion usually used in prediction studies of fairness is the grade point average (GPA). Another useful criterion is performance on some task quite different from the predictor. For example, essays written in English composition courses can be used as a criterion, for comparing performancesfor difrerent groups on multiple- choice tests intended for the prediction ofcollege performance. This kind of criterion has the advantage that it is usually more objective than CPA or specific college course grades especially when essay scores are assigned by readers remote from the instructional setting.

Previous studies of some of the same basic skills measures investigated in the present study have been reported by a number' of writers. Breland (1977a) reported an investigation of the Test of Standard WrittenEnglish (TSWE) and, its use in college English placement for four institutions. The institutions provided data on student performance during college, and Educational TestingService matched these data with other data obtained at the time students applied for college. Student performances on essay tests of writing ability were shown to have a strong relation-, ship to student performance on multiple-choice tests of writing ability as represented by the TSWE. Second, the analyses\Andicated that a brief multiple-choice test of writing ability predicted actual ,ifiting performance during thefreshman year of college as well as or better than a brief essay test given atti he beginning of the freshman year. V As noted above, Breland (1977b) compared men, women, minorities, and non- minorities with respect to performance on the TSWE and subsequent performance in English composition courses as well as performance on brief impromptu essays. No important group differences in traditional correlational analyses for either grade or essay prediction were observed. Analyses of correct and incorrect placement de- c1isions (hits and misses). at specific TSWE cutoff scores revealed no noteworthy group differences whether Outcomes were based on English course grades or on freshman writing performance. From all groups, the TSWE appeared to predict freshman-year writing performance as well as or better than precourse writing samples, high school English grades, or high school rank in class. Because of the limited number of case_k

7 available for analysis, however, it was not possible to conduct analyses within minority subgroups.

Bianchini (1977) presented some basic data for the August 1977 administration of the English Placement Test .(EPT). These basic data included means, standard deviations, KR-20 reliabilities, inter-reader correlations for the essay, and speed and power analyses. Rankin (19!8) visited the 19 campuses of the California State Universities and Colleges (CSUC) using the EPT and reported on its use. This study was onducted to determine how the EPT was being used at CSUC campuses. Several rec endations were made for improvements in instruction and placement. Dunbar,, Minnick, and Oleson (1978) collected data for the Hpyward campus for the purpose of examining the diagnostic capabilities of the EPT, especially for students with minimum writing proficiency. They conducted correlational analyses, multiple.re- . gyession analyses, and discriminant analyses.These analyses indicated that the EPT essay score contributed significantly to the prediction of course performance but that the contribution was not large. Some questions were raised concerning the cost-effectiveness'of the EPT as a result of high observed dorielations between EPT and other measures available.

Michael and Shaffer (1978) collected data at the NorthOdge campus of CSUC and conducted extensive analyses. The purpose of these analyses was to examine comparative validities of the EPT 4nd other measures available for placement as well as comparative validities for different groups. As did Dunbar et al., Michael and Shaffer suggested that there were substantial intercoqelaftons among tests studied. Correlations were also reported between test scores anli fall-semester GPA and grades in a freshman English course. TheEPT-Total score correlated .30 with CPA and .4 with English grades. These correlations were slightly higher, as would be expected, than those for the shorter SAT -y test: .27 with GPA and .41 with English grades. One would expect greater reliability, and consequently higher correlations, using the longer test. Mixed results were obtained from analyses of small groups et black. and Hispanic students, as some results suggested good predictability of course performance and others did not.

In another study Bailey (1978) conducted a factor analysis in which SAT subscores, the TSWE, and EPT subscores were entered. The primary objective in this analysis was to determine if both sets of tests (the EPT and the SAT-TSWEset) would load on a common factor or whether unique properties might beposseSSed by either. A strong common factor was identified accounting, for 80 percent of the variance from all these tests. It was suggested that all the tests have many common properties and that the EPT and the SAT-(including TSWE) may bemeasuring similar abilities.

Instruments

This report presents comparative analyses of seven'academic tests: the Scholastic Aptitude Test (with its verbal and mathematical sections considered separately), the Test of Standard Written English (TSWE), and fqur subtestsof the English PiaCement*Test (EPT). The four subtests are EPT-Reading, EPT-Sentenqe Construction, EPT-Logic and Organization, and EPT-Essay. Each,of the tests is described briefly in the following paragraph's.

SAT-Verbal. This section of the SAT is made up of two verbal sections, each one requiring 30 minutes. The questions measure the ability to understand what has been read, as well as the extent of vocabulary development. SAT-Verbal scores are reported on a scale with a range of 200-800.The national mean for 1980 college- bound seniors was 424, with a standard deviation of 110.

SAT-Mathematical. This has two mathematical sections, each one requiring 30 minutes. The questions measure problem-solving abilities closely related to college work.

2 A

(7) 9

SAT-Mathematical scores are also reported on a 200-800 scale. The national mean for 1980 col-lege-bound 'Seniors was 466, with a standard deviation of 117.

TSWE. The Test of Standard Written English is a 30-minute, multiple-choice test administered with the SAT. The questions eva,luate-,the ability to recognize standard written English, the language of most college textbooks, and the English that college students are usually exiietted to use in papers they write for courses. TSWE scores are reported on a scale of 20 to 60+, with 60+ representing all scores of 60 and above. The national mean for 1980 college-bound seniors was 42.4, with a standard deviation of ILO.'

1 EPT-Reading. A 50-item, 40-minute test designeetomeasureseveral reading skills such as identifying the main idea, interpreting directly or indirectly statedideas, inferring the meaning of words within the context of a reading passage, and recog- nizing, levels of Meaning through figurative phrases. Scores have a range from 120 to 180, with a mean df 150 and a standard deviation of 10:

EPT-Sentence Construction. This is a 50-item, 40-minute test designed to measure how well students can, recognize arrangements of sentence elements that expressthe meaning clearly and correctly. The test has the same scale as EPT-Reading.

EPT-Logic and Organization. Thil is a 50-item, 40-minute test designed to measure the ability to see relationships between words, sentences, objects, andideas. The test has the same scale as EPT-Reading and EPT-SentenceConstruction.

EPT-Essay. A 45-minute essay is written in response to a special topic presented., The essay is scored on a six-point scale by two readers, independently, and.these' two scores are added to yield.a score in the range from 2 to12. In cases of extreme disagreement between readers, a third reader is used toresolve the difference.

Procedures

This report presents comparative analyses of seven academic testsfor six different groups: men, women, Asians, blacks, Hispanics,and whites.2A principal objective in the analyses was to avoid the u3e of CPA orof individual course grades as criteriia, since these are subjective measures and are often biased. Because of the .. objective nature of writing assessment through multiple judgments byreaders of essay samples, the EPT-Essay score based on a45-minute writing sample was viewed as a good standard for comparing other basic skills measuresavailable. Addition-' ally, the EPT-Total score (which includes the essay) was considered a,goodstandard for some analyses becauSe of the large amount of information representedin it.

Correlational analyses were conducted to show basic relationships aming measures for the total sample available and for subgroups as well.RegressionanaTyse,swere conducted to explore further the nature of,the relationShips represented J11 summary form in the correlations. The regression analyses were essentiallyintended.tS determine whether within-group regression lines differed significant for different groups.

Because Correlational/regression' asalyses'as commonlyconducted make aSsumptions of linearity in relationships, analyses were also made at different score.levels as a check on the other analyses. The analyses within score levels.addressed the question: What is the expectation of performance on a criterion measuriven performance within a certain range on the predictor measure?

1. Recent versions of EPT tests require slightlylesa.tetiing time. 2. The analyses are based on data collected by ehe 6aliforla.State ,L4versities and Colleges (CSUC) for students entering in 1977 and matched with College Board data provided by Educational Testing Service.

3 Cprrelational Analyses

1 Tables 1 through 4 show correlations.between the EPT components the EPT-Essay, the EPT- Reading score, the EPT-Sentence Construction score, the EPT-Logic and Organization score and the other available scores. Table 1 would suggest that the EPT-Eltay score is aboUt equally well predicteJ by the other EPT components, the TSWE, and the Jr SAT-V. Only the SAT-M correlationt with the'EPT-Essay score are noticeably lower, as would be expected. This trend is consistent for all groups, except whites. Inter- estingly, the white sample correlations are lowest of all. The lower white corer lations are at.least in part attributable'to the lower white standard deviation (1.71) on the EPT-Essay score. This is somewhat lower than the standard deviations for

, Asians (1.93), blacks .(1.86), and Hispanics (1.94) on the EPT-Essay. Similar differences in variances occurred for most of the other measures studied, and these would further reduce correlations attainable for whites.3

. All the correlations are attenuated somewhat, however, by the generally low re- .. liability of writing sample assessment as represented in the EPT-Essay. As noted in Table 1,the reliability Of a single essay scored independently by two readers was 0 estimated at .38 by Coffman (1966). Breland and Gaynor (1979) used a longitudinal design to obtainalcitest,,,fetest reliability estimate of .52 for a similar,,direct (writing sample) as essment. But Werts, Breland, Grandy, and Rock (1980)' conducted analyses suggesting that the test-retest estimate of .52 may be an overestimate because of the potential of correlated errors of measurement resulting from the writing sample scoring.That is, readers may be systematically influenced by extraneous factors (possibly handwriting quality, neatness, and similar influences). These analyses suggest that the reliability of essay assessment with one sample and two readers is probably in the range from about .35 to about .45.

Table ! shows that the correlational relationships increase somewhat when the multiple-choice EPT-Reading score is ,related to College Board scores (TSWE, SAT-V, SAT-M). SAT-V correlated best with EPT-Reading for all groups. As before, the lowest SAT-V correlations were obtained for the white sample.

In Table 3,the EPT-Sentence Construction score i? correlated with the College Board scores. As with the EPT-Reading score in Table 2, these correlations are generally higher than with the EPT-Essay score. The difference between the EPT- Reading and EPT-Sentence Constriiction scores is that, while the form9r tended to correlate best with SAT -V, the latter tended to correlate best with TSWE. That is, of course, what one would expect.

Table 4Shows the correlational relationships between the EPT-Logic and Organi- zation test and the College Board scores. Here the SAP-NJ appears to be the best correlate, but the white sample, as before, yielded the lowest correlations.

Table 5 relates the composite of the EPT components (the EPT-Total score) and the College Board scores.' These correlations exhibit the same pattern as obtained for the EPT components but are generally higher because of the greater reliability of the EPT-Total score.

In Table 6, EPT and College Board scores are first separately combined to generate a comparative prediction of the essay score for all groups. Second, the EPT-Total score is predicted from the multiple of the College Board scores. These weighted composites appear to predict the EPT-Essay criterion equally well, with only minor variations for some groups. The white sample essay performance is by far the most difficult to predict, and this finding is consistent with all the other correlational analyses. In the analyses td Table 6, it should be noted that SAT-M

'3: See Appendix A, for further details. Note also that in a number of the tables in this report figures for ethnic groups do not add up to the total because identification is not available for all cases.

4 i 0

11 STABLE- 1:--Curkelations with the EPT-Essay Score

a Correlations between the EPT-Essay Score and: EPT- EPT- EPT- Sentence Logic and Reiding Construction Organization TSWE SAT-V SAT-M

Group ti Score Score Score Sco,re Score'Score

Total 10,67 .46 .49 .46 - .45 .23

Men ,766 .43 .48 J.45 .48 .44 .29

Women 5,908 .49 .51 .48 .50 .49 .32

Asian 606 .57 .59 .54 .55 .52 .16

Black 583 .53 .56 .56 .56 .53 .30

Hispanic 445 .5Q .55 .49 .53 .50 .28

Aite 5,236 130 .35 .29 .37 .32 .12

a. Interpretation of these correlations should be madewith an awareness that they are somewhat attenuated due to the low reliability of the essaycriterion itb measure. Coffman (196) has estimated that the score reliability of a single Usay scored independently by two readers (the case for these data) is about.38. Corrections for attenuation for relthbility in the criterion wouldincrease a correlation of .50, for example, to .81..

TABLE 2. Correlations between the EPT-Reading Score andATPa Scorj-s?

a ATP Scores

Group TSWE SAT-V, SAT-M

Total 10,674 :67 :74 .46

Men 4,766 .64 .72 .46

Women 5,908 .69 .75 Jr .50

%35 Asian 7606 .73 .80

Black .66 .76 .51

Hi'spanic 445 .66 .73 .46

White 5,236 .65 .36

a. The Admissions Testing Programof the College Board, represented here by SAT-V, SAT-M, and,TSWE.

5 1 TABLE 3. Correlations'between the EPT-Sentence Construction Score andAPa Scores

ATPa Scores

Group TSWE SAT-V SAT-M

4 Total 10,674 74 .70 .49

Men 4,766 .,72 .68' / .51

Women 5,908 .76 .72 .54

-Asr-iau______606 ..77 .78 39

Black 583 --;--7-5--__.______71 .54 -- Hispanic 445 :71 .72 ------;-53-----____

White 5,236 .66 .60 .39

a. The Admissions Testing Program of theCollege Board, represented here by SAT-V, SAT-M, and TSWE.

a TABLE 4. Correlations between the EPT-Logic and Organization Score and ATP Scores'

ATPa Scores.

Group N TSWE SAT -V SAT-;1

, Total 10,674 ' .67 .71 .52

Men 4,766 .64 .70 : x.53 -- Women 5,908 .69 .73 .54 ------,,

Asian 1,./606 .73 .77 .38 , ) Black , V583 .68 . .77 .60

i Hispanic 445 .64 :71 .51

White 5,236 .55 .61 .42

a. The Admissions Testing Program of the College Board, represerited here by SAT-V, SAT-M, and TSWE.

6 TABLE 5. qarrelatlions between EPT-Total Score and ATPa Stores

a ATP Scores

Group N TSWE SAT -V SArM

Total. 10,674 .76 IN .77 .51 0

Men 4,766 .74, . :76 .54

..Women 5,9(18 .77 .79 .56

Asian 606 .79 .82 .37

Black 583 .77 .80 .57

Hispanic 445 .74 .78 .52 '1

Wblte 5,236 .68 .68 .40

a. The Admissions Testing Program ofthe College Board,represented here by SAT-V, SAT-M, and TSWE.

TABLE 6. Predicting the -Essay Score and EPT-Total Score fromMultiple EPTaand ATPb Scores

Multiple Correlation, EPiesay Score and: Multiple Correlation, b EPT-Total Score and ATP Group' EPT r ATP

1 Total 10,674 .51 .51 .83

Men 4,776 .50 .50 .82

Women 5,408 .53 .53 .84

Asian 606 .61 .58 .86

Black 583 .60 .60 .86

Hispanic 445 .57 .56 .84 4 4f

White 5,236 .37 .39 .75

a. Reading, Sentence Construction,and Logic and Organization. b. TSWE, SAT-V, and SAT-M.

7 clOtributedvery little to the multiple correlation coefficient usually only about one additional hundredth. The final column of Table 6 shows the substantial relationship between the EPT-Total score and the composite of the ATP components.

Regression Analyses

The above correlational analysesdemon/trate the degree of relationship among the several tests, under the assumption that relationships are linear. An examination of the nature4of these linear relationships, however, requires analyses of the regression systems related to the correlations.

Comparisons of regression systems for different groups were conducted through analyses of covariance. Under assumptions of linearity, regression lines are parallel (i.e., the slopes are equal) when no interaction iS inferred between groups and predictors. That is, a unit increase in the predictor produces a pro- portionate increase in the criterion regardless of group membership. Second, if the lines are coincident (i.e., the slopes and intercepts are equal) the equation for one group is essentially the same as the equation for the-other groups. The statistical procedures and other oetails of the covariance analyses are given in Appendix B.

The results of the covariance analyses are given in Table 7. In every ethnic comparison with whites on either TSWE or SAT-V, statisticallysignificant interactions were found (p '.01).. However, tFte small changes in the multiple R's shown in Table 7, as group and interaction influences were added, suggest that the interaction 6 effects were small. When the nonessay components of the EPT were used to predict scores on the EPT-Essay, no significant interactions were found for blacks and whites, but tests of coincidence of the two regression lines showed significantdifferences (p <.01). This suggests that the white regression overpredicts for blacks throughout,the range of EPT scores. A similar pattern is evident for prediction of Hispanic scores. Some interaction is present, but the white regression tends to overpredict Hispanic performance. The pattern for prediction of Asian scores iS the least dis- tin4t, although significant interactions do exist. Again, the multiple R's suggest little influence of group or interaction factors.

The re'sroits for the regression comparisonof men and women are quiteclear. No interaction exists for TSWE, SAT-V, or the EPT component predictors. Since the slopes,areparallel and aff eitests of coincidence are significant,(p <.01), the litregression suggests that higher at all score levels than men. Thus, the essay writing performance 40.1'would tend to be underpredicted (and that of men overpredicted) by a linear e on based on data for both sexes combined.

Analyses by Score Levels

While the* regression comparisons just described were not indicative of important .group d1Sferences, the statisticallysignificant' interactions and intercept differences are intriguing and worthy of further exploration. One approach toward ex- ploring the nature of any group differences is to conduct analyses within score levels. Such a procedure allows for the detection of nonlinearities in the data and also provides a sense of thepractical importance of group differences.

Table 8 presents group comparisons of EPT-Essay performance for fourdifferent score intervals of the TSWE. The percentages shown at the bottom of Table 8 are the observed proportions of group members within a given score'rangeWho wrote better-than-average essays. Tests of significance were based on the percentages for the total frequencies at the left. Thus, for example, it would be expected that' 70.3 percent of those in the sample scoring 50 or above on the TSWE would write better -than- average essays. The tests of significance show that women scoring 50

A X

'FABLE 7. CoVariance Analyses

R R F F Using R Adding for Inter- for Coinci- Groups Regressions Basic Adding Inter- action (Test dences (Test Compared Compared N Predictor Group action of Slope) of Intercept)

lo Blacks vs. whites Essay on TSWE 5,819 .44 .45 .46 25.4* Blacks vs. whites Essay on SAT-V 5,819,,.40 .41 .42 42.8* Blacks vs. whites Essay on EPT-R 5J".40 .40 .40 1.8 15.0* Blacks vg. whites Essay on EPT-SC 5,819 .44 .44 .44 1.3 11.9* k Blacks vs. whites Essay on EPT -L &0 5,819 .40 .40 .40 1.9 8.4*

Hispanics vs. whites/ Essay on TSWE 5,681 .41 .41 .42 19.6* Hispanics vs. whitet Essay on SAT-V 5,681 .36 .37 .38 28.6* Hispanics vs. whites Essay on_EPT-R 5,681 .35 .35* .36 5.6 18.5* Hispanics-,va. whites Etsay on EPT-SC 5,681 .40 .40 .41 5.1 13.5* Hispanics vs. whites Essay on EPT -L &0 5,681 .34 .35 .35 3.8 15.3*

Asians vs. whit-es Essay on TSWE 5,842 .41 .42 .42 18.0* Asians vseWhiCes Essay on SAT-V- 5,842 .06 .37 .38 25.7* Asians vs. whites E4Say on EPT-R 5,842 .37 .37 .37 11.5* Asians vs. Whitei Essay, on EPT-SC 5,842 .41 .41 .41 4.9 7.5* Asians vs. whites Essay on EPT -L &0 5,842 .35 .36 .36 7.2*

Blacks vs. Hispanics Essay on TSWE 1,028 .56 .56 .56 <1 <1 Blacks vs. Hispanics Essay on SAT-V 1,028 .53 .53 .53 <1 1.0 Blacks Is. Hispanics Essay on HET -R 1,028 .52 .52 .52 <1 1.2' Blacks vs. Hispanics Essay on EPT-SC 1,028 .56 .56 .56 <1 <1 Blacks vs. Hispanics Essay on EPT -L &0 1,028 .53 .53 .53 ? 2.3

Mb vs. women EsSay on TSWE 10,674 :49 .51 .51 <1 171.1* Men vs. women Essay on SAT-V 10,674 .45 .49 .49 ,1 279.3* Men vs. women Essay on EPT-R 10,674 .46 .49 .49 <1 224.4* Men vs. women Essay on EPT-SC 10,674 .49 .52 .52 <1 193.0* Men vs. women ESsay on EPT-L&O 10,674 .46 .49 .49 <1 242.9*

a. Where dashes are indicated, the standard practice was followed of not performing tests of coincidence when the interactions are significant. *2<.01 (Although significance occurs among the groups, note that little increase in R was found. This may be due to 'slight nonlinearity of the data qmong the first three groups and, in the last group, to the very large N.)

.1" 15 f

TABLE 8. egsay Writing Performance by Group'1andTSWE Score Level

TSWE

Score Total Men Women '" /titian Black Hispanic White

Frequencies Scoring in Four TSWE_Ranges

33 35 1,738 . 50+ 2,855 1 171 1,684 87

, 40-49 3,914. 1,724 2,190 206 113 146 2,131

30 -39 2,659 '1,300 1,359 176 196 159 1,115

--4 Below 30 1,246 571 6751 137 241 105 252

Frequencies Writing Above-perage Essays

50* ""--y5006 732. 715284 51 23 26 1,229

40-49 2,087 776 1,311 101 62 80 1,136

1 30-39 850 341 509 60 40 32 408

Below 30 167'' 58 109 15 22 14 58

Percentages Writing Above-Average Essays

50+ i70.3 1r2.5* 75.6* 58.6' 69.7 74.3 70.7

40-49 53.3 , 45.0* 59.9* 49.0 54.9 54.8 53.3

. 30-39 32.0 26.2* 37.4* 34.1 20.4* 20.1* 36.6*

Below 30 13.4 t X10.2* 16.1* 10.9 9.1* 13.3 23.0*

*Statistically significant (p .05) deviation from expected percentage.

10 ti I/4 - or abOve on TSWE wrote significantly more above-average essays (75.6 percent) than expected and that men and Asians who scored 50 or above on the TSWE wrote significantly fewer above-average essays than expected. The differences between men and women, moreover, occurred at all four TSWE score levels. Blacks and Hispanics scoring below 40 on TSWE tended to wr.\Ite fewer above-average essays than expected, though the 13.3 percent figure for His anics in the lowest (below 30) range was not statistically significant.

Conversely, whites scoring below 40 oh the TSWE tended to write more above - average essays than would be expected for people in those score ranges. The analyses represented in Table 8 are useful as explanations of the interaction and intercept differences reported in Table 7. The differences between men and women are consistent and are highlighted by Table 8, whereas the multipleR's.pf.-Table 7 , tended to mask the N

Ii r i /Table 9 shows similar analyes for the SAT -V as a predictor. The pattern is quite similar to that observed for the TSWE in Table 8. Women consistently write ' better essays than men in the same SAT-V score intervals, and whites who score below 400 on the SAT-V write better essays than blacks and Hispanics with SAT-V 'scores in the same intervals.

Tables 10, 11, and 12 present comparable analyses f .-EPT- Reading, EPT-Sentence Comtruction, and EPT-Logic and Organization as predictors of essay writing iprform- ance. Like the TSWE and SAT-V, the EPT components'tend to underpredict essay per- , formance for women and to over dict for men. And blacks and Hispanics scoring in the highest EPT intervals ten to write fewer above-average essays than would be expected from their multiple-ch ice test performance.

Tables 8 through 12 may aIs6 be considered as analyses of false negatives and false positives. Those scoring low on a predictor but writing above-average essays are false negatives, and those scoring high on a'predictor but writing below-average essays are false positives. In Table 8, for example, the highest proportions of fal,se negatives predicted by TSWE are for women and whites, and the lowest propor: tions for men, blacks, and Hispanics. Ip Table 9 tile same pattern is shown with the SAT -V .as a predictor. Tables 10 through 12 also show this pattern.One should note, however, that what' is "low" for the EPT components is different from what is low for SAT-V and TSVE.*.Thelaverage score for the EPT components is about J50, so that most of the Figures- in Tables 10, 11, and 12 are for relatively low EPT scores.

411. Another way of viewing the issue'of false negatives is to consider it in con- nection with the EPT-Total score. Even though the correlational comparisons presented previously show substantial relationships between the EPT-Total scbssi6e d the other measures, such analyses do not examine performance by group withi s e- cific score ranges, Since the previous analyses indicated few differences among the various tests, it is enough to limit this lAst comparison to one of the tests. Table 13 presents a comparison of TSWE and EPT-Total scores for all the groups.. This table illustrates the strong relationship between these two tests, which was demonstrated before by the high (.76) correlation reported in Table 5. For those students scoring eor above on the TSWE, 96.0 percent were above avenge(150) on the EPT-Total score. re are some variations across groups, with only 87.9 percentof the black sample with high TSWE scores scoring above average on the EPT-Total, but these differences are not statistically significant. At the other extreme of the TSWE distribution (below 30), only 6.9 percent of those students entering CSUC in 1977 obtained above- average EPT-Total scores. Blacks scoring below 30 on the TSWE seem least likely to perform well on the EPT, since only 2-.5 perCent of blacks scoring below 30 on the TSVF had above-average EPT-Total scores. These comparisons of EPT - Total, scores and TSW scores lead to essentially the same conclusion as the previousanalyses. The smallest proportiOns of false negatives tend to occur for the black and Hispanic groups. The one noticeagle difference between the pattern of Table 13 and thatof ether comparisons is that there were no statistically significantdifferencebetween men and women.

11 17 TABLE 9. Essay Writing Performance by Group and SAT -y Score Level

SAT-V ,Score Total 'Men Women\, Asian Black Hispanic White

Frequencies Scoring in Four SAT-V Ranges

500+ 21'378 1,093 ,1,285 85 18 38 1,442

400-490 4;246 2,000 2,246 208 93 126 2,312

300-390 3,076 1,330 .1,746 196 238 191 1,315

. , Below 300 974 '343 631 117 234 90 147

Frequencies Writing Above-Average Essays

A. 500+ 1,640 651 989- 53 10 30 1,004 , 400-490 2,257 876 -.1,381 103 50 62 1,256

543 300-390 1,105 353 752 66 70 56//

34 Below 300 108 27 81' 11 17 7

Percentages Writing Above-Average Essays

69.6 500+ 69.0 59.6* 77.0* 67.3 55.6, 78.9

49.2 53.8 400-490 53.2 43:8* 61.5* 49.5 53 8'

. 243* 41.3* 300-390 35.9 26.5* 43.1* 33.7

7.8 23.d* Below 300 11.1 .7.9* 12.8 9.4 7.3*

*Statistically significant (p < .05) deviation fromexpecid percentage. J

12 p

TABLE 10. Essay Writing Performance by Group and EPT-Reading Score

EPT- Reading Score Total Men Women Asians Blacks Hispanics Whites

Frequencies Scoring in Four EPT-Reading Score Ranges

150+ 6,522 2,934 3,588 272 132 165 3,773

140-14.9 2,978 1,352 1,626 192 203 191 1,262

130 -139 716 294 422 75 120 52 168 . Below 130 458 186 272 67 128 37 33

Frequencies Writing Above-Average Essays

64 92 2,267 150+ 3,839 4 1,449 2,390 164

140-149 1,116 407 709 60 60 54 - 521

43, 130-139 133 47 4 :.8'6 17 13 7

Below 130 22 4 18 2 5 2, 6

Percentages "Writing Above- Average Essays

49.4* 48.5* 55.2 60.1 . 150+ 58.9 66.6*

140-149 37.5 30.1* 31.2 29.6* 28.3 .41.3

18.6 16.0 20.4 22.7 10.8* 13.5 25.6*

3.9 5.4 18.2* Below 130 4.8 2.2 , 6.6 3.0

*Statistically significant (p.' .05) deviation from expected percentage.

a 13

19 ABLE II. Essay Writing Performande by Group, and EPrSentence4onstruction Score

EPT-Sentence Construction Score Total Men Women Asians Blaqlts Hispanics Whites

Frequencies Scoring in Four EPT-Reading Score Ranges

150+ 7,050 3,092 3,958 307 166 200 4,006

140-149 2,279 1,083 1,196 154 163 131 979

130-139 900 410 490 77 142 78 203 Below 130 445 181 264 68 112 '36 48 / Frequencies Writing Above-Average Essays

150+ 4,171 1,571 2,600 170 84 106 2,527

140-149 758 279, 479 47 44 38 350

130-139 155 49 166 9 14 9 53

2 5 Below 130 26 8. 18 , 7 5

Percentages Writing Above - Average Essays

1?0+ 59.2 50.4* 164.2* 55.3 50.6* 53.0 63.1*

140-149 33.3 25.8* fi 46.9* 30.5 27.0 29.0 35.8

, 4( 17.2 12.0* 22.8* 11.7 9.8* 11.5 26.1* VC)-19

.Be16{4130 5.8 4.4 6.8 10.3 4.5 5.6 10.4

*Statistially significant (p <.05) deviation from expected per.centage. , 0

Jo'

14 20 TABLE 12. Essay Writing Performance by Group and EPT-Organization and Logic Score

EPT-Organization and Logic Score Total Men Women Atians Blacks Hispanics Whites

Frequencies Scoring in Four EPT-Reading Score Ranges

150+ 7,114 3,210 3,904 344 143 201 4,024

140-149 2,237 1,015 1,222 135 152 139 961

130-139 781 325 456 62 123 61 202

Below 130 542 216 326 65 165 44 49

Frequencies Writing Above-Average Essays

150+ 4,105 1,554 2,551 171 ,73 97 2,390

140-149 838 308 530 42 53 46 396

130-139 131 36 95 \ 11 11 3 44

Below 130 36 9 27 4 10 6 7

Percentages Writing Above-Average Essays

150+ 57.7 48.4* 65.3* 49.7* 51.0 48.2*

4. 140-149 37.5 30.3* 43.4* 31.1 34.9 33.1

13gr139 16.8 11.1* 20.8* 17.7 8.9* 4.9*

Below 130 6.6 4.2 8.3 6.2 6.1 13.6

*Statistically significant (p <.05) deviation from expected perce

4 TABLE 13. EPT-Total Score Performance by Group and TSWE Score Level

TSWE Score Range Total Men Women Asian Black Hispanic White

Frequencies in Four TSWE Score Ranges

50+ 2,861 1,175 1,686 87 13 35 1,743

40-49 3,917 1,725 2,192 206 113 147 2,131

160 1,118 30-39 2,666 , 1,304 1,362 176 196

Below 30 1,275 587 688 148 243 107 254

Frequencies with Above-Avdtage EPT-Total Scores

50+ 2,748 1,119 1,629 85 29 32 '1,683

40-49 3,000 1,290 1,710 143 58 102 1,695

30-39 972 449 5.23 60 33 4.- 38 517

Below 30 88 38 50 . f5 6 5 38

Percentages with Above-Average EPT-Total Scores

56+ 96.0 95.2 96.6 97.7 87.9 91.4 96.6

40-49 76.6 74.8 78.0 69.4* 51.3* 69.4 79.5.

46.2* 30-39 36.4 34.4 38.4 34.1 4 16.8* 23.8*

Below 30 6.1 ,6.5 7.3 .4 I 2.5* 4.7 15.0*

*Statistically nificant (p .05) deviation from expected pprcentage.

16 SUMMARY

.Seven tests were compared and contrasted with respect to interrelationships for six different groups. Special emphasis was placed on the use of an essay. The EPT-Essay score was used as a common criterion for making comparisons of the EPT nonessaycom- ponents with the College Board's SAT-V, SAT-M, and "TSWE.The EPT-Total score was also used as a criterion for some comparisons. Correlation, regression, and score- interval analyses were conducted.

The correlational analyses demonstrated the close relationship among SAT-V, TSWE, and EPT scores. There were few'correlational differences across groups, with the exception that the white sample consistently yielded lower correlations for all comparisons than did .the other groups. Whether the relaticinship being examined was with the EPT-Ess4y score, the EPT-Reading score, the EPT-Sentence Construc- tion score, the EPT-Logic and Organization score, or with the EPT-Total core, the correlation for the white sample was always the lowest.The lower wh to correlations were attributed to attenuated variances in the white simple for m st scores. A second interesting observ'ation was that of the relationship between the success of multiple componenti of the EPT and College Board's tests as preqfctors wi h the EPT-Essay score as a common.criterion Both sets of components predicted EPT-Essay ,performance equally well for all" groups ,ftFinally, the EPT-Total score was seen to and bewell,-' predicted by the multiple set,d(College'Board scores (SAT-V, SAT-M, TSWE), even though the SAT-M contributed only minimally to this predictions LsA The regression analyses indicated the nature of relationships among the various test scores. The EPT-Essay score was used as a common criterion, and all groups were ,compaLed and contrasted in regard to the nature of the predictive relationship for each of the nonessay components studied. Both the TSWE and the SAT-V were seen to overpredict minority performance in essay writing. Similarly, .the EPT nonessay components tended to overpredict minority performance in essay writing. The obser- vation of overprediction of minority performance has been a common one in previous studies of all types of tests. The regression comparisons for men and women also followed a pattern often observed in previous studies. Throughout the score ranges for TSWE, SAT-V, and all three EPT nonessay components, t,omen were consistently underpredicted. In other words, women consist&itly per rmed better on essay writing tasks than the test scores in this analysis indicate.

The score interval analyses further illustrated the predictive. relationships. The EIT-Total score appeared to be quite closely related to TSWE scores, as96 percent ofy.those scoring 50 or above on TSWE obtained above-averageEPT-Total scores, but only 7 percent-of those scoring below 30 on TSWE obtained'above-average EPT-Total scores.'These descriptive. comparisons were similar for all groups. Distributions were also used to determine the proportions of false negativesfor different groups

in different TSWE score ranges, using both the EPT-Essay and the EPT-Total score as . criteria. These analyses showed that blacks and Hispanics tended to have the smallest representations of false negatives, while women had the most. The results 'obtainel were in agreement with similar, previous studies. One interesting new finding was observed, however. Although women consistently out-performed men on essay writing, they did not do so when the criterion consisted primarily of multiple-choice measures, as in the EPT-Total score.

17,

el 40o0 REFERENCES

V r Bailey, R. L. "Principal Components Analysis of the Old and theNew: The SAT vs. the CSUC English Placement Test," Measurement and Evaluation in Guidance,11:3 (1978): 162-168.

1 Bianchini, J. C. The California State Universities and Colleges English Placement Test (Educational Testing Service Statistical Report SR-77-70). Princeton: Educational Testing Service, August 1977.

Breland, H. M. A Study of. College English Placement and the Test of Standard. Written English (ETS Project Report PR-77-1, College Board Research and Development Report RDR-76-77-4). Princeton: Educational Testing Service, January 1977a.

Breland, H. M. Group Comparisons for the Test of Standard Written English (ETS Re- Educational Testing Service, August 1977b. . search Bulletin RB-77-15). Princeton:

Breland, H. 4., and J. L. Gaynor. "A Comparisonof/Direct and Indirect Assessments -\ Writing Skill," Journal of Educational Measurement 16:2 (Summer 1979): 119 -128. Clea'ry, T. A. "Test Bias: Prediction of Grades of Negro and White Students in Integrated Colleges," Journal of Educational Measurement, 5:2 (1968): 115-124%

Coffman, W. E. "On the Validity of Essay Tests of Achievement,IfJournal of Educa- tional Measurement, 3:2 (Summer 1966): 151-156.

Dunbar, C. R., L. Minnick, and S.' Oleson. Validation of the English Placement Test at CSUH (Student Services Report No. 2-78/79).Hayward: California State University, October 20, 1978.

Gulliksen, H., and S. S. Wilks. "Regression Tests for Several Samples,"Psychometrika, 15:2 (1950): 91-114. rHumphreys, L. "Implications of Group Differences for Test Interpretation,"Rroceedings of 1972 Invitational Conference on Testing Problems. Princeton: Educational Testing Service, I9731 pp. 56-71.

KleAnbaum, U.G., and L. L. Kupper. Applied Regression Analysisand Other Multi- variable Methods. North Scituate, Massachusetts:Duxbury Press, 1978.

Michael, W. B., and P. Shaffer. "The Comparative V4lidity of the CaliforniaState Universities and Colleges English Placement Test (CSUC-EPT) in the Predictionof Fall Semester Grade Point Average and English Course Gradesof First-Semester Entering Freshmen," Educational and Psychological Measurement,38:4 (1978): 985-1001.

Michael, W. B., and P. Shaffer. "A Comparison of the Validity the Test of Standard Written English (TSWE) and of the California State Universities andColleges English Placement Test (CSUC-EPT) in the Prediction of Grades in a BasicEnglish Composition Course and of Overall Freshman-Year Grade PointAverage," Educational and Psychological Measurement, 39:1 (1979): 131-145.

Murph , N. C. A Preliminary Validation Study, the English Placement Test, California State Universities and Colleges System, Academic Year 1977-78. San Luis Obispo: California Polytechnic State University, September 1978.

Nie,, N. H., C. H. Hull, J.'G. Jenkins, K. Steinbrenner, and D. H. Bent. Statistical Package for the Social Sciences, .2nd Edition." New York:McGraw-Hill Book Company, 1975.

18 Overall, J. E., and D. K. Spiegel. "Concerning least Squares Analysis of Experimental Data," Psychological Bulletin, 72:5 (1969):' 311-327.

Rankin, D. The Reception, Applications, and Impact of the English P ce nt Test in the California State Universities and Colleges. Long Beach, ,Cal fornia: Office' of the Chancellor, August 1978.

'Reschly, D. J., and D. L. Sabers. "Analysis of Test Bias in Four Groups Iith the Regression Definition," Journal af'Educational Measurement, 16 :1. (1979): 1-9.

Temp, G. "Validity of the SAT for Blacks and Whites in Thirteen Integrated Institu- tions," Journal of Educational Measurement, 8:4 (1971): 245-251.

11 WertS, C..E., H. M. Breland, J. Grandy, and D. Rock. "Using Longitudinal Data to Estimate Reliability in the Presence of Correlated meas6reme41 Errors," Educa- tional and Psychological Measurement, 40:1 (1980): 19-29.

Alb

1/4

i 1.

TABLE A-1. Intercorrelations of EPT and ATP Components for Total Sample (N=1 z H Standard Test Mean Deviation 1 2 3 4 5 6 7 8

1. EPT-Essay 7.31 1.85 1.00 .46 .49._%-\.46 ,71 ,45' .23 .49

2. EPT - Reading 150.17 9.14 .46 1.00 .76 .79 .89 .74 .46 "' .67

3 EPT-Sent,ence Construction 150.40 9.32 ..49 .76 1.00 .76 .89 .70 4.49 .74

4 EPT -Logic 150.10 ,9.36 '.46 .79 .76 1.00/ .89 .72 '.52 .67 Y .' .51 .76 5 EPT -Total 150.18 7.84 .71 .89 ''. .89 .89 1.00 .77

6 SAt-Va 42.42 9.50 .45 .74 .70 .72 .77 1.06. .54 .72

7. SAT-Ma 47.40 10.64 .23 .46 .49 .52 .51' .54 1.00 .49 .

8. TSWE - 42.44%9.95 .49 .67 .74 '.67 .76 .72 .49 1.00

a. Note that the scale used for these purposes has beentruncated to 20 to 80, rather than the usual 200 to 800 scale for reported, scores.

010 . I TABLE A -2. Intercorrelations of EPT and ATP Components for Men (N=,4 766)

14 Standard d ,Test Mean Deviation 1 2 3 4 5 6 7 8

. .

1'. EPT-Essay 6.95 1.83 1.00 .43 .48 .45 .71 .44 .29 * .48 4

2. EPT-Reading 150.2 8.85 ..43 1:00 .74 .78 .88 .72 .46,: .64

..." 3. EPT-Sentence Construction 150.13 9.08 .48 .74' 1.00 .74 .88 .68 .51 .72

4. EPT-Logic 150:33 9.10 .45 .78 .74 1.00 .88 .70 .53 .64

EPT-Total 149.73 7.56 .71 .88 .88 .88 1.00, .76 .54 .74

a 6. SAT-V 42.96 9.21 .44 .72 .68 .70 .76 1.00 '.53 .71

7. SAT -Ma 50.83 10.6% .29 .46 .51 . .53 .54 .53 '1.00 ,51

8. TSWE 41.94 9.82 .48 .64 :02 .64 . .74 .71 .51 1.00 4-

a. Note that the scale used for these purposes has been truncated to 20 to 80, rather than the usual 200 to 800 scale for reported scores,

JP. 2" TABLE A-3. Intercorrelations of EPT and ATP'Components for Women (N=5,908)

Standard Test Mean Deviation 2 3 4 5 6 7 '8

1. EPT-Essay,..., 7.60 1.82 1.Q0 .49 .51 .49 .72 .49 .32 .50

2. EPT-Reading 150.12 9.37 .41 1.00 .78 .81 .90 .75 .50 .69

3. EPT-Sentence

Cons.t..46tion 150.63 9.50 .51 .78 1.00 .78 ..90 . .72 .54 .76

/ 4. EPT-Logic 149.92 '9.56 .49 .81 .78 1.00 .90 .73 .54 .69 /

5. EP?-Total 150.54 8.03 .72 .90 .90 .90 ( 1.00 .79 .56 .77

a SAT-V 41.98 97.01 .49 .75 .72 .73 .79 1.00 .57 .74 6. ) 7. SAT-Ma '44.63 97.56 .32 .50 .54 .54 .56. .57 1.00 .54

8. Tswg 42.85 10.05 .50 .69 .76 .69 .77 .74 .54 1.00

a. Note that the scale used for thesepurposes has been truncatedto20 to 80,rather than the usual 200 to 800 scale for reported scores.

ka est

TABLE A-4. Intercorrelations of EPT and ATP Components for Asian Sample (N=606)

Standard .Test Mean Deviation 1 2 3 4 5 6 7 '8

1. EPT-Essay 6.82 1.93 1.00 .57 .59 .54 :75 .52 .16 .55

2. EPT - Reading 145.82 11.14 .57 1.00 .82' .85 .92 .81 .35 .73

3. EPTzSentence4 Construction 146.23 11.39 .59 .82 1.00 .83 .93 .. .78 .39 .77

4. EPT-Logic 147.08 11.21 .54 .85 .83 1.00 .92 .77 .38 .73

5. EPT-Total 146.68 9.61 .75 .92 .93 .92 1.00 .82 .37 .79

a 6. SAT-V 38.90 9.89 .52 .81 .78 .77 :82 1.00 .46' .76

a 7. SAT-M 49.27 9.81 d .16 .35 .39 .38 .37 .46 1.004h. .41

8. TSWE 38.16 10.47 .55 .73 .77 .73 .79 .76 .41 1.00

a. Note that the scale used for these purposes has been truncated to 20 to 80, rather than the usual 200 to 800 scale for reported scores.

20 TABLE A-5. Intercorrel4ions of EPT and ATE Components for Black Sample (N=583)

Standard Test Mean- Deviation 1 2 3 4 5 7 8

1. EPT-Essay 6.20 1.86irs 1.00 .53 .57 .56 .53 .30 .56

2. EPT-Reading 139.91 11.58 .53 1.00 .76 .80 .90 .76 .51 .66

,-,s. . ..

3. EPT- Se,ntence . Construction 140.68 .11.25 :57 .7( 1.00 .75 .89 .71 .54 .75' 0

4. EPT-LogiC 138.31 12.24 .56 1 .80 .75 1.00 .91 .77 .60 .68

.77 5. EPT-Total 140.84 9.65 ,,-- .74 .90 .89 .91 1.00 .80 .57

6. SAT -Ca 32.40 8.25 .53 .76 .71 .77 .80 1.00 .61 ,69 a. 7. SAT-M 34.84 7.91 .30 .51 .54 .60 .57 .61 1.00 .55

.75 . .77 .69' .55 1.00 . 8. TSWE 32.79 9.46 .56 .66

a. Note that the scale used for these purposes has been truncated to 20 to80, ratIet than the usual 200 to 800 scale for reported scores.,

30 TABLE A-6. Intercorrelations.of EPT and ATP Components for Hispanic Sample (N=445)

Standard Test 'Mean Deviation 1 2 3 4 ' 5 6 7. 8

1. EPT-Rssay 6.60 1.94 1.00 .49 .55 .49 .74 .50 .28 .53

2. EPT-Reading 145.61 9.93 49 1.00 .75 ' .77 .88 .73 .47 .66

3. EPT-Sentence r-- Construction 145.53 10.30 .55 .74 1.00 .76 .90 .72 .53 .'71

' 4. EPT-Logic 145.34 10.49 .49 .77 .76 1.00 .89 .71 .51 .64

5. EPT-Total 145.74 8.62 .74 .88 .90 .89 1.00 .76 .52 .75

.70 6. SAT-e 37.02 8.45 .50 .73 .72 .71 .4'.78 1.00 ° .50

7. SAT-M 40.13 8.90 .28 .47 .53 .51 .52 .50 1.00 .45

8. TSWE 36.74 9.40 .53 .66 .71 .64 .75 .70 .45 1.00

a. Note that the scale used for these purposes has bell truncated to 20 to80,rather than the usual 200 to 800 scale for reported scores.

a SS

4 31 TABLE A-7. Intercorrelations of EPT and ATP Components'forWhite Sample (N=5,236)

Standard

Test Mean Deyiation 1 2 3 , 4 5 6 7 8

1. EPT-Essay 7.64 1.71 1.00 .30 .29 .67 .32 .12 .37

2. EPT-Reading 152.74 6.63 .30 1.00 .63 .66 .81 .65 .36 .55

3. EPT-Sentence Construction 152.94 7.06 .35 ,,../(3 1.00 .64 .83 .60' .39 .66

4. EPT-Logic 152.69 6.58 .29 .66 .64 1.00 .81 .61 .42 .55

5. EPT-Total 152.51 5.55 .67 .81 .83 .81 1.00 .69 .40 .68 i 6. SAT-Va 44.72 8.50 .32 .65 .60 .61 .69 1.00 .45 .64

a <. 7. SAT-M 49.32 9.82 .12 .36 .39 .42 .40 .45 1.00 .39

8. TSWE 45.01 8.69 .37 .55 .66 .55 .68 .64 .39 1.00

a. Note that the scale used for these purposes has beed truncated to 20 to 80, rather than the usual 200 to 800 scale for reported scores.' APPENDIX B. REGRESSION ANALYSES: PROCEDURES AND TABLES

Previous research comparing regression systems in group comparisons often has not specified precisely how the comparisons were made. This note is intended to rectify that situation at least for this report.

If the regression of a criterion on a test score is6t;e'samefor different grou0, the test is an unbiased predictor of the criterionItior those grotips. In the present study, to determine the validity of predicting tO.iNT-Essay score from TSWE, SAT-V, and EPT component scores for various ethnic gro4s and for men and women, multiple regression analysis was utilized. The procedulte is similar to the rationale of Gulliksen and Wins. (1950), which is based on evaluation of the quality of errors of estimate, of slopes, and of intercepts. This general methodology has been frequently used in test bias studies (e.g., Cleary, 1968; Humphreys, 1973; Reschley and Sabers, 1979; Temp, 1971).

The.keneral linear model technique of the software provided by the Statistical Package fo/ the Social Sciences (Nie, Hull, Jenkins, Steinbrenner, and Bent, 1975, pp. 381-38P) was used because of its convenience and availability' to most researchers. Since multiple regression may include dummy variables as predictors, pairs of ethnic groups to be compared were used as dichotomous, categorical variables. A least squares approach to analysis of covariance similar to the method employed by Cleary (1968) permitted testing of effects of the interaction of test score by ethnicity and of the effect of ethnicity, holding test score constant.

The single multiple regression for the two group categories (Z = 0,1), score on TSWE (X), the EPT-Essay criterion (Y'), and the interaction between'ace and TSWE (ZX) was developed from the rationale of Kleinbaum and Kupper (1978, Chapter 13) as follows:

Y' = BO + B1X ± BZ + B(ZX) +4Errot..16* 2

For blacks (Z 1) the model reduces to: /) Y ' = (B + B) + (B + B ) X + Error; B 0 2 1 3 and for whites (Z = 0):

Y ' = BO + BlX + Error.

Thus, one can test the slopes and intercepts of two groups.thin a single podel.lq 1), Humphreys (1973) has stated that the critical comparisons..to be made are tests of equal slopes, andntercepts. Following his suggestion the two hypotheses tested were from the s regression e4uation.

1. The two regression lines are parallel; that is, Hp: B, = 0, i.e., X is proportional from group to group. When B = 0, the slope for blacks be- comes the slope for whites. 4

2. The two regression lines are coincident; i.e., Ho: B2 = B3 0, i.e., X is identical after the differences in groups are removed. When B2 = B3 = 0, the model for blacks reduces to the model for whites.

To test hypothesis 1 the following F statistic was computed:

SS (X,Z, XZ) - SS (XZ) regression regression ' F(XZIX,Z) MS (XZ XZ) residual ' '

( 3 To test hypothesis 2:

...... [SS X,Z, XZ )- SS (X)J /2df regression regression F( X2 ,ZIX) = Ms (XZ XZ) residual ' '

If the slopes were significantly different, no test ofcoincidence of lines was

conducted. ,

2 The multiple Rfor (X, Z, XZ) and its various subsets may also.beused, as it represents the proportion of total sums of squares(Overall and Spiegel, 1969).

.4,1

'Ow

i 6 C

...

28 TABLE 13-1. Estimates of Simple Regression Parameters - Ethnic Comparisons

Raw Intercept Coefficient Standardized S.E. of B Groups EPT-Essay (13 ) (B ) Coefficient 0 1 1

Asian TSWE 2.96 .1011 .547 ..t. .0063 SAT-V 2.88 %0101 .517 .0007 EPT-R -7.72 .0997 .574 .0058 EPT-SC -7.82 .1001 .589 .0056 EPT-L&O -6.97 .0937 .543 .0059 Black TSWE 2.59 .1100 .560 .0068 .0008 , SAT-V 2.30 .0120 ,.535 -EPT-R -5.77 .0856 .533 .0056 EPT-SC -6.92 .0933 .565 .:; .0056 EPT-L&O -5.47 .0844 .556 .0052 Hispanic TSWE 2.56 .1101 .533 10083 1, SAT-V 2.32 .0115 .503 .0009 EPT-R -7.47 .0966 .495 .0081 EPT-SC -8.49 .1037 .551 .0075 EPT-L&O -6.67 .0913 .494 .0075 White TSVE 4.36 .0728 .370. 0025 4. 7 SAT-V 4.74 .0065 .322 .0003 EPT-R -4.05 .0765 .297 .0034 T-SC -5.47 .0857 .354 .0031

PT -L &O , -3.87 .0754 .290 .0034

TABLE B-2. Estimates of Simple Regression Parameters -Sex Comparisons

Raw Coefficient Standardized Intercept 1

Groups EPT- ssay on: (B ) ) Coefficient S.E. of BB1 0 1

Females TSWE 3.72 .0907 .500 .0020 SAT-V 3.76 .0092 .487 .0002 EPT- -6.70 .0953 .490 .0022 EPT-SC -7.03 .0972 .506 .0022 EPT -L&0 -6.28 .0926 .486 .0022 Males TSWE 3.20 .0894 .480 .0024 SAT-V 3.16 .0088 .444 .0003 EPT-R -6.50 .0895 ?4,.14 .0027 EPT-SC -7.69 .0975 .485 .0026 EPT-L&O -6.53 .0897 .446 .0026

29 3